NTT Researcher Appointed Maintainer (WebGPU) of llama.cpp, a Widely Used Local AI Inference Engine

Masashi Yoshimura, a researcher at NTT Open Source Software Center, has been appointed as a maintainer of llama.cpp, an open-source local AI inference engine.

llama.cpp is a lightweight, high-performance inference engine that makes it possible to run large language models (LLMs) on devices such as laptops and smartphones. Because it allows LLMs to be used on local devices without relying on the cloud, it has become an important foundational technology in areas where privacy, security, and cost reduction matter. It is also used as the core technology of Ollama, one of the most widely used AI tools in the world, and is used within NTT Group companies as well.

llama.cpp includes WebGPU support, which allows LLMs to run at high speed in web browsers such as Chrome and Safari. He was appointed in recognition of his significant contributions to this area, including speeding up matrix multiplication^*1 and adding support for MoE models^*2. With this technology, users can run LLMs on their own devices simply by opening a browser. This is expected to lead to LLM environments that are both easy to use and highly secure.

^*1Matrix multiplication: The multiplication of multidimensional numerical arrays, which accounts for most of the computation in LLM inference. Speeding it up is key to improving inference speed.

^*2MoE (Mixture-of-Experts) model: A type of LLM architecture. By selectively using only the parts that are needed, it runs fast while maintaining high performance. It is a common design that is also used in state-of-the-art models.