Local AI is a free and open-source tool that allows you to experiment with AI models offline, in private, without the need for a GPU. It is a native app that simplifies the entire process and is designed to be memory efficient and compact. With Local AI, you can perform CPU inferencing with adaptive threading and GGML quantization. It also offers features like model management, digest verification, and inferencing server. Upcoming features include GPU inferencing, parallel sessions, and custom sorting and searching. The tool is crafted with love and is available under the GPLv3 license.
Features
- CPU inferencing with adaptive threading
- GGML quantization q4, 5.1, 8, f16
- Model management with resumable, concurrent downloader and usage-based sorting
- Digest verification with BLAKE3 and SHA256 digest compute feature
- Local streaming server for AI inferencing with quick inference UI and remote vocabulary
Use Cases
- Experimenting with AI models offline
- Managing and organizing AI models
- Verifying the integrity of downloaded models
- Running AI inferencing on a local server
Suited For
- Researchers
- Data scientists
- AI enthusiasts
- Developers