foundrylocal.ai

Overview

Run AI models locally for private, low-latency on-device inference.

foundry local runs AI models locally with on-device inference to keep data on device and reduce cloud dependencies.Requires an Azure subscription and supports ONNX Runtime with CPU, GPU, and NPU hardware acceleration.

Provides an OpenAI-compatible API for integration with existing applications and developer workflows.Includes SDKs for Python, JavaScript, C#, and Rust plus a model hub with documentation and examples.

Targets developers, edge-device deployments, and enterprises needing data privacy, low-latency inference, and local control over models.Install via package managers (example: brew install microsoft/foundrylocal/foundrylocal) and run models with simple CLI commands (example: foundry model run qwen2.5-0.5b).Source code, releases, and community resources are available on GitHub; distributed under the MIT license.

Use Cases

• Create a privacy-first on-device AI assistant for customer support using Foundry Local's OpenAI-compatible API and Python/JS SDKs, delivering low-latency, hardware-accelerated responses on CPU/GPU/NPU so sensitive conversations never leave the device.
• Deploy real-time industrial anomaly detection and predictive maintenance on edge devices with Foundry Local's ONNX Runtime and CLI tools, leveraging the model hub and multi-language SDKs (C#/Rust/Python) for hardware-accelerated, low-latency inference and simplified rollout while keeping telemetry local.
• Create an offline-capable document OCR and semantic search solution for regulated enterprises using Foundry Local's model hub and SDKs to run transformer models on-device, enabling privacy-preserving inference, fast local indexing, and seamless integration into existing applications.

Who Is It For

• Machine learning engineers
• Edge computing specialists
• Software developers
• Data scientists
• Cloud architects