Edit

Share via

Get started with Foundry Local

Important

  • Foundry Local is available in preview. Public preview releases provide early access to features that are in active deployment.
  • Features, approaches, and processes can change or have limited capabilities, before General Availability (GA).

This guide walks you through setting up Foundry Local to run AI models on your device.

Prerequisites

Your system must meet the following requirements to run Foundry Local:

  • Operating System: Windows 10 (x64), Windows 11 (x64/ARM), Windows Server 2025, macOS.
  • Hardware: Minimum 8GB RAM, 3GB free disk space. Recommended 16GB RAM, 15GB free disk space.
  • Network: Internet connection for initial model download (optional for offline use)
  • Acceleration (optional): NVIDIA GPU (2,000 series or newer), AMD GPU (6,000 series or newer), Qualcomm Snapdragon X Elite (8GB or more of memory), or Apple silicon.

Also, ensure you have administrative privileges to install software on your device.

Quickstart

Get started with Foundry Local quickly:

  1. Install Foundry Local

    • Windows: Open a terminal and run the following command: 
      winget install Microsoft.FoundryLocal
    • macOS: Open a terminal and run the following command: 
      brew tap microsoft/foundrylocal
brew install foundrylocal

    Alternatively, you can download the installer from the Foundry Local GitHub repository.

  2. Run your first model Open a terminal window and run the following command to run a model:

    foundry model run phi-3.5-mini

    The model downloads - which can take a few minutes, depending on your internet speed - and the model runs. Once the model is running, you can interact with it using the command line interface (CLI). For example, you can ask:

    Why is the sky blue?

    You should see a response from the model in the terminal: Screenshot of output from foundry local run command.

Tip

You can replace phi-3.5-mini with any model name from the catalog (see foundry model list for available models). Foundry Local downloads the model variant that best matches your system's hardware and software configuration. For example, if you have an NVIDIA GPU, it downloads the CUDA version of the model. If you have a Qualcomm NPU, it downloads the NPU variant. If you have no GPU or NPU, it downloads the CPU version.

Explore commands

The Foundry CLI organizes commands into these main categories:

  • Model: Commands for managing and running models.
  • Service: Commands for managing the Foundry Local service.
  • Cache: Commands for managing the local model cache (downloaded models on local disk).

View all available commands with:

foundry --help

To view available model commands, run:

foundry model --help

To view available service commands, run:

foundry service --help

To view available cache commands, run:

foundry cache --help

Tip

For a complete guide to all CLI commands and their usage, see the Foundry Local CLI Reference.

Next steps