Set Up LEANN for Private Local RAG on macOS

How to get yichuan-w/LEANN running on macOS in 5 minutes. Zero cloud, zero API keys, just your documents and a terminal.

⬅️ Back to tutorials

LEANN is a vector database for RAG that does everything on your machine. No cloud, no API keys, none of your documents leaving your laptop. It hit 11.8k stars on GitHub and its MLsys2026 paper claims 97% storage savings through graph-based selective recomputation. That got my attention.

It indexes 30+ file types, runs on CPU, and plugs into Claude Code through MCP.

What You Need

  • macOS (any recent version, Intel or Apple Silicon)
  • 10 minutes
  • Terminal access

Step 1: Install Prerequisites

If you havent already, install Xcode Command Line Tools for git and the C++ build chain:

xcode-select --install

Then install uv (the Python package manager). The fastest way:

curl -LsSf https://astral.sh/uv/install.sh | sh

Restart your terminal or source your shell config afterward.

Step 2: Install LEANN 

uv tool install leann-core --with leann

This installs the leann CLI globally. Verify it worked:

leann --help

You should see available subcommands for building indexes, searching, and managing projects.

Step 3: Index Your First Project

Navigate to any codebase or document folder and run:

leann build my-first-index --docs $(git ls-files)

LEANN scans your files, generates embeddings, and stores the index locally in a .leann/ directory. The first build takes a couple minutes as it downloads a sentence-transformer model.

If youre not in a git repo, you can pass paths directly:

leann build my-notes --docs ./notes/ ./documents/ --embedding-mode sentence-transformers --embedding-model all-MiniLM-L6-v2

Step 4: Query Your Index 

leann query my-first-index "your question here"

LEANN returns the most semantically relevant documents or code snippets. No internet required.

Step 5 (Optional): Wire It Into Claude Code

The MCP integration is what makes this useful. LEANN exposes an MCP server Claude Code can call right from your chat:

claude mcp add --scope user leann-server -- leann_mcp

Now when you run claude in that project, it can search your indexed codebase. Try asking it something like “find the authentication logic” or “show me how the API client is initialized”.

List available indexes from inside Claude Code with:

leann_list

And search with:

leann_search

Pro Tips

  • Add --no-recompute to your build command to materialize all tensors on disk instead of recomputing. Faster queries, more disk usage.
  • Use --backend hnsw for approximate nearest neighbor search on large indexes.
  • Indexes live in .leann/ per project and a global registry at ~/.leann/projects.json.
  • To update an index after changing files, just run leann build again with the same name.

Cleanup 

claude mcp remove leann-server
uv tool uninstall leann-core

If you want your RAG completely offline with no data leaving your machine, LEANN is the simplest path I have found. No Docker, no GPU, no cloud credits.

Crepi il lupo! 🐺