little-coder

A coding agent tuned for small local models, built on top of pi.

The research story behind all this — why scaffold–model fit matters, how a 9.7 B Qwen beat frontier entries on Aider Polyglot, and what the load-bearing mechanisms actually do — is written up on Substack: Honey, I Shrunk the Coding Agent. Start there if you want the "why"; stay here for the "how".

How it relates to pi

pi is the minimal substrate — agent loop, multi-provider API, TUI, session tree, compaction, extension model. Four built-in tools (read / write / edit / bash) and a ~1000-token system prompt.

little-coder is pi + 20 extensions + 30 skill markdown files + a Python benchmark harness. It doesn't fork pi or shadow its CLI — pi is a plain dependency in package.json, and everything little-coder-specific lives under .pi/extensions/, skills/, and benchmarks/. The launcher runs pi with --no-extensions and wires in exactly the bundled set, so you add your own extension by dropping a directory into .pi/extensions/ (or passing little-coder -e /path/to/ext/index.ts at launch) and remove one of ours by deleting its directory. Note this also means a globally pi install'd package won't load inside little-coder — pi install registers into pi's settings, which --no-extensions skips.

If you've never used pi, it's useful to skim pi.dev first — the rest of this doc assumes pi's model of --agent-import-path, --mode rpc, and .pi/extensions/ auto-discovery.

Install

One-line install (Node.js 22.19+ required):

curl -fsSL https://raw.githubusercontent.com/itayinbarr/little-coder/main/install.sh | bash

Or with npm directly:

npm install -g little-coder

Or with bun:

bun add -g little-coder

That's the whole install. No clone, no npm install in a workspace, no PATH fiddling. little-coder is now on your PATH and works from any directory.

Note for bun add -g users. The launcher (bin/little-coder.mjs) is a Node.js script with #!/usr/bin/env node at the top, so Node ≥ 22.19 still has to be on your PATH for the binary to start — bun is fine for installing/updating the package, but the runtime is Node. If you want a fully node-less setup, replace the shebang in $(bun pm bin -g)/little-coder with #!/usr/bin/env bun.

Run

cd ~/your-project
little-coder --model llamacpp/qwen3.6-35b-a3b

This is the canonical setup little-coder is tuned for: a local llama.cpp server hosting Qwen3.6-35B-A3B. See Local model setup (optional) below for how to serve it.

Cloud models work the same way:

little-coder --model anthropic/claude-haiku-4-5
little-coder --model openai/gpt-4o-mini "What does this codebase do?"
little-coder --model ollama/qwen3.5             # local Ollama
little-coder --model lmstudio/local-model       # local LM Studio (whatever model you have loaded)
little-coder --list-models                      # see everything pi knows about

The agent uses the directory you launched it from as its working directory — Read / Write / Edit / Bash operate on your project, not on little-coder's install path.

Interactive features

• Plan Mode — press alt+p to toggle (a ◆ PLAN MODE indicator shows below the input). Submit a request and little-coder researches it with sub-coders, asks you 1-3 clarifying questions (each with suggested answers and a free-text option), then writes a plan in the chat instead of editing anything. Esc cancels a plan mid-run. (shift+tab stays pi's thinking-level cycle.)
• Prompt history — from an empty input, ↑ recalls your recent prompts (most-recent first), ↓ walks forward. History persists across sessions, so a fresh session can recall prompts from earlier runs.
• Sub-coders (dispatch) — little-coder can spawn isolated child sessions to research a question (read the repo + browse online, read-only) and report back concisely, without cluttering the main conversation. A live panel above the input tracks them. Tune parallelism with LITTLE_CODER_SUBCODER_CONCURRENCY (default 2).
• Sessions — each session is auto-named from your first prompt (rename with /name) and shown in the terminal tab title. Use /resume to list and reopen past sessions for the current directory.
• Read-before-edit — editing a file requires reading it first, so edits match the file's exact current text.
• Third-party extensions (LITTLE_CODER_EXTRA_EXTENSIONS) — path-delimited list (: on POSIX, ; on Windows) of extension paths to layer on top of the bundled set. Each entry can be a direct file (e.g. a pi-ponytail-style extensions/ponytail.js) or a directory containing index.ts / index.js. ~/ is expanded; missing paths log a warning and are skipped. Survives upgrades, no patching the installed package. Example: LITTLE_CODER_EXTRA_EXTENSIONS=~/.local/lib/node_modules/pi-ponytail/extensions/ponytail.js little-coder. (Single-file extensions can still use little-coder -e <path> for one-off loads.)

For local providers (llama.cpp, Ollama, LM Studio) pi expects some value in the API-key env even though local servers ignore it:

export LLAMACPP_API_KEY=noop
export OLLAMA_API_KEY=noop
export LMSTUDIO_API_KEY=noop

LLAMACPP_BASE_URL, OLLAMA_BASE_URL, and LMSTUDIO_BASE_URL override the defaults (http://127.0.0.1:8888/v1, http://127.0.0.1:11434/v1, http://127.0.0.1:1234/v1).

For cloud providers, set the standard env (ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.) and pi will discover it.

Local model setup (optional)

Skip this section if you're using a cloud model.

Option A — llama.cpp (fastest for local; supports Qwen3.6-35B-A3B MoE):

# One-time: build llama.cpp with CUDA (sm_XXX = your GPU arch; Blackwell = 120)
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_CURL=ON
cmake --build build --config Release -j

# Fetch the model GGUF and the matching vision projector.
# The mmproj (~900 MB) is what lets the model see attached screenshots.
pip install -U "huggingface_hub[cli]"
hf download unsloth/Qwen3.6-35B-A3B-GGUF Qwen3.6-35B-A3B-UD-Q4_K_M.gguf --local-dir ~/models
hf download unsloth/Qwen3.6-35B-A3B-GGUF mmproj-F16.gguf            --local-dir ~/models

# Serve it (MoE trick: experts in RAM, attention on GPU → 22 GB model on 8 GB VRAM)
build/bin/llama-server -m ~/models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
   --mmproj ~/models/mmproj-F16.gguf \
   --host 127.0.0.1 --port 8888 --jinja \
   -c 16384 -ngl 99 --n-cpu-moe 999 --flash-attn on

If you only need text and want to skip the projector download, drop the second hf download line and the --mmproj flag — little-coder still works text-only, but the TUI's image attachment will be rejected by the server with a 4xx.

Context window. -c sets the server's context (-c 16384 = 16K above — a conservative default for 8 GB VRAM). little-coder auto-detects the live n_ctx from llama.cpp's /props at startup and registers the model with it, so whatever you pass to -c is what the TUI shows and budgets against — no models.json edit needed. To run larger, relaunch the server with e.g. -c 131072 (128K) or -c 262144 (256K); the KV cache grows with it, so size it to your RAM/VRAM. (--list-models reflects the detected window.)

Option B — Ollama (simpler, but slower on MoE):

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3.5        # 9.7B — the paper's model
# or: ollama pull qwen3.6:35b-a3b

Option C — LM Studio (GUI; OpenAI-compatible server on port 1234):

1. Install LM Studio and download a model (e.g. Qwen3.6 35B A3B GGUF).
2. Open the Developer / Local Server tab, load the model, and click Start Server (default http://127.0.0.1:1234).
3. Run little-coder:

   export LMSTUDIO_API_KEY=noop
   little-coder --model lmstudio/local-model

   The shipped lmstudio/local-model id routes to whatever model LM Studio currently has loaded — no extra config needed for the single-model case. If you serve on a non-default port, set LMSTUDIO_BASE_URL=http://127.0.0.1:<port>/v1. To target a specific model when you have several loaded, add an entry to ~/.config/little-coder/models.json (see Configuring models below).

Serving from another machine on your LAN. Each provider's *_BASE_URL env var accepts any host, not just 127.0.0.1, so you can run inference on a beefier box and connect from a laptop or another device on the same WiFi.

On the server (the box with the GPU):

• llama.cpp: start llama-server with --host 0.0.0.0 (or your specific LAN interface) instead of 127.0.0.1. Everything else from Option A unchanged.
• LM Studio: in the Server tab, enable Serve on local network so it binds 0.0.0.0:1234 instead of 127.0.0.1:1234.
• Ollama: OLLAMA_HOST=0.0.0.0:11434 ollama serve (or set OLLAMA_HOST=0.0.0.0 in the user systemd unit).
• If ufw / firewalld is active, allow your LAN subnet to the relevant port (e.g. sudo ufw allow from 192.168.0.0/16 to any port 8888 proto tcp).
• Find the LAN IP with hostname -I (Linux) or ipconfig getifaddr en0 (macOS).

On the client (the machine running little-coder):

# Pick the env vars matching whichever provider is running on the server
export LLAMACPP_API_KEY=noop
export LLAMACPP_BASE_URL=http://<server-lan-ip>:8888/v1

# Sanity check reachability before launching the agent
curl -s http://<server-lan-ip>:8888/v1/models | head

little-coder --model llamacpp/qwen3.6-35b-a3b

The streaming chat-completions adapter works over a local network the same way it does over loopback — no client code change, no proxy needed. The per-model profile in .pi/settings.json (context/thinking-budget/temperature) still applies because it's keyed by <provider>/<model-id>, which the client picks regardless of where the server lives.

All small-model-specific extensions auto-disable for large/cloud models so they don't interfere.

Configuring models

The shipped model list lives in models.json at the package root. The llama-cpp-provider extension reads it at startup and registers each provider via pi's registerProvider(). Editing this file in your global install does take effect — but it's overwritten on npm install -g little-coder@latest, so for anything you want to keep, use a user override file instead.

User override resolution (first match wins):

1. $LITTLE_CODER_MODELS_FILE — explicit path, useful for ad-hoc tests.
2. $XDG_CONFIG_HOME/little-coder/models.json
3. ~/.config/little-coder/models.json

Merge semantics: each top-level provider key in your override file fully replaces the same key in the shipped models.json. Providers only in your file are added; providers only in the shipped file are kept. (We don't deep-merge per-model fields — you redeclare the whole provider entry, which avoids "your override silently inherited new fields from a future package release" surprises.)

Example — switch the llama.cpp port and bump qwen3.6-35b-a3b to a 150K context, leave ollama untouched:

{
  "providers": {
    "llamacpp": {
      "api": "openai-completions",
      "baseUrl": "http://127.0.0.1:1234/v1",
      "apiKey": "LLAMACPP_API_KEY",
      "models": [
        {
          "id": "qwen3.6-35b-a3b",
          "name": "Qwen3.6-35B-A3B (local llama.cpp, 150K)",
          "reasoning": true,
          "input": ["text"],
          "contextWindow": 150000,
          "maxTokens": 4096,
          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
        }
      ]
    }
  }
}

Then verify with little-coder --list-models — you should see your overridden entry.

LLAMACPP_BASE_URL, OLLAMA_BASE_URL, and LMSTUDIO_BASE_URL env vars still beat both files for those three providers.

Any OpenAI-compatible server (e.g. MLX / omlx)

little-coder registers providers from models.json — it doesn't pick up pi's standalone "picker" extensions. So a server isn't added by installing its pi picker; you add it by declaring a provider. Any OpenAI-compatible endpoint works this way, including Apple's MLX server (mlx_lm.server, often surfaced as omlx). Drop this into ~/.config/little-coder/models.json and pick it with little-coder --model omlx/<id>:

{
  "providers": {
    "omlx": {
      "api": "openai-completions",
      "baseUrl": "http://127.0.0.1:8000/v1",
      "apiKey": "IGNORED",
      "models": [
        {
          "id": "Qwen3-32B-4bit",
          "name": "Qwen3.6-35B-A3B (local omlx, 150K)",
          "reasoning": true,
          "input": ["text"],
          "contextWindow": 150000,
          "maxTokens": 4096,
          "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
        }
      ]
    }
  }
}

Set id to whatever model your server reports, and baseUrl to its /v1 endpoint. Verify with little-coder --list-models.

.pi/settings.json is a separate concern: it controls per-model profiles (context_limit, thinking_budget, temperature, benchmark_overrides) referenced by the <provider>/<id> key. Profiles don't register or describe models — they only tune how little-coder runs against models that are already registered.

Permissions

little-coder gates Bash tool calls against a built-in safe-prefix whitelist (ls, cat, head, tail, git log/status/diff, find, grep, cp, mv, mkdir, touch, etc.) before pi's own confirmation flow ever sees them. rm and sudo are intentionally not on the list — add them via LITTLE_CODER_BASH_ALLOW per deployment if you really need them.

Two env vars control the gate:

Examples:

# Add 'make' (with word-boundary) and 'docker compose ps' on top of the defaults
export LITTLE_CODER_BASH_ALLOW="make ,docker compose ps"

# Skip the gate entirely (use this only inside controlled environments)
export LITTLE_CODER_PERMISSION_MODE=accept-all

Write/Edit confirmations are pi's responsibility; little-coder doesn't interce