CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases

AI-Powered Repository Documentation Generation • Multi-Language Support • Architecture-Aware Analysis

Generate holistic, structured documentation for large-scale codebases • Cross-module interactions • Visual artifacts and diagrams

Quick Start • CLI Commands • Output Structure • Repo Docs • Paper

📚 CodeWiki documents itself — browse the generated documentation for this repository at CodeWiki docs.

Quick Start

1. Install CodeWiki

# Install from source
pip install git+https://github.com/FSoft-AI4Code/CodeWiki.git

# Verify installation
codewiki --version

2. Configure Your Environment

CodeWiki supports multiple LLM providers: OpenAI-compatible, Atlas Cloud, Anthropic, AWS Bedrock, Azure OpenAI, plus subscription mode via Claude Code and Codex CLIs (no API key required).

# OpenAI-compatible
codewiki config set \
  --provider openai-compatible \
  --api-key YOUR_API_KEY \
  --base-url https://api.anthropic.com \
  --main-model claude-sonnet-4 \
  --cluster-model claude-sonnet-4 \
  --fallback-model glm-4p5

# Atlas Cloud — base URL auto-set to https://api.atlascloud.ai/v1;
# API key read from $ATLASCLOUD_API_KEY when --api-key is omitted.
codewiki config set \
  --provider atlas-cloud \
  --main-model anthropic/claude-sonnet-4.6 \
  --cluster-model anthropic/claude-sonnet-4.6 \
  --fallback-model zai-org/GLM-4.6

# Anthropic
codewiki config set \
  --provider anthropic \
  --api-key YOUR_API_KEY \
  --base-url https://api.anthropic.com \
  --main-model claude-sonnet-4 \
  --cluster-model claude-sonnet-4 \
  --fallback-model glm-4p5

# Azure OpenAI
codewiki config set \
  --provider azure-openai \
  --api-key YOUR_AZURE_KEY \
  --base-url https://YOUR_RESOURCE.openai.azure.com \
  --azure-deployment YOUR_DEPLOYMENT \
  --main-model gpt-4o \
  --cluster-model gpt-4o

# AWS Bedrock
codewiki config set \
  --provider bedrock \
  --aws-region us-east-1 \
  --main-model anthropic.claude-sonnet-4-v2:0 \
  --cluster-model anthropic.claude-sonnet-4-v2:0

# Subscription mode (Claude Code) — uses your existing Claude OAuth login.
# Install the Claude Code CLI and run `claude login` first.
codewiki config set \
  --provider claude-code \
  --main-model claude-sonnet-4-6 \
  --cluster-model claude-sonnet-4-6

# Subscription mode (Codex) — uses your existing Codex CLI login.
# Install the Codex CLI and run `codex login` first.
codewiki config set \
  --provider codex \
  --main-model gpt-5.4 \
  --cluster-model gpt-5.5

About Atlas Cloud. Atlas Cloud is a full-modal AI inference platform that exposes LLM, image, and video models (300+) behind a single OpenAI-compatible API, so it works with CodeWiki out of the box. Browse model IDs at the models endpoint and pick a strong coding model for --main-model / --cluster-model; their coding plan offers budget-friendly API access.

Subscription mode routes every LLM call through the local claude / codex CLI binary (via the caw library), so you can run CodeWiki on a Claude Pro/Max or Codex subscription instead of paying per-token API usage. Claude Code's built-in Write/Edit/Bash tools are disabled inside CodeWiki's agent loop so documentation writes still go through CodeWiki's Mermaid-validating editor.

Note on model names. In subscription mode the model string is forwarded directly to claude --model / codex --model, so use the bare CLI model name (e.g. gpt-5.4, claude-sonnet-4-6) — not the litellm-style openai/… or anthropic/… prefix used by openai-compatible. If you previously ran with openai-compatible, re-run config set for both --main-model and --cluster-model to clear any stale prefixes; config set only updates the keys you pass.

3. Generate Documentation

# Navigate to your project
cd /path/to/your/project

# Generate documentation
codewiki generate

# Generate with HTML viewer for GitHub Pages
codewiki generate --github-pages --create-branch

That's it! Your documentation will be generated in ./docs/ with comprehensive repository-level analysis.

Usage Example

What is CodeWiki?

CodeWiki is an open-source framework for automated repository-level documentation across nine programming languages. It generates holistic, architecture-aware documentation that captures not only individual functions but also their cross-file, cross-module, and system-level interactions.

Key Innovations

Supported Languages

🐍 Python • ☕ Java • 🟨 JavaScript • 🔷 TypeScript • ⚙️ C • 🔧 C++ • 🪟 C# • 🎯 Kotlin • 🐘 PHP

CLI Commands

Configuration Management

# Set up your API configuration
codewiki config set \
  --api-key <your-api-key> \
  --base-url <provider-url> \
  --main-model <model-name> \
  --cluster-model <model-name> \
  --fallback-model <model-name>

# Configure max token settings
codewiki config set --max-tokens 32768 --max-token-per-module 36369 --max-token-per-leaf-module 16000

# Configure max depth for hierarchical decomposition
codewiki config set --max-depth 3

# Show current configuration
codewiki config show

# Validate your configuration
codewiki config validate

Documentation Generation

# Basic generation
codewiki generate

# Custom output directory
codewiki generate --output ./documentation

# Create git branch for documentation
codewiki generate --create-branch

# Generate HTML viewer for GitHub Pages
codewiki generate --github-pages

# Enable verbose logging
codewiki generate --verbose

# Full-featured generation
codewiki generate --create-branch --github-pages --verbose

# Incremental update (only regenerate changed modules since last run)
codewiki generate --update

# Incremental update using a specific commit hash to compare against (useful in CI/CD or squashed PRs)
# This overrides the stored commit hash in metadata.json and implicitly enables --update
codewiki generate --compare-to <commit-hash>

Customization Options

CodeWiki supports customization for language-specific projects and documentation styles:

# C# project: only analyze .cs files, exclude test directories
codewiki generate --include "*.cs" --exclude "Tests,Specs,*.test.cs"

# Focus on specific modules with architecture-style docs
codewiki generate --focus "src/core,src/api" --doc-type architecture

# Add custom instructions for the AI agent
codewiki generate --instructions "Focus on public APIs and include usage examples"

Pattern Behavior (Important!)

• --include: When specified, ONLY these patterns are used (replaces defaults completely)

  • Example: --include "*.cs" will analyze ONLY .cs files
  • If omitted, all supported file types are analyzed
  • Supports glob patterns: *.py, src/**/*.ts, *.{js,jsx}

• --exclude: When specified, patterns are MERGED with default ignore patterns

  • Example: --exclude "Tests,Specs" will exclude these directories AND still exclude .git, __pycache__, node_modules, etc.
  • Default patterns include: .git, node_modules, __pycache__, *.pyc, bin/, dist/, and many more
  • Supports multiple formats:
    • Exact names: Tests, .env, config.local
    • Glob patterns: *.test.js, *_test.py, *.min.*
    • Directory patterns: build/, dist/, coverage/

Setting Persistent Defaults

Save your preferred settings as defaults:

# Set include patterns for C# projects
codewiki config agent --include "*.cs"

# Exclude test projects by default (merged with default excludes)
codewiki config agent --exclude "Tests,Specs,*.test.cs"

# Set focus modules
codewiki config agent --focus "src/core,src/api"

# Set default documentation type
codewiki config agent --doc-type architecture

# View current agent settings
codewiki config agent

# Clear all agent settings
codewiki config agent --clear

Token Settings

CodeWiki allows you to configure maximum token limits for LLM calls. This is useful for:

• Adapting to different model context windows
• Controlling costs by limiting response sizes
• Optimizing for faster response times

# Set max tokens for LLM responses (default: 32768)
codewiki config set --max-tokens 16384

# Set max tokens for module clustering (default: 36369)
codewiki config set --max-token-per-module 40000

# Set max tokens for leaf modules (default: 16000)
codewiki config set --max-token-per-leaf-module 20000

# Set max depth for hierarchical decomposition (default: 2)
codewiki config set --max-depth 3

# Override at runtime for a single generation
codewiki generate --max-tokens 16384 --max-token-per-module 40000 --max-depth 3

Configuration Storage

• API keys: Securely stored in system keychain (macOS Keychain, Windows Credential Manager, Linux Secret Service). Falls back to ~/.codewiki/credentials.json in headless/container environments. Set CODEWIKI_NO_KEYRING=1 to force file-based storage.
• Settings & Agent Instructions: ~/.codewiki/config.json

Documentation Output

Generated documentation includes both textual descriptions and visual artifacts for comprehensive understanding.

Textual Documentation

• Repository overview with architecture guide
• Module-level documentation with API references
• Usage examples and implementation patterns
• Cross-module interaction analysis

Visual Artifacts

• System architecture diagrams (Mermaid)
• Data flow visualizations
• Dependency graphs and module relationships
• Sequence diagrams for complex interactions

Output Structure

./docs/
├── overview.md              # Repository overview (start here!)
├── module1.md               # Module documentation
├── module2.md               # Additional modules...
├── module_tree.json         # Hierarchical module structure
├── first_module_tree.json   # Initial clustering result
├── metadata.json            # Generation metadata
└── index.html               # Interactive viewer (with --github-pages)

See it in action: This repository's own docs are checked in under ./docs/ — open ./docs/index.html in a browser for the interactive viewer, or start from ./docs/overview.md.

Experimental Results

CodeWiki has been evaluated on CodeWikiBench, the first benchmark specifically designed for repository-level documentation quality assessment.

Performance by Language Category

Results on Representative Repositories

View comprehensive results: See paper for complete evaluation on 21 repositories spanning all supported languages.

How It Works

Architecture Overview

CodeWiki employs a three-stage process for comprehensive documentation generation:

1. Hierarchical Decomposition: Uses