How CodePilot Works
End-to-end architecture overview — from repository connection to AI-powered answers.
CodePilot follows a pipeline architecture with three main stages: ingestion, indexing, and retrieval. This page provides a high-level view of the entire system.
System Architecture
Click any step to explore • Press ↑↓ to navigate
Request Flow
User Connects a Repository
A user authenticates via GitHub OAuth, installs the CodePilot GitHub App, and selects a repository to analyze. The frontend sends this request to the Express API.
API Enqueues an Ingestion Job
The API creates a repository record in PostgreSQL and enqueues an INGEST job to BullMQ via Redis. The job contains the repository metadata and GitHub installation token.
// API enqueues the job
GithubRepositoryProducer.ingest({
repositoryId: repo.id,
installationId: installation.id,
repoFullName: "owner/repo-name",
defaultBranch: "main",
});Worker Processes the Job
The BullMQ worker picks up the job and begins the ingestion pipeline:
- Clone — Uses
simple-gitwith a GitHub installation token - Scan — Walks the file tree, filtering by supported languages
- Parse — Uses
ts-morphfor AST analysis on TypeScript/JavaScript files - Chunk — Splits code into semantic chunks (functions, classes, components, hooks, exports)
- Embed — Generates 768-dimensional vectors via Ollama's
nomic-embed-text - Store — Inserts vectors into PostgreSQL using pgvector with HNSW indexing
User Queries the Codebase
When a user asks a question in the chat interface:
- The query text is embedded using the same
nomic-embed-textmodel - pgvector performs cosine similarity search against all stored code chunks
- The top-k most relevant chunks are retrieved
- Retrieved chunks + the original query are composed into a prompt
- The prompt is sent to
qwen2.5-coder:3bvia Ollama - The LLM generates a context-aware response grounded in the actual code
Ingestion Pipeline Detail
GitHub Repo → Clone → Scan Files → Parse AST → Chunk Code → Embed → Store in pgvector| Stage | Tool | Output |
|---|---|---|
| Clone | simple-git + GitHub App token | Local repository copy |
| Scan | Custom file walker | Filtered file list (TS, JS, MD, JSON) |
| Parse | ts-morph (AST analysis) | Structured code elements |
| Chunk | AST-aware chunker + fallback chunker | Semantic code segments |
| Embed | Ollama nomic-embed-text | 768-dimensional float vectors |
| Store | Prisma + pgvector | Indexed vectors in PostgreSQL |
Incremental Sync
When a push webhook event is received from GitHub, CodePilot performs incremental ingestion — only re-processing files that changed in the commit:
- GitHub sends a
pushwebhook to the API - The API compares the commit diff to identify changed files
- An
INCREMENTAL_INGESTjob is enqueued - The worker re-processes only the changed files, updating their chunks and embeddings
- Deleted files have their chunks and embeddings removed
This keeps the vector index up to date without re-ingesting the entire repository.
Repository Intelligence
Beyond chat, CodePilot generates structured intelligence about repositories:
- Overview — Auto-generated summary, architecture description, tech stack, main modules, design patterns, and entry points (via
qwen2.5:7b) - Health Score — AI-generated analysis of code quality, including strengths, issues, and scored recommendations