Code Graphs as Context Windows for LLM Code Assistants

Flat context windows fail on large codebases for a reason that has nothing to do with window size. Code is relational: a function's meaning depends on what calls it, what it calls, and what data flows through it, and none of that is visible when an agent just reads files top to bottom. Stuffing more tokens into a bigger window doesn't fix this, because the problem was never really about capacity. It's about shape. Code graphs exist to give that shape back to the model.

Start with what agents actually do without one. An agent opens a file, greps for a pattern, reads the results, opens another file, greps again. Each round costs tokens and, worse, doesn't accumulate anything. The agent finishes a ten-step exploration with the same structural understanding it would have gotten from step one, just more expensive.

Scale turns this from an annoyance into a wall. Context windows in the 128,000 to 1,000,000 token range sound generous until you set them against a real repository. Enterprise codebases commonly run 500,000 to 5 million lines, and 128,000 tokens covers something like 30,000 lines of code. That's a gap measured in orders of magnitude, and no amount of clever prompting closes a gap that size. The mismatch is structural.

That mismatch raises the invoice. Research on production agent workloads (arXiv:2603.27277) reports substantial per-task API costs on non-trivial repositories, with input tokens dominating the bill even when caching is in play. Every grep-and-read cycle re-sends context the model already saw, because nothing about a flat search accumulates state.

There's a quality deficit hiding under the cost, and it's arguably worse. Liu et al.'s 2024 "Lost in the Middle" study found LLM accuracy dropping more than 30% when the answer is in the middle of a long context rather than at the start or end. Chroma's "Context Rot" study followed up and found all 18 frontier models tested degrade as input length grows, with accuracy swinging 20 to 25 percentage points depending purely on where the relevant information happens to sit. Longer context doesn't just cost more, then. It actively makes the model less reliable. The naive fix of throwing more tokens at the problem works against itself.

What a code graph is, and the main forms it takes

The idea has a specific origin point. Yamaguchi, Golde, Arp, and Rieck introduced the Code Property Graph at IEEE S&P in 2014, first implemented in a tool called Joern, and the insight behind it was almost stubbornly simple: instead of picking one representation of a program, merge the three that security researchers and compiler engineers already used separately.

Those three are the Abstract Syntax Tree, which captures the syntactic structure of the code, the Control Flow Graph, which maps out every possible execution path, and the Program Dependence Graph, which tracks data and control dependencies between statements. Individually, each one answers a narrow question. Merged into a CPG, nodes become program constructs carrying typed attributes, and edges become labeled relations between them. What comes out the other side is directed, edge-labeled, and queryable in a way a text file simply is not. You can ask a CPG "what reaches this variable" in a way you cannot ask a folder of source files.

The agent-era version of this keeps the same backbone but reorients around a different customer. A CPG built for a security researcher and a code knowledge graph built for an LLM assistant share ancestry, but the assistant's graph is tuned for the questions an agent actually asks mid-task: show the real callers of this function, not the ones that just share a name. What's the blast radius if this signature changes? Those are graph traversal questions, not text search questions, and that distinction is the whole argument for building the graph.

Graph-based retrieval versus flat retrieval on standard benchmarks

The clearest number on record comes from RepoGraph. Adding it to GPT-4o on CrossCodeEval (Python) lifted code match exact-match from 10.5 to 28.7, and identifier match exact-match from 16.8 to 36.0, roughly 2.7× improvement in code match and 2.1× in identifier match. On SWE-bench, the same approach produced a substantial average relative improvement of 32.8% over a better prompt template alone. Those aren't marginal gains from a better prompt template. They come from a different retrieval mechanism entirely; RepoGraph skips whole files and instead extracts an ego-network, the relevant lines plus their direct structural neighbors, handing the model that subgraph.

GraphCoder tells a similar story from a different angle. Using coarse-to-fine retrieval over a code context graph, it beats baseline retrieval-augmented methods by +6.06 in code match and +6.23 in identifier match, and does it more efficiently than the baselines it's compared against. Efficiency and accuracy move together here, not against each other, even though those two often trade off in retrieval systems.

CodexGraph rounds out the picture. On CrossCodeEval Lite (Python), it hits 27.9% exact match and 67.98% edit similarity, against a best baseline exact match of 21.2%. On SWE-bench Lite it reaches 22.96% Pass@1, and on EvoCodeBench, 36.02% Pass@1. Three independent systems, three different benchmarks, and the same pattern each time: structural retrieval beats flat retrieval, and not by a small margin.

Token efficiency: what graph-native queries cost versus file exploration

Accuracy is one axis. Cost is the other, and it's where the tradeoff gets genuinely interesting rather than one-sided.

Codebase-Memory (arXiv:2603.27277) ran the head-to-head directly: 83% answer quality against a file-exploration agent's 92%, but at ten times fewer tokens and 2.1 times fewer tool calls, across 31 real-world repositories. That's a nine-point quality gap purchased for a substantial reduction in token usage. For most engineering budgets, that's not a close call. The gap isn't fixed, either. On graph-native queries specifically, such as hub detection and caller ranking, the questions a graph is built to answer directly, the graph-based agent matched or beat the file explorer on 19 of the 31 languages tested. The quality deficit occurs mainly on tasks the graph wasn't built for, and shrinks or disappears on the tasks it was.

The production version of this, codebase-memory-mcp from DeusData, extends the approach to 162 languages. Structural questions get answered with dramatically fewer tokens, the whole thing ships as a native executable with no separate language runtime to install. That last detail means no Python interpreter, no managed runtime, and no dependency chain to keep synced with the codebase it's indexing.

There's a second, more surprising data point on token efficiency, and it comes from an unexpected direction: visuals. The paper "LLM Agents Can See Code Repositories" (ASE '26, arXiv:2606.14061) found that giving agents a visualized structural graph alongside the normal text interface cut input token consumption by up to 26%, while holding issue-resolution accuracy steady or improving it. The mechanism is different from a queryable graph server, but the underlying signal is the same one running through this whole argument: structure, in whatever form it takes, compresses the context a model needs to do the job.

Static graphs built at index time versus dynamic graphs built during traversal

Two philosophies compete here, and both have real production use behind them.

The static approach parses the entire repository once, up front, into a persistent graph. Every query after that first pass is a sub-millisecond lookup against an already-built structure. Codebase-Memory and CodexGraph both follow this model, and it's the more common of the two: the work is front-loaded, the payoff is fast, cheap queries for the life of the index.

The dynamic approach, exemplified by DyCoder (built on DyRetriever, arXiv:2608.01927), skips the global parse. It initializes from one or more entry-point functions and expands a partial dependency graph incrementally, as the LLM actually traverses nodes during the task. No upfront indexing pass, no full-repository parse sitting in wait.

Why choose incremental over comprehensive? Because a full repository graph carries real maintenance overhead, in both build time and the ongoing cost of keeping the graph in sync as code changes. For a repository that changes constantly, or a task narrow enough that touching a small fraction of the codebase is all that's needed, building a complete graph upfront is wasted work. Dynamic expansion sidesteps that cost by only building the part of the graph the task actually touches.

The dynamic approach also targets a specific failure mode that similarity-based retrieval keeps running into: semantically similar functions aren't necessarily relevant functions, and retrieving them anyway can actively mislead generation. A function that looks similar in embedding space might have nothing to do with the actual call path the task depends on. Dynamic retrieval follows structural dependency instead of surface similarity, pulling in the full implementation of the function that's actually connected, so the model gets complete, relevant context rather than a plausible-looking distraction.

MCP and live, queryable context servers for agents

Graph-based code representation is not new. CPGs and CodeQL have been around for years, doing exactly this kind of relational analysis, and doing it well. What they haven't had, until recently, is a standard way to talk to an LLM agent. They required specialized databases and purpose-built query languages that were never designed with LLM consumption in mind. A standard protocol for connecting models to tools closes that gap: it's a standard interface that any compatible agent can call, which turns a graph sitting in a database into a live service the agent can query mid-task.

Codebase-Memory is the reference implementation worth knowing (arXiv:2603.27277). It exposes 14 typed structural query tools over MCP, covering things like call-path tracing, impact analysis, and hub detection, backed by a SQLite database and shipped as a single statically linked binary with zero runtime dependencies. Query latency comes in under a millisecond.

CodeGraphContext is a separate project doing similar work: an MCP server and CLI that indexes local code into a graph database and hands structured context to AI assistants on request. A VS Code extension went into alpha in May 2026, and the project logged roughly 31,000 PyPI downloads in the month before June 2026, a reasonable signal that developers are reaching for this outside of any single vendor's ecosystem.

codebadger takes a different angle. It's an open-source MCP server that integrates Joern's CPG engine directly with LLMs, but instead of making the model write raw CPG queries (a genuinely hard query language to get right), it exposes high-level tools: program slicing, taint tracking, dataflow analysis, semantic code navigation. The model asks for the operation it needs in plain terms, and codebadger translates that into the underlying graph query. It's been demonstrated navigating an 8,000-method codebase to audit memory safety patterns, a scale where reading files one at a time simply isn't a viable strategy anymore.

The AST-derived versus LLM-extracted graph debate

Not every code graph is built the same way, and how it's built is not a minor implementation detail. It changes what the graph can be trusted to say.

AST-derived graphs, built through static analysis, are deterministic: run the same code through the same parser twice and you get the same graph. Codebase-Memory follows this model explicitly, and the same general pattern underlies the other static-index approaches in this space. It's fast, it's reproducible, and it's precise about the thing it's precise about: syntax and structure that's actually present in the code. Its limit is exactly that boundary. A relationship that's conceptual rather than syntactic, two functions that implement the same business rule in different ways, say, won't show up as an edge, because there's no syntactic link for the parser to find.

LLM-extracted graphs go after that gap directly. RANGER, for instance, uses an LLM to generate natural-language descriptions of each code entity through a hierarchical, bottom-up process, which lets it surface conceptual links and latent relationships that never appear in an AST. An LLM can hallucinate an edge that doesn't actually exist, or misjudge a relationship that a strict parser would have gotten right.

That distinction stops being academic the moment the graph feeds a vulnerability scan or a patch generator. A hallucinated edge in that context isn't a small accuracy ding, it's the mechanism by which a model produces a confidently wrong answer about whether a codebase is safe.

One useful way to hold both approaches at once: a vector index finds code that looks similar, a graph follows relationships that are actually, verifiably there. Hybrid systems that pair an AST-derived graph for structural precision with embedding-based retrieval for semantic similarity may end up outperforming either technique running alone, though this is an open question in the field right now, not a settled one. Nobody should read the current state of the art as a final verdict either way.

Security and vulnerability detection as a concrete application of CPG-based graphs

Vulnerability detection is where the relational-context argument gets tested under real pressure, because a security bug is, definitionally, a relationship problem. It's untrusted input reaching a sensitive operation through some chain of function calls, and CPGs were built from the start to trace exactly that: taint flows, paths from untrusted sources to dangerous sinks, across function boundaries a flat text search has no way to follow.

codebadger's field results make the case concretely rather than theoretically. Working across an 8,000-method codebase, a scale where manual file-by-file review is simply not tractable, it audited memory safety patterns and turned up a previously unreported buffer overflow in libtiff through graph-guided analysis, not keyword search. In a separate case, it generated a correct patch for CVE-2025-6021, an integer overflow in libxml2, on the first attempt. No iteration, no back-and-forth refinement. The graph handed the model enough structural context up front that a right answer came out the first time.

Security analysis is arguably the hardest test this technology faces, precisely because it demands multi-hop reasoning across call chains and data flows where missing a single hop means missing the vulnerability. Flat retrieval that surfaces code because it looks similar to a known bug pattern, rather than code that sits on the actual call path, fails at the moment it matters most.

That's the relational-context thesis, proven out rather than argued for. The graph didn't hand the model more tokens. It handed the model the one connection, buried three function calls deep, that a keyword search would never have surfaced.