Tools and the Code Graph

Tool comparisons usually focus on the question “which one writes better code?” In a regulated institution that question matters, but it is not enough on its own; the questions of security, compliance and audit must be answered too. In this part I look at the tools through those questions.

In the second half I describe a job the tools do not do well today: seeing in advance what a change the agent makes will affect elsewhere in the system. In a large codebase neither the agent knows this nor does the reviewer always remember it. For that I built a code graph.

Five questions to ask first

• Data policy. Where is the submitted code processed, how long is it kept, is it used for training? And which clause of the contract says so? “We don’t use it for training” can be a marketing sentence or a contract clause; the two are not the same thing.
• Closed operation. Can the tool connect to a local model? Can the vendor’s model run in your data centre? Can it run on a fully closed network?
• Admin controls. Can the institution distribute settings the user cannot change locally? Can it decide centrally which tools and commands are allowed? Does it see usage data?
• Logging. Is every request and every tool call recorded? Where? Can the institution access that record?
• Licence model. Seat, token, or both? Not a technical question, but as we will see in part six, it is what really determines the usage policy.

Two questions added this year

I thought five questions were enough. 2026 added two more.

Can the agent override its own settings? In #56694 from part one, according to the user’s report, Claude Code edited the hook whitelist that was blocking it. In Codex, the broadest mode, danger-full-access, is open to the developer’s choice unless the admin closes it with requirements.toml. In Cursor, the MCPoison vulnerability showed that MCP approval was tied to the server’s name, not its content; Cursor fixed this in version 1.3. Individual holes are being closed, but the question is permanent: every place where the agent can touch its own boundary is a candidate vulnerability.

How controlled is the supply chain? The MCP servers, skills and packages the agent installs are now part of the code:

• Koi Security found in February 2026 that 341 of the 2,857 skills on ClawHub were malicious.
• The postmark-mcp server on npm was secretly sending a copy of every e-mail sent to a third address in September 2025.
• In a 2026 study of five frontier models, 4.6% to 6.1% of the package names the models suggested were made up. Of the 127 package names that all five models fabricated in the same way, 53 were still open for registration; that is, waiting for someone to publish a malicious package under that name.

That is why “can installation from outside the registry be disabled?” is now on the list.

The tools through these questions

I filled in this table in October 2026 by looking at the vendors’ own documentation. These products change almost every month; by the time you read this a few cells may already be out of date, so take a look yourself before deciding. I did not give scores either, because on these questions the difference between 4 and 5 tells you nothing: the tool can either do it, partly do it, or not do it at all. I did not put the two newly added questions in the table; their answers depend on the version more than on the tool.

The conclusion from the table is simple: no tool answers “yes” to all five questions at once. Cloud tools can be governed but are not closed; local models are closed but not governed. Once you accept this, tool selection stops being “which one?” and becomes “which one for which job?” That is the subject of parts four and five.

I also have to add a quality note on local models, because it does not show in the table. In October 2026 frontier models are in the 95–96% band on SWE-bench Verified. The best open models that fit on a single workstation GPU are about 20 points behind; Qwen3.6-27B, for example, is at 77%. Open models in the 400-billion-parameter class bring the gap down to 10 points, but they want multiple data-centre GPUs. Let me also note that most of these figures are the vendors’ own claims.

The shared view of those who use these models in daily work is this: “It does the job, but not at the level of the frontier models.” The gap opens most in refactors that touch multiple files and in long-running agent tasks. I will describe what this means in daily work in part four.

The code graph: where the tools fall short

Think of a system like a banking or insurance core: dozens of solutions, hundreds of projects, services calling each other, shared tables and end-of-day processes.

A developer changes a method. The agent sees that file, maybe that project. But it does not see another method in another solution that depends on it, or a batch job that runs at night. However large the context window, it does not see it, because it does not know which file it should pull into the window.

The code graph fills this gap: a cross-solution dependency graph extracted from the code itself.

How is it built? In .NET I use Roslyn. The compiler infrastructure gives a semantic model; that is, real symbol resolution, not text matching. In Java, JavaParser or the compiler API does the same job.

• Nodes: solution, project, namespace, class, method, table from the ORM mapping, batch job, HTTP endpoint, message queue.
• Edges: calls, inherits from, references project, reads or writes table, triggers batch job, serves endpoint.

Where you keep the graph depends on the size of the system. In a small system a simple table in a relational database is enough. When the number of solutions reaches dozens and the number of connections reaches millions, a graph database like Neo4j makes queries much easier. To keep the graph current I do two things: on every merge I update only the changed part, and at night I rebuild from scratch so that the small drifts accumulated during the day are cleaned up.

How is it used? I exposed the graph as an MCP server. That way, whichever tool is used, my own plugin, the agent running on the local model, or off-the-shelf agents, they all get the same answers to the same questions. The questions are:

• impact(symbol): If I change this symbol, which endpoints, batch jobs and tables are affected?
• callers(symbol, depth): Who calls this, up to how many levels?
• readers(table) / writers(table): Who reads this table, who writes it?
• path(a, b): Is there a dependency path between two symbols?
• owner(symbol): Which team owns this code? (from repo metadata)
• find_existing(intent): Is there already a method or service that does this job? The agent asks before writing and, if there is one, uses it.
• duplicates(block): Where are the copies of this block longer than five lines?
• connectivity(scope): How many calls per thousand lines? If it is falling, the code is not being connected, it is being copied.

The last three questions on the list, find_existing, duplicates and connectivity, gained a separate importance this year. The reason is GitClear’s 2026 report. Examining 623 million changes between 2023 and 2026, the report shows three things that correspond exactly to these three questions: repeated code blocks longer than five lines rose by 81%; the share of code moved by refactoring, that is, reused code, fell from 21% in 2022 to 3.8% in 2026; call density between functions fell by 35%. In short, agents copy existing code instead of reusing it.

When code is produced at this pace, eyeballing it in review is no longer enough. The problem is not only spotting the copy. It is impossible for a human to construct in their head, on every PR, where the code the agent wrote touches the system and whether it fits the existing architecture. You can read a few PRs a day carefully; you cannot read dozens.

So I moved the control to before review and tightened the rules. Before the agent starts writing it asks the graph with find_existing; if a method or service that does the same job exists, it must use it. After writing, duplicates and connectivity kick in: if a new copy has been created, or if the code has been added as a separate island next to the existing structure instead of connecting to it, the agent must fix this before opening the PR. That way the agent is prevented from the outset from straying outside the existing architecture and writing code outside the rules.

This is where the graph stopped being a review tool and became the agent’s working tool. It is also the part I put the most effort into.

What did it gain? Two things; one I expected, one I did not.

The expected gain was in impact analysis. Previously, to find what a change would affect, you had to ask someone who knew the system well and then search the code piece by piece; this sometimes took hours. With the graph this became a single query. More importantly, the list started coming out more complete. However well someone knows the system, nobody can hold all the connections in their head; the graph does not forget.

What I did not expect was the change in the character of review. When the agent said “this change is safe”, nobody had to trust it; the list returned by the impact query became the input to review. What I called “independent control” in part one, a gate independent of the model’s persuasive power that gives the same result every time, was this in practice.

What did not work? On the first attempt I built the graph in more detail than needed; I made every property, every field a separate node. Queries slowed down, maintenance got harder. In the end I saw that stopping at method and service level was enough. When a field-level detail is needed, asking the compiler for it at that moment is far cheaper than holding everything in the graph.

Nor is this approach tied to a language. The same graph can be built in any language where the compiler provides a semantic model. I also developed this graph to cover multiple languages; I will describe how it works in a separate series. Why I say “no autonomous agent without the graph” is for part eight.

Eleven questions to ask the vendor

Ask these in writing. Verbal “yes”es can come out as “partly” in the contract.

1. How long are code and prompts retained? Is zero retention possible?
2. Which clause of the contract says it will not be used for training? Does it apply to subcontractors too?
3. In which region does the model run? Can we choose the region?
4. Can the tool be routed to our gateway or our model? Which features are disabled then?
5. Can your model run in our data centre? In general availability or in preview?
6. Which settings can be locked centrally? Is the lock enforced on the server or on the client?
7. Can file writes, commands and MCP calls be restricted centrally? Can only approved MCP servers be allowed?
8. How is the agent’s network access limited? Can the agent change its own permission settings?
9. In what format are the logs, how are they exported, how many days are they kept?
10. Is pricing per seat or per usage? Can a spending cap be set at institution level?
11. When the contract ends, how are our data and logs returned or deleted?

If the answer to the fourth is “cannot be routed”, then whatever the answers to the others, it means you cannot build the gateway I will describe in part six.

The next part: local model or external model? The three reasons for the gap and which work goes to which model.

Sources: Claude Code issue #56694 (May 2026); OpenAI Codex managed configuration (requirements.toml) documentation; Tenable (CurXecute and MCPoison, CVE-2025-54135/54136; fixed in Cursor 1.3); The Hacker News and Koi Security (ClawHub, 2 February 2026); Semgrep and Security Boulevard (postmark-mcp, September 2025); arXiv 2605.17062 (package name hallucination, 2026); GitHub Blog (Copilot usage-based billing, 27 April 2026); OpenAI Help Center (Codex rate card, April 2026); GitHub Docs (Copilot on GHES, disconnected environment preview); OpenAI (Dell partnership, 18 May 2026); BenchLM, SWE-bench Verified (8 October 2026); GitClear, The Maintainability Gap (2026).