In the previous parts I described a setup in which code does not leave the institution. The question I heard most often while building it was this: how far behind the cloud models such as Claude or GPT does a local model running on our own servers fall?
We looked for the answer by using both. For work that could contain sensitive data we used only the local model; requirements documents, technical analyses and code were in this class. For work that did not contain sensitive data we used external models and kept quality at the top.
We drew the line not by product or customer but by the job itself. Even within the same product there is work that may contain sensitive data and work that will not. The team that knew the work best decided which was which; model selection followed that decision.
Where did the gap show up?
On the first attempt we took the easy road: we gave the prompts and context structure we had prepared for the external model to the local model as they were. The result was disappointing. Instructions the external model understood with ease, the local model either misunderstood or carried out incompletely. The result we got for the same job fell far behind the external model.
We did not stop there. We rewrote the prompts for the local model, strengthened the context and carried the information from the code graph to the model in a more orderly way. The local model did not reach the external model’s level, but the gap closed noticeably and the results rose to an acceptable level.
We experienced the same thing in software development. Developers who used external models comfortably saw both a drop in quality and the trust problem I describe below when they switched to the local model. Once we wrote our own plugin, took control of the context going to the model and brought the code graph into play, the gain in code development rose noticeably.
The real lesson for me was this: the same model gives a completely different result with context and prompts prepared for it. Giving a local model a structure built for an external model, as is, makes the gap look bigger than it is. I grouped where the gap comes from under three headings.
Reason one: model quality
The first guess was “the external model is smarter”. That is true and should not be underestimated. In October 2026 the strongest cloud models such as Claude and GPT are in the 95–96% band on SWE-bench Verified. The best open models that fit on a single GPU are between 72% and 77%; the largest open models in the 400-billion-parameter class reach 80–86%.
The accounts of those who actually use these models draw the picture more concretely. Local models handle roughly 70% of daily coding work and give results close to external models on single-file tasks. On multi-file tasks they fall noticeably behind. A common analogy: the local model is a junior who needs close guidance; the external model is a senior who thinks through the architecture with you.
But how this gap is distributed matters. On simple jobs such as test scaffolding, DTOs, mapping code and documentation, the local model gives almost the same result. The gap opens in complex reasoning: a multi-step refactor, deriving a design from an ambiguous requirement, the question “why?”. And these jobs are a small share of the total volume.
This distribution made our decision easier. For most of the sensitive work the local model was enough. For the complex work where the gap opened, we switched to the largest model we had locally, or broke the job into small steps and supported it with the code graph. This is the root of the tiered usage model I will describe in part five.
Reason two: context and prompts
External models have large context windows and fast answers. Locally, as you enlarge the window, GPU memory and latency grow too. The developer then gives the model less context and makes more turns. Every turn means time, and every turn is a chance for the model to forget something.
My answer to this was architecture, not the model. The context layer and the code graph I described in parts two and three give the model small but correct context. Even if the window stays small, when what goes into it is on target, the gap narrows. A large window compensates for bad context; good context reduces the need for a large window.
The same goes for prompts. The external model can work out what is wanted from a short, loose instruction. The local model wants a clearer, step-by-step instruction with examples. We recovered a significant part of the quality we lost when giving the local model prompts written for the external model simply by rewriting the prompts.
Reason three: trust
This is the reason I did not expect, but perhaps the biggest. Developers trust the external model more; because they already use it outside and there is a known brand behind it. They approach the local model with the prejudice “in-house, surely worse”. They try it less and give up at the first mistake.
Same job, same quality, different intensity of use. In measurement this looks like a productivity gap, but it is actually a trust gap.
The solution is not technical either. Examples showing the jobs the local model does well, pair-working sessions and bootcamps. After a few months, when usage evens out, the measured gap shrinks too.
Industry data points to the same place. In Sonar’s January 2026 survey of more than 1,100 developers, 96% of respondents said they do not fully trust AI output; but only 48% always check before committing. And 35% use AI tools with a personal account rather than a work account.
That last figure is the hidden side of the local model debate. In an institution that allows only a local model, if the local model is found inadequate, the developer can go to an external model with a personal account, and takes the code along. That is why adoption of the local model was a security matter for us: if the local model does not do the job, the ban stays on paper.
Which work goes to which model
Combining the three reasons, the decision is made with three questions, and the order of the questions matters.
- Sensitivity, the eliminating question: Does the job touch the institution’s sensitive data? Code, customer data, requirements and internal documents were in this class for us. If it does, the external model is out of the discussion; whatever the job, local. If it does not, the external model is a candidate.
- Type of work, the selecting question: Simple generation or complex reasoning? If simple, a small model is enough. If complex, a large model; for sensitive work the largest local one, for non-sensitive work the external model.
- Latency, the tuning question: Is line-by-line completion expected? A small, fast local model. Is a document expected? A large model, and waiting a minute is no problem.
These three questions are answered in a minute and turn the question “which model?” into a policy. Below is the version applied to common types of work: one column for work that may contain sensitive data, the other for work that will not.
| Type of work | Contains code or data? | Complexity | Latency | Model |
|---|---|---|---|---|
| Code completion (line) | Yes | Simple | Critical | Local, small |
| Unit test generation | Yes | Simple | Medium | Local, small/medium |
| DTOs, mapping, transformation code | Yes | Simple | Medium | Local, small |
| Code explanation, commit message | Yes | Simple | Medium | Local, small |
| Explaining existing code (onboarding) | Yes | Medium | Low | Local, large |
| Multi-step refactor | Yes | Complex | Low | Local, large + code graph |
| Impact analysis | Yes | Complex | Low | Code graph (not a model) + local, large |
| Error analysis (stack trace + code) | Yes | Medium | Medium | Local, large |
| Questions that clarify a requirement | No | Medium | Low | External model |
| Technical analysis draft (from the requirement text only) | No | Complex | Low | External model |
| Technical analysis enriched with an impact list | Yes (code metadata) | Complex | Low | Local, large |
| Functional test scenario | No | Medium | Low | External model |
| Technical documentation, README | Partly | Simple | Low | Local, large |
| Regulation summary, compliance note | No | Medium | Low | External model |
| Architectural option assessment | No | Complex | Low | External model |
| Analysis containing customer data | Yes (data) | — | — | Local in every case |
The table’s single principle is this: the line is drawn by the kind of data, not the kind of tool.
Where the line blurs
That code and customer data are sensitive is something everyone accepts. The debate begins with information that is not code but does not stay outside the code either.
A requirements document is not code. But it describes the customer’s product, business rules and features about to be released. A good technical analysis also says what is affected in the existing system; that information comes from the code graph: class names, method signatures, table names, batch jobs. This is not code, but it is the map of the code. When an institution’s table and service names go out, the skeleton of the system has gone out too.
We counted both as sensitive and kept them local. Your institution may classify these more loosely; but make that decision consciously, not by default because “it’s not code”.
Should the institution accept this gap?
This is now a management question. A local model means accepting being behind the best model. But in an institution where sensitive data cannot leave, there is no other option anyway. So the question is not “accepting the gap” but “shrinking the gap”.
Two of the three reasons, context and trust, close without changing the model. For the third, complex reasoning, a new but still narrow door is opening in 2026: the vendor’s model running in the institution’s own data centre. This option is limited and expensive today. But once you know that complex work is a minority, running an expensive model in-house for that minority becomes a calculable decision.
The dilemma of 2024 was “local or external?” The question of 2026 is different: which model will be used for which job, where will that model run, and how will it be optimised accordingly? In our experience a significant part of the gap came not from the model but from this last question.
The next part is the rulebook: who enforces this table, how limits are set, what happens when a tool is not enough? With a sample policy file.
Sources: BenchLM, SWE-bench Verified (8 October 2026); Tom Tunguz, local coding models (16 June 2026); byteiota, local models vs Claude comparison (16 June 2026); Sonar, State of Code developer survey (8 January 2026).