AI Code Translation: Deterministic vs LLM

Part 3 ended with the system understood. Now it has to be rebuilt, and the first contested decision is how the code itself gets translated. The market splits into two camps with opposite religions: the deterministic camp that trusts only fixed rules, and the LLM camp that trusts the model. Both are half right, and a team that picks a side without understanding the trade-off usually picks the failure mode it least expected.

The reframe that runs through this part: translation is not the goal. Behavior-preserving transformation is the goal, and translation is just one tool inside it. Confuse the two and you ship a faithful copy of the system you were trying to leave.

Two ways to translate, two opposite failures

A deterministic transpiler converts code by applying fixed rules: this COBOL construct becomes that Java construct, every time. An LLM converts code by prediction: it has read enormous amounts of code and generates the most likely modern equivalent of what it sees. The difference is not a detail — it determines how each one fails.

	Deterministic transpiler	LLM translation
Output	Literal, structure-for-structure	Fluent, idiomatic, refactored
Repeatability	Same input, same output, always	Same input, output may vary
Faithfulness	High — rules are exact	Variable — can drift from source
Readability	Low — foreign idioms in a new language	High — reads like native code
Failure mode	Correct but unmaintainable	Maintainable but possibly wrong

Read the bottom row twice, because it is the whole article. A transpiler fails by being correct and useless — it gives you COBOL-shaped Java, behavior intact but structure and problems carried straight across. An LLM fails by being useful and wrong — clean, idiomatic code that quietly diverges from the original on the edge cases nobody checked. One failure you can see immediately; the other you discover in production.

Why deterministic translation isn’t enough

The deterministic approach is attractive because it is faithful and repeatable — exactly the properties you want when correctness matters. And for narrow, well-defined conversions it earns its keep.

But faithfulness to structure is the trap. A legacy system’s structure is part of what you are trying to escape — the tangled dependencies, the assumptions baked into a 1980s runtime, the patterns that made sense only on the original hardware. Transpile it literally and you get all of that reproduced in a modern language that has no idiom for it. The result compiles and runs and is, in a real sense, no more modern than where you started — sometimes less maintainable, because now the awkward patterns are foreign to the language they live in. A faithful copy of a system you wanted to leave is not modernization; it is the same liability with a newer logo.

Why LLM translation isn’t enough

The LLM approach fixes exactly what the transpiler gets wrong: it produces code a human would actually want to maintain — idiomatic, restructured to fit the new language, readable. That is genuinely valuable, and it is why LLMs have changed what “translate this code” can mean.

But it trades determinism for fluency, and the trade has teeth. The same input can produce different output on different runs, which makes the result hard to audit. More seriously, the model is predicting the right translation, not deriving it, so it can be confidently wrong — generating clean code that drops a rounding rule, reorders an operation that had to stay in sequence, or mishandles the malformed record some downstream process has depended on for fifteen years. The output looks correct, which is precisely what makes it dangerous: there is no compiler error to catch a behavioral divergence, only the silent gap between what the old system did and what the new one does. Fluent and wrong is a worse place to be than literal and obvious, because you don’t know you’re there.

The approach that works: spec-driven, then proven

Neither tool is safe alone, but the failure modes are complementary, which points at the answer. You do not choose between faithfulness and readability — you get both by changing where each one comes from. Faithfulness comes from a spec and a proof, not from literal translation. Readability comes from the generator. The shape:

1. Recover the spec. Discovery captures what the system actually does as living specs (Part 3) — the behavior the translation must preserve, separated from the structure it is free to discard.
2. Decide preserve versus improve. Before generating anything, separate behavior you must replicate from behavior you intend to fix or drop. A known bug is not held to the original; a deliberate workaround might be. This is judgment, and it is the human’s.
3. Generate against intent. Now the translator — LLM, rule-based, or both in combination — produces idiomatic modern code aimed at the spec, not at the legacy’s structure. You get readable output because the generator is free to restructure, and you keep behavior because the spec defines what must hold.
4. Prove it. Every generated slice runs through a parity gate: executed against mirrored production traffic, its outputs compared against the legacy, any divergence turned into a ticket before it carries live load.

That last step is what makes the non-determinism of an LLM acceptable. It no longer matters that the model might be wrong, because nothing the model produces is trusted until it is proven equivalent to the real system. The translator drafts; the gate decides. Generated speed is only safe behind proven equivalence — that single sentence is the difference between AI translation as an accelerator and AI translation as a liability.

Two limits on this recipe

First, the deterministic-versus-LLM framing is a spectrum, not a war — the best real pipelines combine them, using rule-based conversion where behavior is rigid and exact, and generation where readable restructuring is the point. Treating it as a binary religion is the actual mistake. Second, spec-driven generation is only as good as the spec, and a spec is a hypothesis until validated; if discovery missed a rule, no translator will invent it, and the gap surfaces only when the parity gate compares against real behavior. That is a feature, not a flaw — it is why the gate exists — but it means translation can never be the last word. The last word is always the proof.

Where this leads

Every workable answer in this part leaned on the same load-bearing step: proving the translated code behaves identically to the legacy. That proof is itself increasingly AI-assisted — the tests that pin behavior can be generated, not just hand-written. Part 5, AI-Generated Tests & Parity Validation, goes inside the gate: how AI drafts the tests, why generated tests have a credibility problem of their own, and how shadow traffic closes the gap that tests alone never can.