Why Software Rewrites Fail

Part 1 named the big-bang rewrite as a trap that hides its own risk until cutover. This part answers the question that follows: why? Why does a shape chosen by capable teams, for honest reasons, fail with such consistency? The failures are worth studying directly because they are not random. They cluster around a handful of well-documented forces that act on rewrites specifically — and once you can name them, the case for a different shape stops looking like caution and starts looking like the obvious response.

A note before the forces themselves: the teams behind failed rewrites are rarely incompetent. They are usually serious engineers who made a structurally risky choice that felt responsible. Studying the failure is not about blame; it is about seeing the pattern clearly enough to avoid it.

Failure 1: the second-system effect

Fred Brooks named this in The Mythical Man-Month (1975), and it has aged into one of the most reliable observations in software. The second system a team designs is the most dangerous one it will ever build. The first version was disciplined by inexperience and tight constraints, so it shipped lean. The second is built by a team that now “knows better” and is no longer constrained — so it accumulates every feature, generalization, and refinement the first version had to leave out.

A rewrite is a second system by definition. Freed from the old system’s limits, scope balloons: the replacement is asked not just to match the original but to be cleaner, more general, more future-proof. The schedule slips under the weight of the ambition, and the “clean” rebuild becomes a new tangle that arrives late and over-built. The very freedom that made the rewrite attractive is the thing that sinks it.

The fix: force each increment to ship before the next begins. When a slice must reach production and prove itself before the team moves on, there is no room for the second-system sprawl — scope is bounded by the slice, and ambition has to earn its place one increment at a time.

Failure 2: the knowledge you discard silently

A legacy system’s most valuable asset is not its code — it is the behavior that code encodes, including the rules nobody ever wrote down. The rebate logic for one large account. The validation that exists because of an incident in 2014. The ordering that must hold for reasons long forgotten. This knowledge lives in the source and in a shrinking number of heads.

In Things You Should Never Do, Part I (2000), Joel Spolsky made the enduring case against the from-scratch rewrite using Netscape, whose decision to rebuild its browser cost it years of market position. His core point: the ugly code you want to throw away is ugly because it encodes years of fixes for real-world edge cases. Rewrite from a clean slate and you discard all of it silently — then rediscover each case, one at a time, as a production incident after cutover. The new system implements what the team thinks the old one did; the gap between that and what it actually did becomes a stream of “new” bugs that were really old, solved problems thrown away.

This compounds with a demographic clock. The people who remember why a system behaves as it does are often the ones nearest retirement — for the COBOL estates that run much of finance and government, the average developer’s age has been reported at around 58, with roughly 10% of that workforce retiring each year (IBM, reported via Fujitsu, 2020). The knowledge is leaving on a schedule, whether or not the rebuild is ready for it.

The fix: capture the behavior before you change it, derived from the actual codebase rather than from memory or stale documentation, and hold the replacement to what the old system actually does — not to a specification of what it was meant to do. That discipline has its own name and its own series: parity validation.

Failure 3: the moving target

A rewrite assumes a fixed target — the system as it exists today — and then spends years trying to hit it. But the legacy system cannot be frozen. The business still needs changes: a new regulation, a pricing update, a competitor to answer. So the old system keeps evolving while the rebuild chases it, and the gap never quite closes.

This produces a cruel dilemma covered in Part 4: either the business freezes the roadmap so the target stops moving — and falls behind the market for the years the rebuild takes — or it keeps shipping, and the rebuild is permanently chasing a system that refuses to hold still. Neither resolves well. A big-bang quoted at two years is, in practice, a program aimed at a target that has moved by the time it arrives.

The fix: modernize through the running business rather than against a frozen snapshot of it. When you replace one slice at a time, each slice targets a small, current piece of behavior and locks it in immediately — the target for any given increment is small enough that it doesn’t move out from under you before the increment lands.

Failure 4: the sunk-cost ratchet

The previous three failures could, in principle, be caught and corrected. The fourth is what stops that from happening. A big-bang rewrite delivers no value until cutover, so for the entire build there is nothing to objectively measure the program against — only spend. As the spend accumulates, the cost of admitting the approach is wrong rises with it. Two years in, “stop and rethink” means writing off two years, which is politically almost impossible. So the program ratchets forward: each increment of sunk cost makes turning back harder, parity gaps get patched live under pressure rather than triggering a reassessment, and the program pushes to a cutover it can no longer safely reverse.

This is the mechanism that turns the other three failures from recoverable mistakes into terminal ones. It is also the one most invisible from inside, because every individual decision to continue feels rational in the moment.

The fix: make stopping cheap and continuation a real choice. When the program delivers working, proven software every few increments, you can stop after any slice with value already banked — the option a big-bang never offers. The decision to continue is then made on the merits each time, not extorted by sunk cost.

The pattern behind the failures

Failure	Root force	The fix in one line
Second-system effect	Unconstrained ambition	Ship each slice before starting the next
Discarded knowledge	Behavior never captured	Prove parity against the legacy, per slice
Moving target	Legacy keeps changing	Target small, current behavior per increment
Sunk-cost ratchet	No value until cutover	Bank value early; keep stopping cheap

Read the middle column and the common root is unmistakable: every one of these forces is amplified by concentration — concentrating the whole rebuild into one effort proven at one distant date. The right-hand column does the same thing in every row: it distributes what the rewrite concentrates, across many small increments each cheap to get wrong. The failure is structural, and so is the remedy. None of this is a claim that incremental replacement is free or guaranteed — Part 4 onward is honest about its real costs — only that it removes the specific forces that make rewrites fail so reliably.

Where this leads

These forces explain why rewrites fail. The natural next question from any executive hearing this is how often — and what the actual evidence says, separated from the numbers that get quoted loosely. Part 3, Modernization Failure Rate Statistics, gives you the sourced figures, names what each one does and does not prove, and shows how to use them in a business case without overreaching — because a statistic used dishonestly is no better than no statistic at all.