VSAM to Relational Migration

Part 6 handled the relational data; this part handles the data that was never relational. VSAM — IBM’s indexed file system — predates relational databases and still holds vast amounts of production data on mainframes: account master files, transaction logs, reference data, all stored as keyed records in flat files that COBOL reads and writes directly. Migrating VSAM to a relational store is a distinct and in some ways harder problem than DB2, for one reason that shapes everything: there is no schema to start from. The structure lives in the code, not in the data.

What VSAM is

VSAM (Virtual Storage Access Method) is a way of organizing records in files for fast keyed and sequential access. The variants you will meet:

• KSDS (Key-Sequenced Data Set) — the common one: records indexed by a key, retrievable directly by that key or read in key order. The closest thing to a primary-key table, except it is a file.
• ESDS (Entry-Sequenced) — records in the order they were written; think append-only log.
• RRDS (Relative Record) — records addressed by relative position number.

In all cases, a record is a flat run of bytes. What those bytes mean — the fields, their types, their positions — is defined not in the file but in the COBOL copybook the programs use to read it. The file is structure-less; the copybook is the schema, and the copybook lives in the source code. This is the crux of the whole migration.

VSAM also has no SQL, no relational constraints, and no enforced relationships between files. A “join” between two VSAM files exists only as COBOL navigation logic — read a key from one file, use it to look up another. The data model of a VSAM-based system is real, but it is implicit, distributed across the copybooks and the access code rather than declared anywhere.

Why there is no schema to migrate

A relational migration (Part 6) starts from a defined schema: tables, columns, types, keys, constraints. You translate a structure you can read. A VSAM migration has no such starting point. Before you can design a relational target, you have to recover the structure, from three sources:

• The copybooks, which define record layouts — and frequently include REDEFINES (the same bytes interpreted as different record shapes depending on a type field) and OCCURS DEPENDING ON (variable-length repeating groups). A single VSAM file can hold several logical record types distinguished only by a discriminator field, with completely different layouts.
• The access patterns, recovered from how the COBOL programs actually read and write the files — which keys are used, which files are looked up together, what the navigation logic implies about relationships. This is where the implicit data model is reconstructed.
• The data itself, sampled and profiled to confirm the layouts hold across decades of records, including older ones written under earlier versions of the copybook.

Discovery is what makes this tractable: AI-accelerated reading of the copybooks and access code together recovers the latent schema far faster than tracing it by hand, and captures it as a living spec the team reviews. But the design of the relational target — normalize or keep wide, how to model the variant record types, where to put the implicit relationships — is a modeling judgment made by senior engineers. The schema is reconstructed and designed, not read off and converted.

VSAM concept	Relational target
KSDS keyed file	Table with primary key
ESDS append log	Insert-ordered / event table
Record bytes + copybook layout	Typed columns from the recovered layout
REDEFINES variant records	Separate tables or typed/discriminated rows
OCCURS DEPENDING ON arrays	Child table (normalized) or array column
File-to-file COBOL navigation	Foreign keys / explicit relationships

The same encoding traps, and a few more

Everything that made DB2 data tricky applies here too, often more rawly because VSAM data is closer to the metal: EBCDIC text, packed-decimal (COMP-3) numbers, binary fields, and sign conventions all have to be decoded correctly per field. With VSAM there is the added hazard that the same physical file may contain records written under different copybook versions over the years — layout drift that a single mapping will silently corrupt. Profiling the actual data, not just trusting the current copybook, is how that gets caught.

The migration discipline

The model is the same incremental, parity-first one used throughout, with schema recovery as the added front-end step:

1. Recover the schema from copybooks, access patterns, and data profiling; design the relational target.
2. Keep VSAM authoritative. The mainframe keeps reading and writing the files throughout; nothing freezes.
3. Migrate and keep in sync. Data moves into the relational target with the encoding and variant-layout traps handled explicitly, and ongoing changes are propagated so the target tracks the files.
4. Reconcile. Records are compared field-by-field and in aggregate against VSAM until they provably agree — the same reconciliation discipline as DB2, made harder by the absence of an original schema to anchor the comparison.
5. Cut over gradually, behind the strangler facade, once the data reconciles and the slices reading it prove parity. The VSAM files retire last.

Because there is often no specification and no schema to test against, this is exactly where proving functional equivalence against the running system — rather than against a document — earns its place. The behavior of the COBOL accessing the files is the spec.

Where relational isn’t the right shape

VSAM migration is genuinely harder to estimate than DB2 work, and it would be dishonest to suggest otherwise: the schema-recovery step means you do not fully know the shape of the problem until discovery has read the copybooks and profiled the data, so this is precisely the kind of workload where a fixed-scope discovery phase before any large commitment matters most. And not every VSAM file should become relational. A high-throughput keyed-access file whose access pattern is purely “look up by key” may map better to a modern key-value store than to a relational table — forcing it into relations can cost performance for no benefit. The target follows the access pattern, not a default preference for SQL.

Where this leads

That completes the technical tour — the transactions, the relational data, the batch, and now the indexed files. With the how understood across the stack, the conversation in any real organization turns to the question that actually decides whether a program happens: what does it cost, and is it worth it? Part 9, Mainframe Modernization Cost, takes on the economics — the cost drivers specific to the mainframe, the total cost of ownership the platform’s bill hides, and the cost of delay that is usually the largest number in the business case.