ShipSafe

What is ShipSafe?

ShipSafe rehearses a database migration against a copy of realistic synthetic data before the release goes out. Given a candidate SQL migration file, it spins up a fresh temporary SQLite database, seeds five synthetic rows, and runs five checks in sequence on that database. The result is a GO or BLOCK verdict backed by a JSON evidence file you can inspect.

IBM Bob IDE investigated the failing migration in this project, identified the root cause, designed and implemented the safe fix, reran all checks, and produced the release decision.

The scenario: BLOCK → GO

The sample app tracks delivery orders. Version 2 of the app adds a delivery_window column — a time slot for each delivery. The unsafe first migration declared this column NOT NULL with no default value. SQLite refuses to add a required column to a table that already has rows with no value for it — so the migration fails, and the release is blocked.

The repaired migration uses DEFAULT NULL, which SQLite accepts. Existing orders get a null delivery window (treated as "not yet assigned"). The old app ignores the new column. Rollback to the old app after the new app has written a row still works, because the old app's queries never mention delivery_window.

Both results are from real CLI runs — no hardcoded verdicts. See the committed evidence:

The five gates

Every run checks all five gates in order on the same temporary database. A FAIL in any gate marks all later gates SKIP and sets the verdict to BLOCK. GO requires all five to PASS.

What IBM Bob IDE did

The repository was handed to Bob IDE with the failing rehearsal already committed. Bob:

1. Read the real evidence/red-run.json output and identified the OperationalError: Cannot add a NOT NULL column with default value NULL.
2. Traced the failure to the ADD COLUMN delivery_window TEXT NOT NULL declaration in examples/unsafe.sql.
3. Designed the safe fix: ADD COLUMN delivery_window TEXT DEFAULT NULL, which SQLite accepts for tables with existing rows.
4. Edited migrations/candidate.sql, reran the rehearsal, confirmed all five gates PASS, and verified the pytest suite still passes (23 tests).
5. Audited the gate evidence objects for fabricated claims; found and fixed three issues documented in LIMITATIONS.md.
6. Produced the committed green evidence and saved task session screenshots to bob_sessions/.

Limitations

ShipSafe is a local SQLite simulation, not a production migration plan.

• The database is synthetic and temporary. No production data is ever read or written.
• SQLite behaviour differs from Postgres, MySQL, and other production RDBMS. A migration that passes here may still fail against a real database.
• The preservation gate checks only the five seeded rows, not arbitrary table contents.
• All gates run single-threaded. No concurrency or transaction-isolation testing is done.
• Elapsed seconds measure local wall-clock overhead, not database performance.
• statements_in_script counts semicolon-delimited segments of the SQL text before execution — a text property, not a count of statements SQLite actually ran.