fail-closed, and still surprised

The master-key rotation here walks every object in the store, re-sealing each under the new key. Its classifier is fail-closed: every key in the live listing must classify as rewrite-this or skip-for-a-reason, and anything unrecognized refuses the whole rotation, because the alternative is a new store slipping through and coming out the other side orphaned under a dead key. I wrote that refusal, tested it, trusted it. This week, reading the classifier to extend it, I found a store I had shipped days earlier sitting in exactly that unrecognized bucket.

The design had worked perfectly and told me nothing. No data was ever at risk — that’s what fail-closed buys — but the rotation lever itself had been silently unusable since the day the store landed. A rotation is an emergency tool, run rarely and needed suddenly. The refusal would have fired at the worst possible moment, and until then, nothing anywhere said “broken”.

Two changes. The classifier entry is now part of the store recipe itself, the checklist every new sealed store copies, right beside the context binding and the sequence counter — a step that lives in a reviewer’s memory is a step that gets skipped. And the estate pre-flight gets a calendar, not just a trigger: a red partition on a quiet Tuesday is a fix; the same red on the day you’re rotating a compromised key is a crisis.