The Restore Rehearsal Kept Stopping

Week ending November 13.

I'm getting ready for database upgrades that will mean downtime for my sites and for Red itself. I had told Red to prepare the backups first. I would choose the downtime after.

The backups were finished the week before. This week I asked Red to finish the recovery checks. That meant rehearsing a restore and testing what came back, without changing anything live.

On Tuesday the rehearsal kept stopping.

One check expected a number and got the same value as text, so the run stopped. More than once the run needed my go-ahead, and the request expired before anything started. Red asked me to say when I was ready instead of trying again on its own.

One restore came back and then failed a test. A test user had been refused a record. The rule that refused it was right, and the test was wrong. Red corrected the test and left the rule alone.

The fixes were reviewed independently before the next run.

By the end of the day, most of the rehearsal had passed and part of it had not. Red still marked the whole batch as failed.

As of Wednesday, the rehearsal isn't finished. The next attempt is prepared and waiting on me. No upgrade has started, and I haven't chosen the downtime.