public
Created

Twas the night before Opsmas..

  • Download Gist
night-before-opsmas.txt
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
'Twas the night before Christmas, when all through the racks
Not a server was alerting, not even Compaqs.
The backups were written to tapes with care
In hopes that later the data would be there.
 
The machines were nestled all snug in their sleds
Whilst visions of vengeance danced in their heads;
And oncall in his three-wolf and I in my rack.
Had just settled down for some syn and some ack.
 
When from the old pager arose such a clatter
Oncall sprang from his seat to see what's the matter.
Away to the terminal he flew with a flash!
Tripped on some cables and fell with a smash.
 
The pager buzzing and beeping, lit with a glow
Tired and angry, Oncall hurled it with such a throw.
Rubbing open his eyes when what should appear
But a cascading failure, the biggest one of the year!
 
Servers, applications and databases all turning red
Oncall's denting the desk, banging his head.
The air dry, with a smell of burning tire.
I think Oncall's pager may have caught fire.
 
And then, with a jolt, the phone began to scream
A torrent of angry users, from every remote team
"Sorry, I'm working as fast as I can!"
His voice trailing, eyes shifting to the SAN.
 
In a monotonous cadence the red light did blink
Oncall gasped, and his heart started to sink.
I've seen this look before, he's terribly afraid
Wondering if enough disks survived to rebuild the RAID.
 
Opening new shells, on his keyboard he did mash
"How could all these disks so suddenly crash?"
Not enough left, what remained was one too few
Enraged and upset Oncall started to stew.
 
Thinking about striping and distributed parity
Oncall sat upright with a moment of clarity.
He mumbled to himself, shuffling across the floor
"We have backups, I'll just need to restore!"
 
Opening the cabinet holding the tapes for the drives,
His mouth fell open, and he broke out in hives.
Red in his face and eyes welling with tears:
"These disks haven't been backed up for a couple of years!"
 
Just then the door swung open with a rattle
Must be the next guy's turn to do battle.
With a calm voice, despite things looking bleak
Oncall explained "I'm on vacation the rest of the week"
 
Grabbing his coat, not wanting any more,
He winked at me and opened the door.
Smile on his face, nothing more to fear,
"Hey good luck man, I'll see you next year!"

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.