Bryan CantrillMay 24, 2024

A decade ago, Joyent accidentally rebooted an entire datacenter, an experience that I described in my 2017 GOTO Chicago talk, "Debugging Under Fire":

https://www.youtube.com/watch?v=30jNsCVLpAE

On the next Oxide and Friends, @ahl we will be joined by folks who were at Joyent a decade ago, both to recall the fateful outage and to reflect on its ramifications, both at Joyent and beyond. Join us, Monday, 5p Pacific:

https://discord.gg/dqUCRwsx?event=1243638578484088842

Show threadFilip HajnyMay 24, 2024
@bcantrill I remember that incident Jabber discussion! And do I miss the bots!
Show threadMark Mayo
@mamash @bcantrill I’ve either completely forgotten this (which would be typical) or it was after my time. Will enjoy the story telling regardless! :)
May 24, 2024 at 8:41pmWeb
Show threadMark MayoMay 24, 2024
@mamash @bcantrill I do remember the time we were working so long to hit some deadline that we “lost” Ben on the conference call for a few minutes and when he came back on and David asked what happened all he said was “I can’t see”. 😂😳
Show threadChristopher HorrellMay 25, 2024
@mmayo @mamash @bcantrill oh I very much remember this event and the jabber chat! I was logged into the headnode at the time and suddenly it started rebooting…