abadideaJune 2023: a Google data center in France floods and they call it a “water intrusion event”
March 2026: an Amazon data center in the Middle East is literally struck by a fucking ballistic missile in a hot war and they call it “impacted by objects”
Sven@0xabad1dea if it was "literally struck by a ballistic missile" they'd not be talking about bringing it back up in a few hours...
NKT@HeNeArXn @0xabad1dea of course they would!
eestileib (she/hers)Ballistic Missile just means it finishes its burn relatively early (contrast to Cruise Missile eg).
There are many sizes and payloads, they aren't all nuclear.
@eestileib @HeNeArXn @0xabad1dea if it was a nuke, it wouldn't be back up any time soon. But it also wouldn't be from Iran. Israel or the US have nukes though. Neither are part of the worldwide nuclear monitoring program, unlike, er, Iran...
If it had been a nuke we would all either know, or be dead.
Tyrone Slothrop@HeNeArXn @0xabad1dea FWIW we’re currently at “full recovery is still expected to be many hours away”
Mx Rey (Fuck ICE) 💜 🏳️⚧️@slothrop @HeNeArXn @0xabad1dea latest update:
"We are expecting recovery to take at least a day, as it requires repair of facilities, cooling and power systems, coordination with local authorities, and careful assessment to ensure the safety of our operators."
yeah that's gonna take a bit
@HeNeArXn presumably the missile was not aimed directly at the data center but at something nearby. If your neighbor is exploded off the face of the earth by a missile and your power generator is taken out by the kinetic impact, you have been struck by a missile, you’re just luckily alive enough to complain about it
@0xabad1dea or its fragments from an intercepted missile or drone, or from something fired at an incoming missile, or bits from a plane, or ... and they might not even know yet what exactly it was. Hence, "objects".
JdeBPThe power generator was not taken out by any sort of impact, here, though. It was taken out by the fire department.
https://mastodonapp.uk/@JdeBP/116155667315973528
@wild1145
#UnitedArabEmirates #AWS #Amazon #CloudComputing #DataCentres #HansGruber
Ryan Wild@JdeBP @0xabad1dea @HeNeArXn Looks like the two UAE AZ's were due to drone strikes. AWS have updated their status at 4:19pm pst yesterday. It looks like it's the Bahrain regions az that was colatoral damage of drone strikes.
I don't think that there's actually a contradiction here between what they said earlier and later. The root cause is the missile strike, whereas the proximal cause was the fire department shutting off the power in response to the fire that that caused.
It's an interesting thing for a data centre customer to consider: In the event of a fire, gas leak, explosion, or some such, is the fire department going to, as S.O.P., shut off, without any grace period, *both* primary and backup power?
Because my reading of what #Amazon first wrote is that that is what the #UnitedArabEmirates fire department's response was: shut off all power, mains and backup generators, immediately.
Hence #HansGruber. (-:
It's not unreasonable of the fire department, but it's also salutary to remember that there are going to be cases, to DR plan for, where it is actually procedure to have an unclean shutdown of all machines at once.
@0xabad1dea @HeNeArXn @vmstan
#AWS #CloudComputing #DataCentres
@JdeBP Ah yes, I'd mis-read one of the earlier replies in the thread. I think part of why this is so noticeable is the AWS region is clearly a fairly small region, it's not super common now but in the newer regions especially having 3 Data centres mapping to 3 AZ's is not uncommon, but it does mean these sorts of things can have a much bigger impact.
This is the first time at least that I've found in AWS history that a region is significantly impaired due to effectively the loss of two geographically isolated AZ's (I don't remember from my time working there what the rules were around distance but they won't be exactly next door to one another!).
It'll be really interesting to see if everything can be recovered, given it's physical damage to the actual DC facilities there's likely to be a lot more work involved making the sites safe and getting it operational, it takes years to build these sites and they aren't exactly simple...