He talked about electric cars. I don't know anything about cars, so when people said he was a genius I figured he must be a genius.

Then he talked about rockets. I don't know anything about rockets, so when people said he was a genius I figured he must be a genius.

Now he talks about software. I happen to know a lot about software & Elon Musk is saying the stupidest shit I've ever heard anyone say, so when people say he's a genius I figure I should stay the hell away from his cars and rockets.

@rodhilton You obviously know nothing about software. Elon will show the world his true genius as soon as he single-handedly rewrites entire Twitter stack... in PHP...

@hu_logic @rodhilton

Elon read a blog from 10 years ago where someone claimed to write Twitter in a weekend as a side project, and got all excited that he could rewrite Twitter quickly.

@nobodyzhome It's a distributed systems design interview question, meant to be solved at a high level, in the abstract. I think he literally thinks the implementation is therefore that easy.... 😂🤷 @hu_logic @rodhilton

@reneestephen @nobodyzhome @rodhilton Twitter had to optimize and prioritize network routing between its DCs in order to scale properly. That's the kind of problems such implementation would have to solve at the grunt level.

To get it done right takes time, money and quality engineers.

Elon thinks he can do it with twenty sleep-deprived "10X Hardcore 2.0 SWEs" in 6 months on a shoestring budget... while shutting down several DCs in January.

@hu_logic Worse, the system is currently still running because of those engineers and he is slapping himself on the back for it not failing outright when they left... as if they were literally running on little hamster wheels to route the packets themselves or something. Idk, the guy is a deeply stupid grifter.

I'm mad about him destroying a community and some cool tech because he is so deeply insecure.

A lot of us screamed this for a decade+, and now it's too late.
@nobodyzhome @rodhilton

@reneestephen @nobodyzhome @rodhilton Imagine you're part of 10X Hardcore 2.0 skeleton crew and have been troubleshooting Sev1 production issue for a week. As a result, you haven't had any time to write any "code" so Elon fired you...

Then, it turns out that Sev1 was caused by Elon randomly unplugging shit in the DC...

@hu_logic jfc somewhere, someone's carefully designed and rehearsed DR plan* just quietly went into effect and failed over successfully and he doesn't even realize what happened. 🤦

*Assuming it isn't blue/green or continual deployment bouncing between locations on every push already.

@nobodyzhome @rodhilton

@reneestephen @nobodyzhome @rodhilton Somewhere, some CTO talking to his VP of infrastructure: "Sure, we have DR plans and architecture. But do we have Musk-proof DR plans and architecture? Make it happen!"

@hu_logic none of this will matter soon anyway because he is definitely not paying his bills -- either on-prem or cloud. Can't fail over if there's nowhere to fail over to! *clever-temple-tap.jpg*

@nobodyzhome @rodhilton

@hu_logic @reneestephen @nobodyzhome @rodhilton
I’ve hard that’s what people at SpaceX and Tesla actually have to do. Perhaps Twitter was such a debacle because Elon didn’t have a reliable set of “handlers”.

@nobodyzhome @hu_logic @rodhilton @reneestephen

we’re assuming somebody didn’t point him at a rack of IDK, HVAC, tell him it was vital vital server equipment and turn it back on as soon as he left the room.

Or equally likely, he’s just fried HA and yeah, everything still works…

@nobodyzhome @hu_logic @rodhilton @reneestephen

Or just a loss in surge capacity.

The difference between NBD and OMG is often no more than when it happens.

But fried HA means someone needs to spend time, maybe dollars, standing shit back up. Which is not always risk free either.

@reneestephen I’ve uhhhh got bad news. I have it on good authority that nobody at Twitter knows whether it’s possible to “cold start” the system from nothing.

@joeblubaugh @reneestephen GOOOOOOOOD.

Apparently, the world needs a rude awakening to the fact that people who belong into a guareded care facility have no place in the economy (or politics for that matter), and all other ways I can think of carry immense collateral damage.

@joeblubaugh @reneestephen This is probably true of basically every big web service. Netflix, AWS, Gmail, all of these services run quite well, and I bet nobody on their teams would feel confident about having to do a true shutdown and cold boot.
@tcb @joeblubaugh @reneestephen Netflix is pretty hardcore on DiRT and chaos engineering. I remember reading something about a 24h cold start from tape on a new data centre type test in an early article on chaos monkey. Probably the service most likely to cold start.
@reneestephen @joeblubaugh I’ve been wondering about this for a while. I think the chances of a system as complex as Twitter having at least one set of circular dependencies is basically nailed-on. It just depends how critical a system they’re in.
@hu_logic @reneestephen @nobodyzhome @rodhilton Your screenshot was so ridiculous I just wasted time going to confirm that it was real.
@nparikh @reneestephen @nobodyzhome @rodhilton NGL, I lost some braincells and faith in humanity reading his Tweet - "Twitter stack is so shit that I've unplugged half of its nodes and it still works! See? Total rewrite!"

@hu_logic @reneestephen @nobodyzhome @rodhilton It is *literally* what a chaos monkey is. Usually the idea is the chaos monkey is a concept or tool, not an actual guy let alone the CEO.

The line about Rube Goldberg is a pretty strong hint about what he doesn’t know about engineering, too.

@nparikh @reneestephen @nobodyzhome @rodhilton Some people have a gift of taking a failing startup and turning it into a multi-billion-dollar enterprise.

Elon has a gift of taking a $44B enterprise and turning it into a failing startup...

@hu_logic @reneestephen @nobodyzhome @rodhilton Somebody trying to be *reverse* Steve Jobs is definitely a new thing to witness.
@nparikh @hu_logic @reneestephen @[email protected] @rodhilton I'm no fan of Saint Jobbies, but at least he knew how to make stuff succeed and (mostly) when to keep his mouth shut and not fire people. A lot of the technobros aren't really all that tech but they are shrewd; but Musk just revels in absolute chaos, nothing else matters as long as he's the centre of attention. :|
@nparikh @hu_logic @reneestephen @nobodyzhome @rodhilton
There's stuff you let the chaos monkey turn off and on, and then there's Accounts Receivable and Advertisers (he turned off 50% of them one day) and Content Moderation.
@hu_logic @nparikh @reneestephen @nobodyzhome @rodhilton Yeah apparently the self-proclaimed “smartest person in the world” has never heard of redundancy or failover clusters.
@MGoCoder @hu_logic @reneestephen @nobodyzhome @rodhilton Somehow he also doesn’t connect it to why it can feel like a bunch of Rube Goldberg machines. It’s like Lewis Black’s bit on “if it weren’t for my horse” — if you spend too much time thinking about it you’ll get an aneurysm.
@nparikh @MGoCoder @reneestephen @nobodyzhome @rodhilton Somebody tried to explain "systems integration" to Elon. That person has been fired.

@MGoCoder I mean, where would he have? For real making cars is all low-latency on-prem automation and MechEng, a lot of factories are deeply bad at BI data and observability, even newer ones. Same with rocketry.

He also hasn't had people at Twitter whose job it was to explain this shit to him and steer him away from the worst ideas (or just ignore him/lie to him and do what should be done despite him), like in his other companies.

@hu_logic @nparikh @nobodyzhome @rodhilton

@reneestephen @MGoCoder @hu_logic @nparikh @nobodyzhome @rodhilton

I've seen the stories about how at Tesla and SpaceX (and, presumably, Neuralink), they have a thick layer of bureaucracy to manage him specifically.

Full self-driving aside, his main two companies manage because he's not doing the engineering, and just knows a bit of the jargon. Twitter, he's actually trying to do the work and showing off he doesn't know how.

@nparikh @hu_logic @reneestephen @nobodyzhome @rodhilton Musk is Poe's Law personified. It's impossible to write a parody more ridiculous than what he himself tweets.
@hu_logic @reneestephen @nobodyzhome @rodhilton wait, is this screenshot real? it can't be real, can it?
@hu_logic @reneestephen @nobodyzhome @rodhilton I thought the solution to all technical issues is to turn the machine off and turn it back on again. It worked in #Jurassic Park. Apparently #ElonMusk agrees.
@riggbeck @hu_logic @reneestephen @nobodyzhome @rodhilton Oh, I never considered that Elon Musk could be a velociraptor.
@JediMB @hu_logic @reneestephen @nobodyzhome @rodhilton I agree. According to Jurassic Park, velociraptors work together to achieve a common goal. Musk is more like one of those dinosaurs who uses his power to muscle in on the kills made by others. A sort of scavenger.

@riggbeck @hu_logic @reneestephen @nobodyzhome @rodhilton I meant that the raptors were the ones who benefited from the "turn it off and on again" method, though.

And in the movie continuity, the Big One took over the pride on Nublar by killing most of the other raptors and bullying the remaining two into following her, so...

(I guess the analogy falls apart when it comes to the Big One's heightened intellect.)

@riggbeck @hu_logic @reneestephen @nobodyzhome @rodhilton The trick is to turn them off, HIT THEM HARD, and then turn them on again.

@hu_logic @reneestephen @nobodyzhome @rodhilton

Lmao, this reminds me of when I was working at a place that had a data center with a storage farm. The director was taking some people on a tour, there were two tour groups, a morning and an afternoon one.

To demonstrate the fault tolerance he showed the group that he could just pull a hard drive and things would keep humming along. He plugged it back in and it started recovery.

@hu_logic @reneestephen @nobodyzhome @rodhilton

Did the same thing for the afternoon group, only it was the same node and it was still recovering and didn't have enough redundancy left, so it took down the whole storage cluster for a while.

@flabberghaster we all wait for that not to happen with twitter..
That amount of new users
is a little bit to much for one day. 😇

Musk is doing a nice job with every week, every fortnight creating a new wave..
@hu_logic @reneestephen @nobodyzhome @rodhilton

@hu_logic @reneestephen @nobodyzhome @rodhilton

I'm willing to bet Musk doesn't understand fractals, Goldberg, or how Twitter works.

@hu_logic @reneestephen @nobodyzhome @rodhilton
clearly he's never had to understand anything complicated before.
@hu_logic @reneestephen @nobodyzhome @rodhilton he's playing the IT equivalent of the russian roulette.

@hu_logic @reneestephen @nobodyzhome @rodhilton

He’s modeling himself on Free Guy’s villain, except with none of Taika Waititi’s charm and charisma.

@hu_logic @reneestephen @[email protected] @rodhilton "Fractal of Rube Goldberg machines"? What does that even mean? Rhetorical question obvs, he has a habit of coming out with this sort of crap. At least it keeps his legion of weird fanbois happy.
@reneestephen @hu_logic @nobodyzhome @rodhilton This. The thing about Elon saying incredibly stupid shit about software is that he's been saying incredibly stupid shit about it for a decade if not longer and everybody either outright believed or wanted to believe he's a super genius about all the things and all of us who criticized him are just haters. Sorry I'm still a little bitter about that. :P