AI6YR BenFormer Uber self-driving chief crashes his Tesla on FSD, exposes supervision problem
https://electrek.co/2026/03/17/former-uber-self-driving-chief-tesla-fsd-crash-supervision-problem/
**VERY glad the guy and his kids are okay, but it would have been something else if the Uber self-driving chief had been incinerated or killed by a self driving car. 🤔
"...What makes this account particularly striking is Krikorian’s background. At Uber’s Advanced Technologies Center, he ran the team building autonomous vehicles and trained human safety drivers on exactly when and how to intervene when a self-driving system fails...."
🤔
LOL this is the problem with relying on AI tools, as well...
"...His core argument: Tesla is asking humans to supervise a system that is specifically designed to make supervision feel pointless. As he puts it, an unreliable machine keeps you alert, and a perfect machine needs no oversight, but one that works almost perfectly creates a trap where drivers trust it just enough to stop paying attention.
The research backs this up. Psychologists call it the “vigilance decrement”, monitoring a nearly perfect system is boring, boredom leads to mind-wandering, and drivers need 5 to 8 seconds to mentally reengage after an automated system hands control back. But emergencies unfold faster than that...."
Johanna, CanCon variety@ai6yr every time
This publication comes to mind:
https://how.complexsystems.fail
As does a Human Factors lecture I attended last century (ugh) on the amount of money spent on psychological research to make fighter plane cockpits human-goof-proof, ON TOP of the extended, intense, and repeated training pilots go through.
One of the points in the early 90's was cars were becoming too complex for mere untrained humans to cope with, with next to no thought about the human-tech interface required.
Shannon Clark@johannab @ai6yr it’s also where standards help and “innovation” breaks muscle memory and consistency. Cars have always had quirks and differences but increasingly their user interfaces are becoming so different between makes sometimes in small until it causes a crash ways
- I have two cars (a Volvo and a Kia) their interfaces do some things exactly opposite of each other (one you push up to control the windshields the other you push down) that’s minor
More major - their safety systems differ
Kelsey Jordahl@ai6yr @Rycaut @johannab Teslas (at least the ones I have seen and driven) do have a brake pedal, but you can set them to drive like a golf cart, where taking your foot completely off the "gas" pedal will bring it to a complete stop. Pressing the brake pedal will always engage the brakes AFAIK. I have a different brand EV and it can also do this (though I choose not to).
A related problem in this "paradox of automation" discussion is that a lot of these techbros got in their imaginations that they should completely redesign the entire cockpit. They think they can "optimize" and "be more efficient" by moving or removing manual controls, since "self driving" means they're not needed.
IIRC, there has been more than one forensic investigation where people died in Teslas not in the crash, but because they couldn't get out of the fire.
I think any innovative genius who thinks they can completely redesign an entire human-tech interface like a car cockpit should first have to prove themselves by getting the entire world to adopt a new non-qwerty keyboard within two fiscal years and demonstrate through replicable peer-reviewed research that typographical errors no longer happen so they can safely remove the backspacer.
Then I'll listen to their ideas about door handles or signal levers or brake pedals.
Purrbiscat@johannab @ai6yr @kajord @Rycaut I just do not understand the need to remove all manual controls. HUGE aircraft, such as the Boeing 777 and the Airbus 340 and many others, are programmed to fly from runway to runway, including takeoff and landing. The hard- and soft-ware must pass extreme testing before being used in live situations. Yet, the cockpit designers didn't remove all the manual controls. They understand that not every flight goes smoothly. While landing these behemoths manually is difficult, the pilot and co-pilot are highly trained to do exactly that.
The hubris of these techbros thinking their "AI", that has repeatedly FAILED simple testing, let alone any rigorous testing, is up to self-driving at all is incredible. And taking away the controls for manual driving is the epitome of stupidity.... and these guys are designing cars?
@silver_buttercat @johannab @ai6yr @kajord they really are stupid - and unaware of how real people live and need their vehicles.
I argue with a friend who is a huge proponent of self-driving cars/taxis (he seriously argues that no one will drive themselves in the future) - he doesn't seem to at all understand how parents use cars - or even simple stuff like the fact that I almost NEVER leave the house for a single destination - that as a working parent I'm almost always making multiple stops
@silver_buttercat @johannab @ai6yr @kajord so to take this morning - I left the house, dropped my son off at school, then went to run an errand at the grocery store (waiting in line until they opened - bought the latest "bag drop" at Trader Joes for my wife + did some grocery shopping) put my groceries in my car, walked over to get myself coffee & breakfast, and only then came home. And that's relatively simple compared to most household errands I run which involve more stops & bags
@silver_buttercat @johannab @ai6yr @kajord but when my son was much younger there was zero chance "self-driving taxis" would have been a viable option for us as parents. Starting with the need for safely, correctly installed car seats (which doesn't go away with self-driving cars - babies & toddlers still need special seats that have to be installed correctly to actually function to protect them in case of accidents) plus the wipes/diapers/bags/emergancy food/toys we kept in the car at all times
@silver_buttercat @johannab @ai6yr @kajord and my son may have been atypical but at times it would take 45 minutes or longer to get him to get in (or out) of the car when he was being a stubborn toddler and didn't want to go home from preschool.
Or when he was a baby or toddler he would fall asleep in the car and we would either keep driving or pull over and let him sleep - in either case not something we would have done in a driverless vehicle (imagine one driving off with a sleeping baby)
@Rycaut @silver_buttercat @johannab @kajord Not atypical... getting kids in and out of cars was a PITA when they were that age. And (I know, I know burning gasoline) at some points we w would sometimes take a long unnecessary drive because the kids would zonk out and sleep while on a drive. There's probably a better, eco-friendly way to do that, lol.
@ai6yr @silver_buttercat @johannab @kajord yup - we bought a hybrid that could be all electric for short bursts when my son was young - helped mitigate the damage from those long unnecessary drives (which did introduce me to some beautiful parts of the Bay Area I might not have otherwise seen) This was nearly a decade ago - if I had a young child now I'd definitely have an EV (just not a Tesla - I love our Kia EV6 - fantastic vehicle)
@Rycaut @silver_buttercat @ai6yr @kajord
oof, thanks for that nightmare fuel. I went through a severe bout of postpartum anxiety where night terrors and intrusive thoughts mostly fixated on something happening to my daughter because she was stuck in her carseat. 😬
pls don't worry about triggering that, it's a decade past resolved, but I mean YES I GET THAT WORRY AND HOW DO PEOPLE THINK IT OK TO LET THESE THINGS LOOSE IN THE WORLD.
@johannab @silver_buttercat @ai6yr @kajord Sorry in any case - it is indeed nightmare fuel (and perhaps I should have added a CW retroactively) but yup, I really don't think the people building driverless cars/delivery robots have really thought about how real people especially parents (and in the case of deliveries elders or heck anyone living in a multiunit building) live. And they don't get that a school bus driver's real job isn't (just) moving a vehicle from place to place
bjb 
