AI6YR BenFormer Uber self-driving chief crashes his Tesla on FSD, exposes supervision problem
https://electrek.co/2026/03/17/former-uber-self-driving-chief-tesla-fsd-crash-supervision-problem/
**VERY glad the guy and his kids are okay, but it would have been something else if the Uber self-driving chief had been incinerated or killed by a self driving car. 🤔
"...What makes this account particularly striking is Krikorian’s background. At Uber’s Advanced Technologies Center, he ran the team building autonomous vehicles and trained human safety drivers on exactly when and how to intervene when a self-driving system fails...."
🤔
LOL this is the problem with relying on AI tools, as well...
"...His core argument: Tesla is asking humans to supervise a system that is specifically designed to make supervision feel pointless. As he puts it, an unreliable machine keeps you alert, and a perfect machine needs no oversight, but one that works almost perfectly creates a trap where drivers trust it just enough to stop paying attention.
The research backs this up. Psychologists call it the “vigilance decrement”, monitoring a nearly perfect system is boring, boredom leads to mind-wandering, and drivers need 5 to 8 seconds to mentally reengage after an automated system hands control back. But emergencies unfold faster than that...."
Johanna, CanCon variety@ai6yr every time
This publication comes to mind:
https://how.complexsystems.fail
As does a Human Factors lecture I attended last century (ugh) on the amount of money spent on psychological research to make fighter plane cockpits human-goof-proof, ON TOP of the extended, intense, and repeated training pilots go through.
One of the points in the early 90's was cars were becoming too complex for mere untrained humans to cope with, with next to no thought about the human-tech interface required.
Shannon Clark@johannab @ai6yr it’s also where standards help and “innovation” breaks muscle memory and consistency. Cars have always had quirks and differences but increasingly their user interfaces are becoming so different between makes sometimes in small until it causes a crash ways
- I have two cars (a Volvo and a Kia) their interfaces do some things exactly opposite of each other (one you push up to control the windshields the other you push down) that’s minor
More major - their safety systems differ
Kelsey Jordahl@ai6yr @Rycaut @johannab Teslas (at least the ones I have seen and driven) do have a brake pedal, but you can set them to drive like a golf cart, where taking your foot completely off the "gas" pedal will bring it to a complete stop. Pressing the brake pedal will always engage the brakes AFAIK. I have a different brand EV and it can also do this (though I choose not to).
A related problem in this "paradox of automation" discussion is that a lot of these techbros got in their imaginations that they should completely redesign the entire cockpit. They think they can "optimize" and "be more efficient" by moving or removing manual controls, since "self driving" means they're not needed.
IIRC, there has been more than one forensic investigation where people died in Teslas not in the crash, but because they couldn't get out of the fire.
Tesla blamed the victims claiming the doors could still be opened manually and they didn't even "try". But where did they put the manual door releases? I can't remember which model it was in which accident, but the "oops power's off, use the manual door handle" handle was somewhere on the floor of the car by the side of the seat?
That's a standards violating, deadly, design failure.
Tony “Abolish ICE” Arcieri🌹🦀
Gabriel NI think any innovative genius who thinks they can completely redesign an entire human-tech interface like a car cockpit should first have to prove themselves by getting the entire world to adopt a new non-qwerty keyboard within two fiscal years and demonstrate through replicable peer-reviewed research that typographical errors no longer happen so they can safely remove the backspacer.
Then I'll listen to their ideas about door handles or signal levers or brake pedals.
Rob CarlsonTHE WHOLE WORLD, Rob, it's gotta be the optimal solution for humanity, proven. 😆
Best I've managed is to use one of those half-QWERT kbds where the right side keys are typed mirror-imaged on the left. It slows my touch typing to about half-speed with more typos. It's the optimal solution for the population of fast QWERTY touch-typists who only have left arms and don't speed-type competitively.
Same person couldn't drive stick, though!
derptron
Purrbiscat@johannab @ai6yr @kajord @Rycaut I just do not understand the need to remove all manual controls. HUGE aircraft, such as the Boeing 777 and the Airbus 340 and many others, are programmed to fly from runway to runway, including takeoff and landing. The hard- and soft-ware must pass extreme testing before being used in live situations. Yet, the cockpit designers didn't remove all the manual controls. They understand that not every flight goes smoothly. While landing these behemoths manually is difficult, the pilot and co-pilot are highly trained to do exactly that.
The hubris of these techbros thinking their "AI", that has repeatedly FAILED simple testing, let alone any rigorous testing, is up to self-driving at all is incredible. And taking away the controls for manual driving is the epitome of stupidity.... and these guys are designing cars?
@silver_buttercat @johannab @ai6yr @kajord they really are stupid - and unaware of how real people live and need their vehicles.
I argue with a friend who is a huge proponent of self-driving cars/taxis (he seriously argues that no one will drive themselves in the future) - he doesn't seem to at all understand how parents use cars - or even simple stuff like the fact that I almost NEVER leave the house for a single destination - that as a working parent I'm almost always making multiple stops
@silver_buttercat @johannab @ai6yr @kajord so to take this morning - I left the house, dropped my son off at school, then went to run an errand at the grocery store (waiting in line until they opened - bought the latest "bag drop" at Trader Joes for my wife + did some grocery shopping) put my groceries in my car, walked over to get myself coffee & breakfast, and only then came home. And that's relatively simple compared to most household errands I run which involve more stops & bags
@silver_buttercat @johannab @ai6yr @kajord but when my son was much younger there was zero chance "self-driving taxis" would have been a viable option for us as parents. Starting with the need for safely, correctly installed car seats (which doesn't go away with self-driving cars - babies & toddlers still need special seats that have to be installed correctly to actually function to protect them in case of accidents) plus the wipes/diapers/bags/emergancy food/toys we kept in the car at all times
@silver_buttercat @johannab @ai6yr @kajord and my son may have been atypical but at times it would take 45 minutes or longer to get him to get in (or out) of the car when he was being a stubborn toddler and didn't want to go home from preschool.
Or when he was a baby or toddler he would fall asleep in the car and we would either keep driving or pull over and let him sleep - in either case not something we would have done in a driverless vehicle (imagine one driving off with a sleeping baby)
@Rycaut @silver_buttercat @johannab @kajord Not atypical... getting kids in and out of cars was a PITA when they were that age. And (I know, I know burning gasoline) at some points we w would sometimes take a long unnecessary drive because the kids would zonk out and sleep while on a drive. There's probably a better, eco-friendly way to do that, lol.
@ai6yr @silver_buttercat @johannab @kajord yup - we bought a hybrid that could be all electric for short bursts when my son was young - helped mitigate the damage from those long unnecessary drives (which did introduce me to some beautiful parts of the Bay Area I might not have otherwise seen) This was nearly a decade ago - if I had a young child now I'd definitely have an EV (just not a Tesla - I love our Kia EV6 - fantastic vehicle)
@Rycaut @silver_buttercat @ai6yr @kajord
oof, thanks for that nightmare fuel. I went through a severe bout of postpartum anxiety where night terrors and intrusive thoughts mostly fixated on something happening to my daughter because she was stuck in her carseat. 😬
pls don't worry about triggering that, it's a decade past resolved, but I mean YES I GET THAT WORRY AND HOW DO PEOPLE THINK IT OK TO LET THESE THINGS LOOSE IN THE WORLD.
@johannab @silver_buttercat @ai6yr @kajord Sorry in any case - it is indeed nightmare fuel (and perhaps I should have added a CW retroactively) but yup, I really don't think the people building driverless cars/delivery robots have really thought about how real people especially parents (and in the case of deliveries elders or heck anyone living in a multiunit building) live. And they don't get that a school bus driver's real job isn't (just) moving a vehicle from place to place
bjb 
I_give_u_worms@i_give_u_worms @kajord @johannab @Rycaut So, I once heard (in person, at a tech industry cocktail party) one of El*n's best friends and confidants talk about his desire to do just that.
@i_give_u_worms @kajord @johannab @Rycaut I don't remember the whole conversation but I remember that part, because it was a conversation about the inconveniences of pedestrians being in their way, and how it ought to be legal to run over the homeless, and anyone who was poor (they specifically mentioned "gardeners and housekeepers" I recall) because they "have no net worth" and "are worthless to society". And this was ostensibly someone the industry looks up to (even now).
Bekah@ai6yr This was the person speaking’s own opinion, or his relay of E’s desires?
Whoever holds this opinion should just buy an island (presumably without gardeners and help) and leave the rest of us alone.
@bolderbekah @i_give_u_worms @kajord @johannab @Rycaut Ah, yes, and that's how you get to Epstein's Island.
@johannab @kajord @ai6yr the lack of manual door controls in the back seats of Teslas some models have hidden as in you need to read the manual ways to open a door in back in case of a power loss but some models don’t have any way to force the read doors open is a huge reason as a parent I never considered a Tesla. I don’t want to have to try to teach a young child how to open a door in a very complex manner in an emergency.
Suzanne Aldrich (she/her)
John@johannab @kajord @ai6yr @Rycaut there’s an excellent report on the subject by Bloomberg’s that really drives the point home:
The Dangerous Feature in Tesla's Doors
@johannab @kajord @ai6yr and rescuers haven’t been able to open to doors from the outside to help people.
It’s horrible design - and deeply problematic to not design for the worst case situations and have that inform design choices. The bar to switch from very well established controls (mechanical door handles) should be exceptionally high. Flush handles arguably helps aerodynamics but it was an active choices to complicate doors
@Rycaut @kajord @ai6yr to give Tesla (not melon husk, the real actual designers and engineers) the teeeeniest benefit of the doubt, their oversights are a consequence car-culture capitalism.We have never in the slightest way considered the safety of people outside the car, or even occupants inside the car at any time a catastrophic deceleration was not happening.
Regulation that places the dollar not the human as the subject beneficiary is evil.
Nazo