AI6YR BenFormer Uber self-driving chief crashes his Tesla on FSD, exposes supervision problem
https://electrek.co/2026/03/17/former-uber-self-driving-chief-tesla-fsd-crash-supervision-problem/
**VERY glad the guy and his kids are okay, but it would have been something else if the Uber self-driving chief had been incinerated or killed by a self driving car. 🤔
Scott P ✅LOL this is the problem with relying on AI tools, as well...
"...His core argument: Tesla is asking humans to supervise a system that is specifically designed to make supervision feel pointless. As he puts it, an unreliable machine keeps you alert, and a perfect machine needs no oversight, but one that works almost perfectly creates a trap where drivers trust it just enough to stop paying attention.
The research backs this up. Psychologists call it the “vigilance decrement”, monitoring a nearly perfect system is boring, boredom leads to mind-wandering, and drivers need 5 to 8 seconds to mentally reengage after an automated system hands control back. But emergencies unfold faster than that...."
SupermoosieSupermoosie (@[email protected])
Tesla is asking humans to supervise a system that is specifically designed to make supervision feel pointless. As he puts it, an unreliable machine keeps you alert, and a perfect machine needs no oversight, but one that works almost perfectly creates a trap where drivers trust it just enough to stop paying attention. The research backs this up. Psychologists call it the “vigilance decrement”, monitoring a nearly perfect system is boring, boredom leads to mind-wandering, and drivers need 5 to 8 seconds to mentally reengage after an automated system hands control back. But emergencies unfold faster than that. https://electrek.co/2026/03/17/former-uber-self-driving-chief-tesla-fsd-crash-supervision-problem/
@SuperMoosie They only give control back as you are about to die
M.S. Bellows, Jr.@ai6yr @SuperMoosie ... so they can avoid liability by pretending that the crash is the driver's fault, not the car's.
@msbellows @SuperMoosie Yeah, funny how that system works.
Russell@msbellows @ai6yr @SuperMoosie The NHTSA rules for collecting crash data includes if an autonomous system was active within 30 seconds of the time of impact.
"Reporting Requirements" https://www.nhtsa.gov/laws-regulations/standing-general-order-crash-reporting
Becca@ai6yr someone has to take the fall and it won't be tech bros
Tim HergertCall this the #JustinTrudeauManeuver
Take credit while things are smooth sailing but pull the ripcord and bail when things look like they're about to get a little bit spicy.
Ehay2kFull Self Doom.
Mark Whybird@ai6yr It’s the exact opposite of the Tullock's spike thought experiment.
“…in which Tullock suggested that if governments were serious about reducing road casualties, they should mandate that a sharp spike be installed in the center of each car's steering wheel, to increase the probability that an accident would be fatal to the driver. Tullock's idea was that the normal process of risk compensation would then lead to safer driving by the affected drivers, thereby actually reducing driving fatalities.”
https://en.wikipedia.org/wiki/Gordon_Tullock?wprov=sfti1#Tullock's_spike
Jim Hubbell
Johanna, CanCon variety@ai6yr every time
This publication comes to mind:
https://how.complexsystems.fail
As does a Human Factors lecture I attended last century (ugh) on the amount of money spent on psychological research to make fighter plane cockpits human-goof-proof, ON TOP of the extended, intense, and repeated training pilots go through.
One of the points in the early 90's was cars were becoming too complex for mere untrained humans to cope with, with next to no thought about the human-tech interface required.
Paul CantrellClassic among classics.
Also, there’s a 99pi about exactly this: https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/
Children of the Magenta (Automation Paradox, pt. 1) - 99% Invisible
On the evening of May 31, 2009, 216 passengers, three pilots, and nine flight attendants boarded an Airbus 330 in Rio de Janeiro. This flight, Air France 447, was headed across the Atlantic to Paris. The take-off was unremarkable. The plane reached a cruising altitude of 35,000 feet. The passengers read and watched movies and slept.
@inthehands @ai6yr oh, cool, thanks! I love 99pi, except for the fact that their back-catalogue is longer than I have years left to live, I suspect. I've listened on-and-off for over a decade and they had quite the archive when I started!
Raglan Niall 
@inthehands @johannab @ai6yr that two-parter was awesome. While the tech may have improved between then and now, no decent solution to the fundamental problem of paying attention/ being ready for failing automation has been proposed.
No kidding, I had apparently actually listened to those ones but forgotten, and now I've listened again AND need to pile those into my "slow the fuck down with AI in everything" references pile. "automation paradox" is being baked in to the whole stack right now. Particularly terrifying when I think of my previous medical systems roles, because we seem to have dropped any idea of regulating life-endangering tech, too.
@johannab @Niall @ai6yr
Yeah. In multiple spheres, I’m increasingly resigning myself to “people are going to have to learn it for themselves” mode — shifting focus from global prevention to local mitigation, away from trying to control others and toward protecting what’s already in my sphere of personal control.
In my software consulting days, I often found myself trying to get companies not to hit themselves in the head with a hammer, but often it turned out to be best to just go ahead and let them do that and then ask “How’d that work out for you?” It’s painful to see in advance the needless damage, the waste, but sometimes it’s the only thing that works.
Shannon Clark@johannab @ai6yr it’s also where standards help and “innovation” breaks muscle memory and consistency. Cars have always had quirks and differences but increasingly their user interfaces are becoming so different between makes sometimes in small until it causes a crash ways
- I have two cars (a Volvo and a Kia) their interfaces do some things exactly opposite of each other (one you push up to control the windshields the other you push down) that’s minor
More major - their safety systems differ
@johannab @ai6yr I’ve caught myself driving my Volvo expecting it to beep if a car is in my blind spot when I’m merging or making a turn.
It doesn’t have that safety feature my Kia does. But if I’ve been driving the Kia for a few days in a row then drive the Volvo for a while I instinctively expect that feature to be there.
Which is to say it’s easy to get complacent for a few moments - and that’s all it takes to have an accident
CaseyL
Bekah@CaseyL I was resistant to buying one with backup camera and lane assist etc, because people get dependent on them and stop looking around.
But I gotta say the blind spot sensor is lovely even though I drive a sedan and not a giant car (just a light on the side mirror, it might beep if I actually tried to merge over, I’m not sure).
I still drive defensively from years of driving a two seater that fit into everyone’s blind spot even when SUVs were smaller.
Nazo@nazokiyoubinbou @Rycaut @johannab Yeah, so how "one pedal mode" works, is they go when you press the pedal, and if you let go of the pedal it stops the car. You don't hit a brake pedal. But, that trains people to NOT PRESS THE BRAKE PEDAL in other cars if they switch vehicles.
kajer :notverified:@nazokiyoubinbou @ai6yr @Rycaut @johannab there is a "zero" point at about 1/4 the pedal travel. That is coasting / no braking. 0 throttle input is full Regen braking.
This also activates brake lights even though your speed doesn't change that much. You ever follow a Tesla where they CONSTANTLY flash their brakes? One pedal mode and poor foot work.
@kajer @ai6yr @Rycaut @johannab I have never followed one, no. I only see them flying up in my rear view mirror and then missing me by 0.3cm as they swerve without a blinker at the last possible instant.
I just remembered I've done this before in a game at least. I had a wheel for my computer and something detected it wrongly, refusing to see the brake axis and treating the throttle as a combined brake+gas like this with negative being braking and positive being throttle.
It was horrible. I felt like I was constantly threading a needle and it actually made my foot hurt.
I started to say I can't imagine how they continuously do that, but I suppose they don't. I imagine they have their foot all the way down most of the time, then all the way up at the last possible instant the rest.
Violet Madder@kajer @nazokiyoubinbou @ai6yr @Rycaut @johannab
Oh now THAT is bizarre.
And also rather horrible.
Although, on the other hand I guess that ends the issue I have ALWAYS despised, where automatic drive cars are constantly trying to scoot forward if you take your foot off the brake, leading in part to me wondering what happens if somebody passes out and the car just keeps moving under power, but also the hugely annoying tendency for people to constantly creep forward at every damn red light and stop sign...
🏴🏳⚧🏴☠@ai6yr @nazokiyoubinbou @Rycaut @johannab
Jesus Christ. one pedal, no buttons, touchscreen for shifting and other controls, terrible door handles? I knew they were death traps but holy shamoley.
@ai6yr @nazokiyoubinbou @Rycaut @johannab
This one time, I was driving a big pickup to transport a couch. Approaching the Bay Bridge in very heavy traffic, a motorcycle split the lanes past me.
A van in front of us abruptly changed lanes right in front of the bike without signaling.
I did not touch the pickup's brake pedal with my foot.
I let go of the steering wheel with my right hand and grabbed for a nonexistent front brake lever in mid-air like I was trying to sympathetically brake on the motorcyclist's behalf.
The bike saw the van move and dodged in time and was fine-- but it was the weirdest sensation, having my muscle memory cross over and kick in like that!
Climate Jenny 2.1@violetmadder @ai6yr @nazokiyoubinbou @Rycaut @johannab Muscle memory is real. I haven’t been driving a stick shift car for a while, but if I need to brake suddenly I tend to automatically slam my left foot onto a non-existent clutch pedal.
@ClimateJenny @ai6yr @nazokiyoubinbou @Rycaut @johannab
It's BAD if I do that and my clutch foot finds the automatic car's brake pedal instead when I'm only trying to change gears.... 😬
AN/CRM-114@flyingsaceur @ClimateJenny @violetmadder @ai6yr @Rycaut @johannab I have never driven a manual transmission car, but my left foot keeps trying to find a clutch anyway when I brake.
Apparently it's infectious. Be careful. 😁
smells of bikes@ClimateJenny I do this so often my friends mostly won't let me drive automatics.
@ClimateJenny @violetmadder @ai6yr @nazokiyoubinbou @Rycaut
I've lived through the experience of learning to drive on an automatic, doing that for almost 15 years, then learning stick because my bf had a manual tx car and then the next one I bought was also manual (both car and bf worked out fine, still married to one of them ...)
Then right before the pandemic he relented on upgrading the now 16 year old car and we got RAV4 hybrid, but my getaway car was still a Yaris with a stick ...
@ClimateJenny @violetmadder @ai6yr @nazokiyoubinbou @Rycaut
So for a couple of years we were both bi-transmissional.
Gave up the older second car partway through the pandemic and just share the RAV now.
The muscle memory can catch on, but it takes WEEKS of regular practice to reorient, in normal, safe, controlled conditions, and while doing it I was cautious-unto-paranoid about not getting into trouble. No highways or hills for 2-3 months, that sort.
YOU CANNOT DO IT IN AN EMERGENCY.
@ClimateJenny @violetmadder @ai6yr @nazokiyoubinbou @Rycaut
Also, other than the stick & clutch vs. automatic, Toyotas of similar vintage are pretty familiar-feeling one to the other. they change things subtly and aesthetically, but don't, like, remove the speedometer from your forward visual field.
Kelsey Jordahl@ai6yr @Rycaut @johannab Teslas (at least the ones I have seen and driven) do have a brake pedal, but you can set them to drive like a golf cart, where taking your foot completely off the "gas" pedal will bring it to a complete stop. Pressing the brake pedal will always engage the brakes AFAIK. I have a different brand EV and it can also do this (though I choose not to).
A related problem in this "paradox of automation" discussion is that a lot of these techbros got in their imaginations that they should completely redesign the entire cockpit. They think they can "optimize" and "be more efficient" by moving or removing manual controls, since "self driving" means they're not needed.
IIRC, there has been more than one forensic investigation where people died in Teslas not in the crash, but because they couldn't get out of the fire.
Tesla blamed the victims claiming the doors could still be opened manually and they didn't even "try". But where did they put the manual door releases? I can't remember which model it was in which accident, but the "oops power's off, use the manual door handle" handle was somewhere on the floor of the car by the side of the seat?
That's a standards violating, deadly, design failure.
Tony “Abolish ICE” Arcieri🌹🦀
Gabriel NI think any innovative genius who thinks they can completely redesign an entire human-tech interface like a car cockpit should first have to prove themselves by getting the entire world to adopt a new non-qwerty keyboard within two fiscal years and demonstrate through replicable peer-reviewed research that typographical errors no longer happen so they can safely remove the backspacer.
Then I'll listen to their ideas about door handles or signal levers or brake pedals.
Rob CarlsonTHE WHOLE WORLD, Rob, it's gotta be the optimal solution for humanity, proven. 😆
Best I've managed is to use one of those half-QWERT kbds where the right side keys are typed mirror-imaged on the left. It slows my touch typing to about half-speed with more typos. It's the optimal solution for the population of fast QWERTY touch-typists who only have left arms and don't speed-type competitively.
Same person couldn't drive stick, though!
derptron
Purrbiscat@johannab @ai6yr @kajord @Rycaut I just do not understand the need to remove all manual controls. HUGE aircraft, such as the Boeing 777 and the Airbus 340 and many others, are programmed to fly from runway to runway, including takeoff and landing. The hard- and soft-ware must pass extreme testing before being used in live situations. Yet, the cockpit designers didn't remove all the manual controls. They understand that not every flight goes smoothly. While landing these behemoths manually is difficult, the pilot and co-pilot are highly trained to do exactly that.
The hubris of these techbros thinking their "AI", that has repeatedly FAILED simple testing, let alone any rigorous testing, is up to self-driving at all is incredible. And taking away the controls for manual driving is the epitome of stupidity.... and these guys are designing cars?
@silver_buttercat @johannab @ai6yr @kajord they really are stupid - and unaware of how real people live and need their vehicles.
I argue with a friend who is a huge proponent of self-driving cars/taxis (he seriously argues that no one will drive themselves in the future) - he doesn't seem to at all understand how parents use cars - or even simple stuff like the fact that I almost NEVER leave the house for a single destination - that as a working parent I'm almost always making multiple stops
@silver_buttercat @johannab @ai6yr @kajord so to take this morning - I left the house, dropped my son off at school, then went to run an errand at the grocery store (waiting in line until they opened - bought the latest "bag drop" at Trader Joes for my wife + did some grocery shopping) put my groceries in my car, walked over to get myself coffee & breakfast, and only then came home. And that's relatively simple compared to most household errands I run which involve more stops & bags
@silver_buttercat @johannab @ai6yr @kajord but when my son was much younger there was zero chance "self-driving taxis" would have been a viable option for us as parents. Starting with the need for safely, correctly installed car seats (which doesn't go away with self-driving cars - babies & toddlers still need special seats that have to be installed correctly to actually function to protect them in case of accidents) plus the wipes/diapers/bags/emergancy food/toys we kept in the car at all times
@silver_buttercat @johannab @ai6yr @kajord and my son may have been atypical but at times it would take 45 minutes or longer to get him to get in (or out) of the car when he was being a stubborn toddler and didn't want to go home from preschool.
Or when he was a baby or toddler he would fall asleep in the car and we would either keep driving or pull over and let him sleep - in either case not something we would have done in a driverless vehicle (imagine one driving off with a sleeping baby)