Things I regret: Looking at github issues for the open-source self-driving car conversion kit.

Thousands of people use these things on public roads.

https://github.com/commaai/openpilot/issues/34346

22 Acura RDX - steering disables under 45mph · Issue #34346 · commaai/openpilot

Describe the bug Steering is completely disabled below 45mph on release branch. There is no visual warning saying it is disabled, the car just drives in a straight line until it about to go offroad...

GitHub
Candidate for understatement of the year: "This can be a safety issue if a driver is misled into thinking that steering is occurring"

The most damning evidence I've ever seen for "this device is only used by single people":

If you have a passenger they *also* have to be looking at the road or it sets off the "pay attention" alarm.

Or they can put their head on the dashboard.

Honestly the funniest thing I've seen this year. I know it's only mid-January, but still.
you had ONE JOB
Credit to @m for spotting the nightmare github issue that started all this
@jonty @m this is horrifying
@q @jonty "You can't stand in the way of progress!" I yell, while leaping out of the path of the oncoming vehicle... 
@jonty what the actual fuck have you found‽
@Floppy I just looked at the internet James. This keeps happening.
@jonty see, that's the mistake you're making. That we're ALL making.

@jonty it blows my mind that someone sells a device that sticks onto the windscreen, and people plug it into their car's CANbus and let it drive them around.

10,000 users on the roads, and a disclaimer that says "THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."

I'm beginning to think humans were a mistake, let alone computers.

@Floppy @jonty oh god, that's the geohot bollocks isn't it. "Oh self driving isn't hard I don't know what the fuss is about I can do that in a few hundred lines of code"
@Floppy @jonty See when I used a beta build of Finamp so I could get android auto support for my music, there was no way that it could make my truck go careening into a pedestrian. So it's okay for THEM to just say "ymmv it might not work right yet".

@[email protected] I'm increasingly of the opinion that we made a big mistake in coming down from the trees in the first place. @jonty

#hhgg

@Floppy @jonty tbh it only says this in "licensing", and who reads licenses these days?

This should have never been made public, let alone being sold with all these "new low price - it drives your car for $999!". Or at the very least it should have come with huge warnings everywhere telling that using it on public roads is illegal, not this fig leaf of "you are responsible for complying with local laws" buried in "licensing" on GitHub.

On the first glance it seems that if you buy this $1k device from their website, you won't even visit the page that says this disclaimer. "For research purposes only" is only on the GitHub to serve as legal insurance apparently; they're selling the device to regular car owners, as "purpose built to run openpilot", and nowhere on their website does it say that it's "for research purposes". It's as if you would have bought a new car from a car dealership, and it had "only for research purposes" hidden somewhere deep in the EULA.

The creators of this shit should be prosecuted imo.

@Floppy @jonty oh, let's not get started on what humans do behind the wheel
@Floppy @jonty driving and neural networks do not mesh together, no matter how meaty the neural networks are. the fact that meaty neural networks invented railroads during urbanization specifically and then basically forgot to use them should be all the proof we need.

@Floppy @jonty "It blows my mind that..."

...someone sells a car that exposes steering control on CANbus. That's criminally negligent.

It's criminally negligent for the steering wheel not to be rigidly mechanically coupled to the rack and tie rods.

@jonty jfc and in a RAM?!
@hayles Does what it says on the tin
@jonty lol wtf
@jonty This is like that old "GNU car" meme IRL
@jonty passengers are an edge case. Workaround: remove passenger.
@jonty "This is a disaster, that is the exact scenario I need comma to work in."
@jonty Suicide by airbag mode. 🤦
@jonty There's a part of me that's like "this is pretty bad" but then another part of me that knows that most companies I've worked in have usually been considerably worse at resolving stuff like this in a timely manner than most open-source projects, and is horrified by what this implies about commercial alternatives.
@jonty JFC, you'd think people would be smart enough to run this stuff on private roads, parking lots or closed test tracks, this stuff is absolutely not ready for public roads.
@alextecplayz @jonty I mean you may _want_ to think that. But *gestures at everything*
@alextecplayz @jonty It's hard to reach 45mph at parking lots...
@geert @alextecplayz @jonty the one documented instance of achieving 88 mph in a carpark escaped the small space through the 4th dimension.
Comma.ai President Says Tesla FSD v13 Is “Really Good,” Declares “We Are Buying NVIDIA…

When a netizen tagged AMD's CEO to comma.ai president's X post, he responded by declaring that his company was opting for NVIDIA's GPUs.

Wccftech
@jonty it... it has unit conversion errors. On the speed. JFC. https://github.com/commaai/openpilot/issues/34348
22 Acura RDX - max speed on comma mismatches max speed in instrument cluster - confimred mph/kph error · Issue #34348 · commaai/openpilot

Describe the bug The 2022 Acura RDX shows the max speed on the instrument cluster, plus comma shows the max speed on the screen, but these max speeds do not match. The instrument cluster is a littl...

GitHub
@Floppy @jonty giving "move fast and break things" an entire new meaning
@Floppy @jonty wasn't this how one of the Apollo shuttles went down? Never learn anything from history.

@jonty Oh hell this is going to turn me into one of those, "sorry kids, you're not allowed to be on roads in any form ever again, it's for your own good" dads.

Like . . . people just turning loose killer robots for fun and tinkering with them as they go.

@jonty the MIT all-caps warranty disclaimer doing heavy lifting on this one
@jonty how many people died using this
@max Currently none, but there's only ~4k active units on the road of ~10k they've sold. It's just a matter of time.
@jonty I'm glad shit like that is so illegal in #Germany that this will get people in jail!
@kkarhan @jonty To be clear, it's very illegal in the US too if not properly registered. Which I assume none of these are, because hobbyists aren't going to go through the kind of licensing process that a company would. It's a matter of time before one of them causes an accident and the insurance company of the car they hit sues them for all they're worth.

@simonbp @jonty granted, in Germany cops would instantly pull over someone with some #DIY #SelfDriving and definitiely confiscate the license and impound the vehicle as evidence for a case of "criminal negligence" and multiple violations of traffic code.

  • Cuz not even engineers would get permits to do such experiments on public roads!

@kkarhan @jonty If the cops notice.

The whole point of this device is that it just looks like an ordinary dashcam from the outside. Unless there is an accident, it's unlikely anyone would be pulled over while using this device.

@simonbp @jonty OFC, that's the same with illegally loud mufflers...

Kölner Fahrrad-Cops: Die Stilllegung des Mercedes CL 500 (SPIEGEL TV Reportage)

YouTube

@jonty so their software (!!!) will happily keep running even when a core functionality isn’t responding, and yet they claim to have functional safety?

Either they’re lying about meeting ISO-26262 or automotive control safety standards are a complete joke

@cinebox @jonty What they've likely done is said that nothing in the system is safety critical because it's a level 2 system and the driver can take over at any time. If you make that argument, the only safety critical function is allowing disengagement at any time.
@jonty I saw complaints about compatibility with a non-Tesla EV I was eyeballing, in a video about said EV, and immediately felt like that was a point in the car's favor that you had to actually drive it yourself.

@jonty "THIS IS NOT A PRODUCT." and yet it costs $999????

"THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY" I am sure everyone using it understands that

What an irresponsible thing to do

@danvolchek @jonty yeah, the comma.ai website does not give that impression at all
@jonty i hope the "boston" in the username is just cuz they're a Celtics fan or something, and not because this homicidal asshole is driving through my neighborhood
@jonty So what I'm reading here is that the same modern cars that gatekeep things like heated seats behind a subscription will happily let you connect up a third-party gadget that immediately makes it dangerous to use for everyone.
@jonty https://www.youtube.com/watch?v=6ikxBWUAjmI I'm going to just slam this thing into a car I'm not even familiar with and haven't tested it on!
Putting openpilot in a Rental Car! | Level 2 Autonomy w/ commma 3X

YouTube
@jonty To get hired there you need to complete a technical challenge from their leaderboard or bounties list. The GitHub issues page may be an indicator of how that's working out for them. https://comma.ai/jobs
comma.ai — jobs

@jonty I for one am terrified that “open source self driving car conversion kit” is a thing

@jonty The commercial stuff isn't much better. My car came with lane centering, and it's only usable on the interstate.

My semi has some of this stuff too, but only for automated braking. That being said, it will randomly kill the cruise control with no notice and sometimes it will read an American square interstate sign and decide that it's a round EU one and display the speed limit in KM/H. What makes this worse, is that the system they install now is seemingly untouched and unupgraded over the ones they were installing in 2010.

@jonty yeah but think about all that shareholder value that was created. if a few people lost their grandmother, thats a sacrifice the line will make
@jonty my cousin had one of these things on his dash. “Oh yeah that’s broken, I don’t use it. It was fun to get working though.” 😱

@jonty

I'll be receiving my Comma.ai device today. A few of the bugs you featured are actually problems with the car's feature, not the OpenPilot device. Or the OpenPilot device wasn't installed symmetrically (yes, they should have an offset setting).

The devices do NOT have an operating system; you must load the OS yourself. This is a minor barrier to entry, but at least people can't buy one, and Plug-n-Play-n-Die

In some cars, you must run an exploit (real shellcode) to extract security keys to MiTM the communications.

Huge fucking security risk; my gut says most of these devices can be remotely attacked or, at the minimum, with hardware access, can install software so you can remotely manage the device. With remote device management, you can control the car.

But......If you want to Hack your car or get into CAN-Bus ruckus, comma.ai is a pretty good product. They also do pay you for github code work, I couldn't speak to the value for the work. But I do expect to get paid for a few fixes with the RAV4.

Edit:
I've received my device and got it installed. The device comes with a sort of bootstrapped OS and once connected to WiFi, you can download the OS and have it install. Not much of a Barrier.

The 'Chill Mode' default mode, is only a slightly better manufacture drive assist. When you enable experimental mode, it will detect stop lights/signs, does a lot better.

In either mode, you should be ready to drive; everything makes that clear.

Now that everything is working out-of-box, going to see how they can be hacked, and hopefully find some vulns in the car as well.