Raghav AgrawalApr 4

RE: https://mastodon.social/@impactology/116346202875671550

Pipes & valves skids for pharma and food. or machining tools, conveyer belts for appliances. Basically making the tooling for factories. This is real engineering, process engineering

Orders of magnitude more complex than tooling for making software

If level of rigour for quality testing used for Clean-In-Place (CIP) piping skids of a biologics facility was used for AI software tooling, imagine the kind of software reliability we'd have instead of whatever the fuck today's genai tools spit out

Show threadRaghav AgrawalApr 4
Real engineering deals with the laws of thermodynamics and material science. things that don't have bugs you can just patch later. Once something breaks or is contaminated you can't just be like we'll tackle this later.
Show threadRaghav AgrawalApr 4

It's funny

There are industrial automation companies like abb robotics and fanuc with engineers and researchers doing much more rigorous research and engineering work with a much higher bar of safety than AI companies of silicon valley and none of their researchers do the cult like complexity theatre performance that researchers at these frontier model labs do

Show threadNot a Spring OnionApr 4

@impactology
For industrial robotics, workers had to die first before someone had the idea to put that robot arm in a cage.

Accidents still happen, e.g. because two maintenance engineers decide that one goes inside the cage and work on the robot arm while the other tests the controller outside...

Show threadRaghav AgrawalApr 4

@wakame

For industrial robotics, workers had to die first before someone had the idea to put that robot arm in a cage

Woah really?

Show threadNot a Spring OnionApr 4

@impactology

First fatality in 1979:

https://en.wikipedia.org/wiki/Robert_Williams_(robot_fatality)

Robert Williams (robot fatality) - Wikipedia

Show threadNot a Spring Onion

@impactology

And this is the last case I heard of (the one I hinted at in the previous post):
https://www.dw.com/en/robot-kills-worker-at-volkswagen-plant-in-germany/a-18556982

IIRC, the "typical thing" happened:
Let's split up the work, let's to things in parallel. Like a dozen times before. That robot is likely not even connected to power.
Etc.

Volkswagen: Robot kills worker installing it

Volkswagen has disclosed that a robot has killed a contractor involved in its installation. The fatal accident happened at VW's Baunatal plant, north of Frankfurt on Monday.

Deutsche Welle
Apr 4 at 2:26pmWeb
Show threadNot a Spring OnionApr 4

@impactology
IMHO we have a lot of safety stuff in industrial production now because:

a) Dead workers make bad publicity.
b) Laws.
c) Implementing safety is cheaper than paying for the insurance otherwise.

After the Boeing crashes due to bad software, I don't believe a) and b) are going to help us with AI.

I hope for insurance companies to realize what kinds of epic damage an AI tool without oversight can do.

And that "we used vibecoding like everyone else" isn't an excuse that a judge will accept.

Show threadRaghav AgrawalApr 4
@wakame I think vibecoding will mostly remain a demo-ing, presentatation ware tool. For people to present an idea at stage 1, that's it
Show threadNot a Spring OnionApr 4

@impactology
I know of at least two tools being used in production that were vibecoded.
And that's just my small corner of the world.

Of course no manager will say: "Just generate something and deploy it."
The resourceful employee will just do that by themselves.

Show threadRaghav AgrawalApr 4

@wakame

https://mastodon.social/@impactology/116346981818073311

What's the orbital welding equivalent level of care in software making