Today a PM on my team showed the team a fancy dashboard they had built with AI on my request. Questions from other PMs

Q: How is the data stored?
A: I don’t know.

Q: Where is this deployed, do you have a server?
A: I don’t know. I asked Claude to deploy it and it did.

This is the future.

This is how I must sound to car enthusiasts asking me if my engine is a v8.
@carnage4life I think it has… an engine? In the front somewhere? No tomato juice that I know of.

@carnage4life 🤦‍♂️

Blue team is going to be so painful.

@patmikemid @carnage4life I guess switching teams is overdue…

#sarcasm

@carnage4life That sounds like a security nightmare.
@bryanredeagle @carnage4life after decades of improving IT security practices we just don't do it anymore
@carnage4life It's all good fun until something goes wrong #AI

@jillL @carnage4life Claude will only too happily reclaim something it deployed 6 weeks ago and now that dashboard is toast.

All this jazz over written language prompts being the future only leads me to ST:TNG Moriarty episodes when LaForge accidentally creates a Sherlock Holmes program capable of defeating *Data*.

Whoopsie.

@carnage4life Wow that really doesn't seem acceptable. Can Claude not make pull requests and defend itself from a critical reviewer?
@carnage4life To be fair, as a client engineer, this is usually how I treat the backend as well. 😆
@carnage4life makes me not worry about Skynet

@carnage4life that’s…concerning

On more than one level

And not just on the part of the PM

@carnage4life Imagine asking these same questions to an engineer.

How does it work?
I don’t know

Where is the documentation?
There is none

This would be very concerning but here, it’s an AI so it’s amazing!

It is not. That dashboard will never be updated, no one understand how it works. How can you even tell the data it presents is accurate?

The future…
@carnage4life That's like asking someone who made your food about what is in it and they go " I dunno. Food stuff?"

That's pretty much the answer you'll get in most restaurants. They really have no idea what exactly goes into the food they're serving you.

@romabysen @carnage4life

@EverydayMoggie you can not seriously think that chefs don't know what is in the food.
@romabysen @EverydayMoggie try asking the wait staff for allergy information next time you are in a restaurant

I do mean that. Maybe in high-end restaurants they might know, but in an everyday restaurant, much of the food includes manufactured ingredients. For example, I've had to ask what kind of oil is used in the food due to allergies, and most places couldn't be more specific than "vegetable oil."

I've basically stopped eating in restaurants because of this problem. (I have additional allergies, so it isn't just the oil.)

@romabysen

@carnage4life

They don't know how or where it is deployed? So, they have no idea how much the deployment is costing .... interesting way to conduct business.

@bjb @carnage4life or who else has access to the data. Including locality requirements for e.g. EU citizens.

Also, if you don't know where the data is, I'm guessing it's not backed up.

Somebody is going to make a killing on all this, but I'm not sure we know whom.

@carnage4life @drwho That sounds like a quite spectacular data privacy nightmare.
@Em0nM4stodon @carnage4life And a disaster recovery nightmare.
@carnage4life it is as it should be
@carnage4life i loved telling the juniors at my old job, if it starts working for no reason, it's gonna stop working for no reason when it's least convenient
@carnage4life
Q: if we find out the data is wrong, will we actually be able to fix it?
A: No, management will get upset that correcting the numbers will put bumps in their trend lines
@carnage4life everything is good until you have to understand all of it
@carnage4life all this talk about shifting privacy left, and the industry really prefers to just ignore privacy entirely ><

@carnage4life for the record, the situation you describe is likely illegal in jurisdictions with GDPR (we're not a lawyer though)

GDPR really only creates some very basic requirements to know what your system actually does. it's very frustrating that that's too much for so many companies.

@carnage4life move fast and break things 🫠
@carnage4life absolute clown show 🤡🤡🤡
@carnage4life it's the return of the enterprise critical Microsoft Access app that lives on someone's desktop under a desk.
@carnage4life As the kids say, "Thanks, I hate it."