nixCraft 🐧The Tea app is vibe-coded and has over 4 million users. Users' private data was breached, and all government-issued verification IDs, photos, and private messages were leaked. This is why you need a good team of experienced developers, IT and infosec folks to secure your infrastructures. You can’t just vibe code using AI and get away with poor jobs. Now they are going to use the same shity AI in government and everywhere and we will see more such things. What a mess
Look, Even with a highly experienced team of human developers, things can go wrong in security world. AI, however, has no concept of privacy and struggles with basic distinctions, like differentiating "A" from "B." Security is already incredibly challenging. The fact that everything was stored, unencrypted, in a publicly facing database is astonishing. That seems like it should be considered criminally negligent. Sadly this is a reality now as they keep firing real humans in favour of AI 🤖
It wouldn't be surprising if Apple and Google started showing warnings like, "This app was created using AI, and there's a high chance of your privacy and data being leaked," before users download such applications. I know Google likely wouldn't do it, but Apple just might 🤭
Apostolos GiokasDon’t do what? Warn the users?
mikeTesteLinuxQlub@nixCraft New paid service: check if the app is AI generatid, wont stop to steal from ours pockets, don't they?
Nafeon the Bear@nixCraft but even if they warn the users. People will still download it, because there isn't an alternative to some apps. Self regulating capitalism is like that...
cid_terronMy predextion for the EU :
Apple will show the warning and increase the share you have to pay for AI apps...
Google will hide the apps
Sweeney will sue both
tipjip@nixCraft
Things will get interesting when AI companies are held liable for the shoddy work of their products. Kind of similar to the question of liability with self driving cars.
AI in its current form is not economically viable in the long term.
Matt Palmer@tipjip what leads you to believe that there will ever be meaningful liability for any consequences of poor quality LLM output?
@womble
Nobody wants to take responsibility for fucking up. So someone will sue someone else.
That may end in the producers or users being held liable. Or it might end in a change of law. Or it will be the end customers' problem. But no matter what, the cost will become visible and be factored into purchase decisions.
That makes me believe that there is a reasonable chance that poor quality LLM output will have consequences either from the courts or from the market.
@tipjip the counterargument, though, is the last 40+ years of absolutely appalling software quality (both in a security sense, as well as general fitness for purpose) has not caused any of what you claim will happen due to LLM-derived software.
@womble But is the situation still the same? Up till now software was a tool and responsibility on how to use it was yours. If you bought the wrong tool that was on you. Now you could argue that you bought a machine that produces something. But the products are defective due to the design of the machine.
I do not not disagree with you. I just think that the discussion we are having here will also happen on a larger scale and - hopefully - have consequences.
Alan@nixCraft so we need a vibe coded app that grinds through breached data to find info about our legislators and send it to them? Or to the media or both? 🤔
Schenkl | 🏳️🌈🦄@nixCraft Open Data!!!!1!!11
Mark Lynch@nixCraft vibe coding = shitehawks with no respect for expertise wanting to make money from a skill they haven’t taken the time to learn and thinking “how hard can it be?”
As it happens, quite hard, and hopefully they get sued even harder
DavyJones@nixCraft The other lesson we need to learn is that any entity you are interacting with online could be doing something similar. If you are posting personal data online you have to assume a significant probability of it being released publicly.
Knut 🏳️🌈 🇳🇴🧸@nixCraft the tea app thing? Vibe coding is garbage. AI doesn't take pride in its work. Toil away at improving it and being the best it can be. Only a real #programmer can do that. On a real programmer has passion. #ai will never have that.
@nixCraft Open Data for everyone!
Mike J👹🐀 🤘🏻@schenklklopfer @nixCraft Information wants to be free!
Suzanne@nixCraft AI is like asbestos. Put it in everything, damage people, first ignore that for a long time, then spend decades to safely remove.
*edit: now this post gets boosted so much I feel obliged to say my 13y old son came up with the analogy some time ago. But I do really agree.
Andy H@suzannespirit @nixCraft Love this analogy!
Spitfire@suzannespirit I am saying this for while now, too. It'll be hard and painful to pry all that LLM junk out of our systems in the future I'm afraid :/
Matteꙮ Italia@nixCraft TBH you don't even need to have a "team" of infosec professionals to tell you that your DB shouldn't be exposed publicly on the internet, especially if it includes personal data of your users; a half decent programmer should be able to reach this conclusion.
@nixCraft BTW is there some first hand source on the vibe coding claim? It feels absolutely plausible, but from a quick search I couldn't find a good source on this (and OTOH it wouldn't be the first time to have data breaches due to gross incompetence of human coders who just left S3 buckets open to everyone).
Takiro 🎨@nixCraft
I still think it isn't a consequence of vibe coding but the actual goal.
I still think it isn't a consequence of vibe coding but the actual goal.
Jesse Skinner@nixCraft I tried vibe coding to migrate a legacy app from PHP to JavaScript. Claude 4 reproduced the php session cookie as a user_id cookie !!!
🪨@nixCraft Dude, you have a large following, you need to fact check the things you post.
The Tea app doesn't have 4 million users, it has 1.6M according to their website. And the leak is over 72k images, which only a small portion of those users, not "all" government issued IDs as you claim.
And there is no evidence of vibe coding, in fact the Tea app existed before vibe coding existed so this is just not possible. It's a classic unsecure Firebase app, that happened before vibe coding and will continue to happen.
The Tea app doesn't have 4 million users, it has 1.6M according to their website. And the leak is over 72k images, which only a small portion of those users, not "all" government issued IDs as you claim.
And there is no evidence of vibe coding, in fact the Tea app existed before vibe coding existed so this is just not possible. It's a classic unsecure Firebase app, that happened before vibe coding and will continue to happen.
rozodru@nixCraft I'm a consultant/freelancer and within the past like 3 to 4 months I've been getting contacted by a lot of places to essentially fix "vibe coder" slop. The closest thing I can compare it to is when several years ago companies were offshoring work to India and months to years later they need people to also come in and fix things that they offshored. History repeats itself.
The tech debt from what I'm seeing right now is massive. insanely massive. There's no refactoring here - this is "we need to rebuild from the ground up cause all this AI slop can't scale" Like I said most of my requests now are to essentially be a digital janitor for vibe coders so much so I'm having to now reference other devs I know to these places.
Honestly if you freelance as a dev be it front end, back end, software, whatever SERIOUSLY start advertising on linkedin or where ever saying you can clean this shit up. there are SO many places that need rescuing but are way to embarrassed to publicly admit they need saving from vibe coding. Just make a post on Linkedin and I guarantee you messages will start flooding in.
Okuna@Okuna @nixCraft yup, I remember those days well. The problem is now no one is doing quality checks on anything a vibe coder prompt monkey spits out. the "vibe coder" is the one that is supposed to be doing the checks and they're not. OR they "are" but they just don't have a clue what they're checking because they don't know it.
They can write a decent prompt, but they just hit tab and hope the AI knows what it's doing. The problem is the AI simply needs to provide a solution. doesn't matter if it's correct or not just as long as it provides something.
As an example looking over diff's of one place i'm currently consulting for they were "tabbing" on things that were simply removing a couple words from a comment and the AI would call that "a fix". the "dev" or vibe coder would then push this to production. Literally one commit had a diff of "-- //this is" and it got PUSHED with the commit message of "fix for such and such function" IT WAS THE REMOVAL OF TWO WORDS FROM A COMMENT!
@rozodru @nixCraft I am in this business for more than 30 years. I have not seen once, that the long run or the technical dept was put into consideration. All projects I led, I asked for a technical writer and budget for IT Security. These were the first which got cancelled in the budget.
Then, several years later, in one company they had to make a big project to update their code because they got hacked. Cost much more than doing it right from the beginning. A friend of mine phrased it perfectly:
“There is never enough time to do it right,
But always enough money to fix it.”
Once I told the boss of my boss: Going live with this release will kill us latest 2 years. His reply: If you are still in this position in two years, you have a problem with your career.
Then, several years later, in one company they had to make a big project to update their code because they got hacked. Cost much more than doing it right from the beginning. A friend of mine phrased it perfectly:
“There is never enough time to do it right,
But always enough money to fix it.”
Once I told the boss of my boss: Going live with this release will kill us latest 2 years. His reply: If you are still in this position in two years, you have a problem with your career.
fedops 💙💛
Stéphane Charette 🇨🇦
Vassil Nikolov | Васил Николов
Rubén Rubio@rozodru @nixCraft Nice advise! I have been cleaning code written by a junior dev using LLM. It is worse than if he had written it by himself.
I think I am going to start advertising trainings on how to develop without AI. There will be a huge need of developers that will need to learn to write software by themselves.
Nicolas Rinaudo
Badly-read literary snob@NicolasRinaudo @rozodru @nixCraft indeed it does! I was about to quote toot! This post made me think that I really enjoy getting on a pile of crap and figure out things!
@RosaCtrl @rozodru @nixCraft I worry that this won't be quite as fun. I really enjoy inheriting a code base that's "bad" because the developers were rushed or inexperienced. These code bases are still planned out - even if the plan was flawed, even if it wasn't always followed, even if there was more than one competing for supremacy.
Vibe coding is not that. There is no plan.
@NicolasRinaudo @rozodru @nixCraft yeah, but if refactoring isn’t enough and rewrites are necessary then we would be back to how things were a few years ago, right? That may not be ideal, but good enough
@NicolasRinaudo @rozodru @nixCraft yeah, but if things are that bad, I feel actual engineers will have leverage.
But honestly I’m more worried about the effect these things will have on the field once the delusion of vibe coding fades away. I’ve never been forced to use and specific IDE, for example, but I can imagine being forced to use agents on the hopes I’ll become faster, and that’s what makes more anxious
The Agender Kiwi
Solinvictus 
@nixCraft reminds me of this one
RejZoR@nixCraft This reminds me of one time I got notification about a breach and I didn't recognize the service and why would it have my data. Turned out it was some "anti fraud" service used by god knows what online shop I've used and they had a breach. So cool when your data gets leaked by some 3rd party that's allegedly protecting you from fraud and you're not even aware they are processing your private data.
Parsingphase@nixCraft Where’s the source for this being vibe-coded?
Falling forward 🍉@parsingphase @nixCraft
This is what I was going to ask.
And does vibe coding now include the privacy settings for a database in production? Because "coding" and production deployments are two different things.
�@parsingphase @nixCraft This would be too easy. If you suck at your job, blame AI?
noplasticshower
Antonio J. Delgado❤️🧡💛💚💙💜@nixCraft what has been puzzling me is the data protection related to people that are not their users. So a woman report a man, send a photo, maybe name and city. This is enough information to identify the men. So it's personal data, of someone that it's not a user of the platform, and can't access the platform or, request taking down information about them? And then they might also leaked that data with opinions about him and second hand facts.
Niels Moseley@nixCraft the trouble with this is that often one doesn’t know how apps are coded or how capable the developers are.
FossThought@nixCraft Or just stop cramming everything into databases! If you don't collect the data in the first place, there's nothing to hack.
lemgandi@nixCraft Also cf:
Two Major AI Coding Tools Wiped Out User Data After Making Cascading Mistakes - Slashdot
An anonymous reader quotes a report from Ars Technica: Two recent incidents involving AI coding assistants put a spotlight on risks in the emerging field of "vibe coding" -- using natural language to generate and execute code through AI models without paying close attention to how the code works und...
bezlishke@nixCraft dat awesome
Corbin Davenport@nixCraft I haven’t seen any proof it was vibe coded?
GrumpyDad 🇺🇦🇵🇸@nixCraft Why would they need the IDs?
Luna chan@grumpydad @nixCraft To make sure all users are female which shows the paranioa and toxic mindset behind it.