L4sBotData poisoning: how artists are sabotaging AI to take revenge on image generators
HejMedDigLet’s see how long before someone figures out how to poison, so it returns NSFW Images
You can create NSFW ai images already though?
Or did you mean, when poisoned data is used a NSFW image is created instead of the expected image?
Definitely the last one!
companies would stumble all over themselves to figure out how to get it to stop doing that before going live. source: they already are. see bing image generator appending “ethnically ambiguous” to every prompt it receives
The Nightshade poisoning attack claims that it can corrupt a Stable Diffusion in less than 100 samples. Probably not to NSFW level. How easy it is to manufacture those 100 samples is not mentioned in the abstract
Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Data poisoning attacks manipulate training data to introduce unexpected behaviors into machine learning models at training time. For text-to-image generative models with massive training datasets, current understanding of poisoning attacks suggests that a successful attack would require injecting millions of poison samples into their training pipeline. In this paper, we show that poisoning attacks can be successful on generative models. We observe that training data per concept can be quite limited in these models, making them vulnerable to prompt-specific poisoning attacks, which target a model's ability to respond to individual prompts. We introduce Nightshade, an optimized prompt-specific poisoning attack where poison samples look visually identical to benign images with matching text prompts. Nightshade poison samples are also optimized for potency and can corrupt an Stable Diffusion SDXL prompt in <100 poison samples. Nightshade poison effects "bleed through" to related concepts, and multiple attacks can composed together in a single prompt. Surprisingly, we show that a moderate number of Nightshade attacks can destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images. Finally, we propose the use of Nightshade and similar tools as a last defense for content creators against web scrapers that ignore opt-out/do-not-crawl directives, and discuss possible implications for model trainers and content creators.
yeah the operative word in that sentence is “claims”
I’d love nothing more than to be wrong, but after seeing how quickly Glaze got defeated (not only did it make the images nauseating for a human to look at despite claiming to be invisible, not even 48 hours after launch there was a neural network trained to reverse its effects automatically with like 95% accuracy), suffice to say my hopes aren’t high.
You seem to have more knowledge on this than me, I just read the article 🙂
It can only target open source, so it wouldn’t bother corpos at all. The people behind this object to not everything being owned and controlled. That’s the whole point.
Unfortunately for them there’s a lot of jobs dedicated to cleaning data so I’m not sure if this would even be effective. Plus there’s an overwhelming amount of data that isn’t “poisoned” so it would just get drowned out if never caught
Takes image, applies antialiasing and resize
Oh, look at that, defeated by the completely normal process of preparing the image for training
Imagine if writers did the same things by writing gibberish.
At some point, it becomes pretty easy to devalue that content and create other systems to filter it.
if writers did the same things by writing gibberish.
Aka, “X”
Just don’t out your art to public if you don’t want someone/thing learn from it. The clinging to relevance and this pompous self importance is so cringe. So replacing blue collar work is ok but some shitty drawings somehow have higher ethical value?
cm0002Right, if you post publicly, expect it to be used publicly
Yeah, no. There’s a difference between posting your work for someone to enjoy, and posting it to be used in a commercial enterprise with no recompense to you.
How are you going to stop that lol it’s ridiculous. Would you stop a corporate suit from viewing your painting because they might learn how to make a similar one? It’s makes absolutely zero sense and I can’t believe delulus online are failing to comprehend such simple concept of “computers being able to learn”.
Computers can’t learn. I’m really tired of seeing this idea paraded around.
You’re clearly showing your ignorance here. Computers do not learn, they create statistical models based on input data.
A human seeing a piece of art and being inspired isn’t comparable to a machine reducing that to 1’s and 0’s and then adjusting weights in a table somewhere. It does not “understand” the concept, nor did it “learn” about a new piece of art.
Enforcement is simple. Any output from a model trained on material that they don’t have copyright for is a violation of copyright against every artist who’s art was used illegally to train the model. If the copyright holders of all the training data are compensated and have opt-in agreed to be used for training then, and only then would the output of the model be able to be used.
It’s literally in the name. Machine learning. Ignorance is not an excuse.
That’s just one of the dumbest things I’ve heard.
Naming has nothing to do with how the tech actually works. Ignorance isn’t an excuse. Neither is stupidity
they create statistical models based on input data.
Any output from a model trained on material that they don’t have copyright for is a violation of copyright
There’s no copyright violation, you said it yourself, any output is just the result of a statistical model and the original art would be under fair use derivative work (If it falls under copyright at all)
Considering most models can spit out training data, that’s not a true statement. Training data may not be explicitly saved, but it can be retrieved from these models.
Existing copyright law can’t be applied here because it doesn’t cover something like this.
It 100% should be a copyright infringement for every image generated using the stolen work of others.
You can get it to spit out something very close, maybe even exact depending on how much of your art was used in the training (Because that would make your style influence the weights and model more)
But that’s no different than me tracing your art or taking samples of your art to someone else and paying them to make an exact copy, in that case that specific output is a copyright violation. Just because it can do that, doesn’t mean every output is suddenly a copyright violation.
However since it’s required to use all of the illegally obtained and in-licensed work to create it, it is a copyright violation, just as tracing over something would be. Again, existing copyright law cannot be applied here because this technology works in a vastly different way than a human artist.
A hard line has to be made that will protect artists. I’d prefer it go even farther in protecting individual copyright while weakening overall copyright for corporate owners.
illegally obtained […] work
It what jurisdiction is it illegal?
And is “obtained” even the right word?..
There’s currently multiple lawsuits in the courts to decide just that.
If they’re scraping the internet to add to a database of training data, I’d consider that obtaining and storing the work.
Cyber YukiAh yes, just because lockpickers can enter a house suddenly everyone’s allowed to break and enter. 🙄
What a terrible analogy for learning 🙄
It’s not learning
You’re not learning, that much is obvious.
Flying SquidAre you actually suggesting that if I post a drawing of a dog, Disney should be allowed to use it in a movie and not compensate me?
Everyone should be assumed to be able to look at it, learn from it, and add your style to their artistic toolbox. That’s an intrinsic property of all art. When you put it on display, don’t be surprised or outraged when people or AIs look at it.
AI does not learn and transform something like a human does. I have no problem with human artists taking inspiration, I do have a problem with art being reduced to a soulless generation that requires stealing real artists work to create something that isn’t original.
AI does not learn and transform something like a human does.
But they do learn. How human-like that learning may be isn’t relevant. A parrot learns to talk differently than a human does too, but African greys can still hold a conversation. Likewise, when an AI learns how to make art by studying what others have made, they may not do it in exactly the same way a human does it, but the products of the process are their own creations just as much as the creations of humans artists that parrot other human artists styles and techniques.
you don’t know how humans learn and transform something
regardless, it does learn and transform something
Catoblepas“Just don’t make a living with your art if you aren’t okay with AI venture capitalists using it to train their plagiarism machines without getting permission from you or compensating you in any way!”
If y’all hate artists so much then only interact with AI content and see how much you enjoy it. 🤷♂️
using it to train their plagiarism machines
That’s simply not how AI works, if you look inside the models after training, you will not see a shred of the original training data. Just a bunch of numbers and weights.
| Just a bunch of numbers and weights
I agree with your sentiment, but it’s not just that the data is encoded as a model, but it’s extremely lossy. Compression, encoding, digital photography, etc is just turning pictures into different numbers to be processed by some math machine. It’s the fact that a huge amount of information is actually lost during training, intentionally, that makes a huge difference. If it was just compression, it would be a gaming changing piece of tech for other reasons. YouTube would be using it today, but it is not good at keeping the original data from the training.
Rant not really for you, but in case someone else nitpicks in the future :)
If the individual images are so unimportant then it won’t be a problem to only train it on images you have the rights to.
They do have the rights because this falls under fair use, It doesn’t matter if a picture is copyrighted as long as the outcome is transformative.
I’m sure you know something the Valve lawyers don’t.
It has literally nothing to do with plagiarism.
Every artist has looked at other art for inspiration. It’s the most common thing in the world. Literally what you do in art school.
It’s not an artist any more than a xerox machine is. It hasn’t gone to art school. It doesn’t have thoughts, ideas, or the ability to create. It can only take and reuse what has already been created.
The ideas are what the prompts and fine tuning is for. If you think it’s literally copying an existing piece of art you just lack understanding because that’s not how it works at all.
Fighting the uphill battle to irrelevance …
Man, whenever I start getting tired by the amount of Tankies on Lemmy, the linux users and decent AI takes in users rejuvenates me. The rest of the internet has jumped full throttle on the AI hate train
The “AI hate train” is people who dislike being replaced by machines, forcing us further into the capitalist machine rather than enabling anyone to have a better life
No disagreement, but it’s like hating water because the capitalist machine used to run water mills. It’s a tool, what we hate is the system and players working to entrench themselves and it. Should we be concerned about the people affected? Yes, of course, we always should have been, even before it was the “creative class” and white collar workers at risk. We should have been concerned when it was blue collar workers being automated or replaced by workers in areas with repressive regimes. We should have been concerned when it was service workers being increasingly turned into replaceable cogs.
We should do something, but people are titling at windmills instead of the systems that oppress people. We should be pushing for these things to be public goods (open source like stability is aiming for, distributed and small models like Petals.dev and TinyML). We should be pushing for unions to prevent the further separation of workers from the fruits of their labor (look at the Writer’s Guild’s demands during their strike). We should be trying to only deal with worker and community cooperatives so that innovations benefit workers and the community instead of being used against them. And much more! It’s a lot, but it’s why I get mad about people wasting their time being made AI tools exist and raging against them instead of actually doing things to improve the root issues.
Thing is, its capitalism thats our enemy, not the tech that is freeing us up from labour. Its not the tech thats the problem, its our society, and if fucking sucks that I’m just as poor as the rest of you, but because I finally have a tool that lets me satisfactorily lets me access my creativity, I’m being villianized by the art community, even though the tech I am using is open source and no capitalist is profiting off of me
The “AI hate train” runs on fear and skips stops for reason, headed for a fictional destination.
Not saying that there aren’t people like that, but this ain’t it. This tool specifically targets open source. The intention is to ruin things that aren’t owned and controlled by someone. A big part of AI hate is hyper-capitalist like that, though they know better than saying it openly.
People hoping for a payout get more done than people just being worried or frustrated. So it’s hardly a surprise that they get most of the attention.
nightshade and glaze never worked. its scam lol
Shhhhh.
Let them keep doing the modern equivalent of “I do not consent for my MySpace profile to be used for anything” disclaimers.
It keeps them busy on meaningless crap that isn’t actually doing anything but makes them feel better.
So it sounds like they are taking the image data and altering it to get this to work and the image still looks the same just the data is different. So, couldn’t the ai companies take screenshots of the image to get around this?
Not even that, they can run the training dataset through a bulk image processor to undo it, because the way these things work makes them trivial to reverse. Anybody at home could undo this with GIMP and a second or two.
In other words, this is snake oil.
Uriel238 [all pronouns]The general term for this is adveesarial input, and we’ve seen published reports about it since 2011 when ot was considered a threat if CSAM could be overlayed with secondary images so they weren’t recognized by Google image filters or CSAM image trackers. If Apple went through with their plan to scan private iCloud accounts for CSAM we may have seen this development.
So far (AFAIK) we’ve not seen adversarial overlays on CSAM though in China the technique is used to deter trackng by facial recognition. Images on social media are overlaid by human rights activists / mischief-makers so that social media pics fail to match secirity footage.
The thing is like an invisible watermark, these processes are easy to detect (and reverse) once users are aware they’re a thing. So if a generative AI project is aware that some images may be poisoned, it’s just a matter of adding a detection and removal process to the pathway from candidate image to training database.
Similarly, once enough people start poisoning their social media images, the data scrapers will start scaning and removing overlays even before the database sets are sold to law enforcement and commercial interests.
KᑌᔕᕼIᗩArtists and writers should be entitled to compensation for using their works to train these models, just like any other commercial use would. But, you know, strict, brutal free-market capitalism for us, not the mega corps who are using it because “AI”.