jwzFeb 18

One of the things that is amazing about AI derangement syndrome is that when you call someone out for buying into that shit -- someone you've previously been on quite good terms with for decades -- their response to you saying, "hey, you should stop doing that, there is no ethical use of this tech" is a level of offense as if you have just fucked their cat.

Some of you probably know who I'm talking about. If not, you probably have examples yourselves.

Show threadScott GallowayFeb 18
@jwz Interesting, you don't think small open weight models running as part of a local OSS app are ethical?
Show threadjwzFeb 18
@scottgal Absolutely the fuck not, you absolute troll. Where did the data come from? Burning down forests and bulk plagiarism. It is fruit of the poisoned, burning tree. "Interesting".
Show threadkNeo gHau
@jwz @scottgal Hey Mr hostile, what do you think "open weight models" are? I'd like you to explain exactly why you think open weight models are plagiarism and theft? Maybe you should spend less time on the social medias, which cooks your brain traps you in echo chambers and is destroying society.
Feb 18 at 2:26pmMastodon for Android
Show threadcobweb 2026 editionFeb 18
@kneoghau you're not familiar with jwz are you
Show threadBornachFeb 18

@cobweb @kneoghau
Also not familiar with why "open weights" cannot be called "open source".

https://www.deeplearning.ai/the-batch/how-metas-llama-nlp-model-leaked/

Is there any open weights model that cannot be traced back to Meta's Llama leak, and was trained using data for which licensing permission to use it in this way was obtained from the owners of said data?

How Meta's LLaMA NLP Model Leaked

Meta’s effort to make a large language model available to researchers ended with its escape into the wild. Soon after Meta started accepting applications...

How Meta's LLaMA NLP Model Leaked
Show threadmirthFeb 19
@bornach @cobweb @kneoghau I don't think many of the open weights models are derived from Llama, but all or nearly all are trained on datasets like Common Crawl which are problematic. Regardless, having weights is somewhat akin to having a compiled executable in that it is the product rather than the source. There is some research effort to build models which are fully reproducible, like Allen Institute's models, but they still use Common Crawl.
Show threadBornachFeb 19

@mirth @cobweb @kneoghau
And there's also the whole ethical minefield of distilled models to consider.

And the ethics of AI research relying on a microtasking gig economy that pays poverty pittances to an exploited class of data annotators.

This part of Fei-Fei Li's ("godmother if AI") talk always disturbed me once I learned how she acquired the training dataset
https://youtu.be/40riCqvRoMs?t=7m7s
(At 7:07)

My unsettled feelings were never assuaged
https://ssir.org/articles/entry/ai-workers-mechanical-turk

How we teach computers to understand pictures | Fei Fei Li

YouTube
Show threadmirthFeb 20
@bornach @cobweb @kneoghau Distilled models from that standpoint are essentially the sum of all the input models plus additional training data. More problematic, if anything.
Show threadbigiainFeb 19
@cobweb Pretty sure @jwz ‘s blog wasn’t in @kneoghau ‘s training set. (Hopefully that training set contains terabytes of the hairyballs.jpg that requests with orange site referrers get.)
Show threadkNeo gHauFeb 19
@bigiain @cobweb Yeah I don't follow social media influencers, especially if said influencers give logan paul like vibes in responses. So in answer to your "don't you know who I am?"style comment, no I don't know and don't care too.