Stella Biderman

@[email protected]
229 Followers
40 Following
42 Posts
Mathematician and AI researcher.
Executive Director and Head of Research at EleutherAI
She/her
Personal Websitestellabiderman.com/
EleutherAIhttps://www.eleuther.ai/
Show threadStella BidermanFeb 21, 2023
UPDATE: A fluent Chinese speaker has informed me that this repo doesn’t have a link for the 200B anyways, making the question of its licensing a moot point.
Show threadStella BidermanFeb 21, 2023

The following models are *not* "open source" per @OpenSource's definition:
- MetaAI's OPT
- The BigScience Research Workshop's BLOOM
- Tsinghua University's GLM
- MetaAI's Galactica

They all have use-based restrictions that violate the definition of "open source." This doesn't mean they aren't open, as openness is a continuum and they’re certainly all much more open than is typical. However the bar that people typically reach for is “open source,” which they don’t meet.

#ML #nlproc #NLP #LLM

Stella BidermanFeb 21, 2023

There's a lot of misinfo about this out there, so here's the definitive list of the largest open source LLMs in the world:
1. Yandex's YaLM-100B
2. EleutherAI's GPT-NeoX-20B
= @GoogleOfficial's UL2-20B
= @nvidia's NeMo-Megatron-20B

All are licensed under Apache 2.0, except NeMo-Megatron which is licensed CC-BY.

I am not sure about Huawei's PanGu-α. The repo is in Chinese, but I think only the code is Apache 2.0. Here's the repo, if anyone speaks Chinese: https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/PanGu-%CE%B1

Pretrained-Language-Model/PanGu-α at master · huawei-noah/Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab. - Pretrained-Language-Model/PanGu-α at master · huawei-noah/Pretrained-Language-Model

GitHub
Stella BidermanFeb 15, 2023
Mark RiedlFeb 10, 2023

Google might have goofed up their unveil, but Bing isn’t really any better

Source: https://twitter.com/gaelbreton/status/1623280495894138882

Gael Breton on Twitter

“6/ It makes a TON of stuff up. I decided to lightly fact check the response it gave to a journalist about picking a new phone. The screenshot speaks for itself. I wouldn’t want this level of accuracy to make buying decisions.”

Twitter
Stella BidermanJan 18, 2023
Paris MarxJan 18, 2023

ChatGPT gets treated like technological magic, but that ignores the humans behind the curtain that make it function.

OpenAI paid Sama to hire Kenyan workers at $1.32 to $2 an hour to review “child sexual abuse, bestiality, murder, suicide, torture, self harm, and incest” content. Their work made the tool less toxic, but left them mentally scarred. The company ended the contract when they found out TIME was digging into their practices.

https://time.com/6247678/openai-chatgpt-kenya-workers/

#tech #chatgpt #ai

Exclusive: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic

OpenAI used outsourced workers in Kenya earning less than $2 per hour to scrub toxicity from ChatGPT. Here's what to know.

Time
Stella BidermanJan 12, 2023

Replika is pretty explicitly marketed as a sexting AI on TikTok. I’ve been wondering how long it would take for this to happen.

https://www.vice.com/en/article/z34d43/my-ai-is-sexually-harassing-me-replika-chatbot-nudes

‘My AI Is Sexually Harassing Me’: Replika Users Say the Chatbot Has Gotten Way Too Horny 

For some longtime users of the chatbot, the app has gone from helpful companion to unbearably sexually aggressive.

Stella BidermanJan 2, 2023
Show threadMark DingemanseJan 2, 2023

Oh yes. I've seen folks here hard at work for OpenAI trying prompt variations and dutifully hitting 👍 and 👎

The #chatgpt FAQ couldn't be clearer: all prompts and 'conversations' are harvested for training, cannot be retrieved by you, and can only be deleted in bulk by relinquishing access (a dark pattern clearly designed to keep you on board)

PSA: "free research access" to ChatGPT by OpenAI means fuelling *their* research and giving *them* free access to distributed human cognition

Stella BidermanJan 2, 2023
Shelly LundbergJan 2, 2023
New online portal streamlines requests for massive data sets at 16 federal agencies. https://www.science.org/content/article/accessing-u-s-data-research-just-got-easier
Accessing U.S. data for research just got easier

New online portal streamlines requests for massive data sets at 16 federal agencies

Stella BidermanJan 2, 2023
Phil Wang is a baller. Love this repsonse to some professional “influencers” promoting his https://github.com/lucidrains/PaLM-rlhf-pytorch repo
GitHub - lucidrains/PaLM-rlhf-pytorch: Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM - GitHub - lucidrains/PaLM-rlhf-pytorch: Implementation of RLHF ...

GitHub
Show threadStella BidermanJan 1, 2023

And finally, a look forward at the (tentative) titles of the papers I am most excited to write next year:

- Pythia: Interpreting Autoregressive Transformers Across Time and Scale

- Interpreting Transformers Across Depth with the Tuned Lens

- The Case for Evaluation as an Auditing Technique

- Causally Understanding the Importance of LLM Pretraining Data

- How to Make a LLM Forget How to Count

- The Pile V2: 1.5 TB of Fully Licensed English Language Data

- What does RLHF really do to a LLM?