My life goal is to make a comic book for the comedy gold my mom spins into my life.
And a graphic novel for all the ingots of tragedy. š¤£
My mom is in parent time-out for the following dialogue during work this week:
Mom: Jonathan, we need to talk
Me: whatās wrong are you okay?
Mom: do you have sea salt?
Me: I, what?
Mom: SEA SALT, LIKE SALT BUT FROM THE SEA
Me: ā¦uh yeah? But Iām at work, can weā
Mom: Get KOREAN sea salt.
Me: okay, sure!
Mom: Did you just say *sure* to your *mother*?!
Me: -internally- oh shit
Mom: Well, thatās fine, WHEN IM DEAD no one will bother you about sea salt! *hangs up*
@arnicas thatās actually verbatim what I did once upon a time. Just did a token classification over locations and their various dependencies as a proxy for āwords describing placesā
It wasnāt terrible!
A recent trend is to fine-tune open-source LMs on ChatGPT outputs (e.g., Alpaca, Self-Instruct, Vicuna), with the aim of broadly imitating the model. In our new paper, we critically analyze this approach.
https://arxiv.org/abs/2305.15717 š[1/N]
š¦š: https://twitter.com/arnavg_/status/1662189000667590656
An emerging method to cheaply improve a weaker language model is to finetune it on outputs from a stronger model, such as a proprietary system like ChatGPT (e.g., Alpaca, Self-Instruct, and others). This approach looks to cheaply imitate the proprietary model's capabilities using a weaker open-source model. In this work, we critically analyze this approach. We first finetune a series of LMs that imitate ChatGPT using varying base model sizes (1.5B--13B), data sources, and imitation data amounts (0.3M--150M tokens). We then evaluate the models using crowd raters and canonical NLP benchmarks. Initially, we were surprised by the output quality of our imitation models -- they appear far better at following instructions, and crowd workers rate their outputs as competitive with ChatGPT. However, when conducting more targeted automatic evaluations, we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems.
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved With Text
103M documents containing 585M images interleaved with 43B English tokens
MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text. - GitHub - allenai/mmc4: MultimodalC4 is a multimodal extension of c4 that interleaves millions of images...
kasdeya
Aran Komatsuzaki