Leshem ChoshenJul 16, 2023

This entity wasn't in the pretraining😢
Don't cry, little ML researcher

Take new term definitions
Continue them with follow-up sentences
Distill your model (D-KL) to continue the same, without the definition
You know the new terms now
Go prompt them tiger🐯

https://arxiv.org/abs/2306.09306

#NLProc #modelRecycling #ModelEditing #machinelearning

Propagating Knowledge Updates to LMs Through Distillation

Modern language models have the capacity to store and use immense amounts of knowledge about real-world entities, but it remains unclear how to update their implicit "knowledge bases.'' While prior methods for updating knowledge in LMs successfully inject facts, updated LMs then fail to make inferences based on these injected facts. In this work, we demonstrate that a context distillation-based approach can both impart knowledge about entities and propagate that knowledge to enable broader inferences. Our approach consists of two stages: transfer set generation and distillation on the transfer set. We first generate a transfer set by simply prompting a language model to generate a continuation from the entity definition. Then, we update the model parameters so that the distribution of the LM (the student) matches the distribution of the LM conditioned on the definition (the teacher) on the transfer set. Our experiments demonstrate that this approach is more effective in propagating knowledge updates compared to fine-tuning and other gradient-based knowledge-editing methods without compromising performance in other contexts, even when injecting the definitions of up to 150 entities at once.

arXiv.org
Leshem ChoshenJul 13, 2023

Got multiple datasets and you want to improve your own task
Proposed: multitask improvement to the fuse and tune
But more importantly

weight fused model without gradients!

http://arxiv.org/abs/2307.03506
#nlproc #fusing #modelRecycling #machinelearning

Derivative Free Weight-space Ensembling

Recent work suggests that interpolating between the weights of two specialized language models can transfer knowledge between tasks in a way that multi-task learning cannot. However, very few have explored interpolation between more than two models, where each has a distinct knowledge base. In this paper, we introduce Derivative Free Weight-space Ensembling (DFWE), a new few-sample task transfer approach for open-domain dialogue. Our framework creates a set of diverse expert language models trained using a predefined set of source tasks. Next, we finetune each of the expert models on the target task, approaching the target task from several distinct knowledge bases. Finally, we linearly interpolate between the model weights using a gradient-free-optimization algorithm, to efficiently find a good interpolation weighting. We demonstrate the effectiveness of the method on FETA-Friends outperforming the standard pretrain-finetune approach.

arXiv.org