Got multiple datasets and you want to improve your own task
Proposed: multitask improvement to the fuse and tune
But more importantly
weight fused model without gradients!
http://arxiv.org/abs/2307.03506
#nlproc #fusing #modelRecycling #machinelearning
Recent work suggests that interpolating between the weights of two specialized language models can transfer knowledge between tasks in a way that multi-task learning cannot. However, very few have explored interpolation between more than two models, where each has a distinct knowledge base. In this paper, we introduce Derivative Free Weight-space Ensembling (DFWE), a new few-sample task transfer approach for open-domain dialogue. Our framework creates a set of diverse expert language models trained using a predefined set of source tasks. Next, we finetune each of the expert models on the target task, approaching the target task from several distinct knowledge bases. Finally, we linearly interpolate between the model weights using a gradient-free-optimization algorithm, to efficiently find a good interpolation weighting. We demonstrate the effectiveness of the method on FETA-Friends outperforming the standard pretrain-finetune approach.
Zip it: Fuse models with themselves first
Merge models trained on different tasks by correlations between activations
https://arxiv.org/abs/2305.03053
#CV #machinelearning #fusing #merging
Typical deep visual recognition models are capable of performing the one task they were trained on. In this paper, we tackle the extremely difficult problem of combining distinct models with different initializations, each solving a separate task, into one multi-task model without any additional training. Prior work in model merging permutes one model to the space of the other then averages them together. While this works for models trained on the same task, we find that this fails to account for the differences in models trained on disjoint tasks. Thus, we introduce "ZipIt!", a general method for merging two arbitrary models of the same architecture that incorporates two simple strategies. First, in order to account for features that aren't shared between models, we expand the model merging problem to allow for merging features within each model by defining a general "zip" operation. Second, we add support for partially zipping the models up until a specified layer, naturally creating a multi-head model. We find that these two changes combined account for 20-60% improvement over prior work, making it more feasible to merge models trained on disjoint tasks without retraining.
An exercise with glass stringers, which are about as thick as angel hair pasta.
I used a candle flame to heat the glass stringers and shape to bend, then used a small dab of glass glue to secure the stringers to a precut pieces of glass. Fire to full fuse in the kiln (roughly a 12 hour cycle), and you end up with these coasters that were made and given away as anniversary gifts.
The Wanhao Duplicator CNC Heat Sealer
One custom, compliant heat exchanger, coming right up!
[Thane Hunt] needed to find a way to make a variety of different heat-seal patterns on a fluid heat exchanger made from polyolefin film, and didn't want all the lead time and expense of a traditional sealing press machined from a steel plate. Pattern prototyping meant that the usual approach would not allow sufficient iteration speed and decided to take a CNC approach. Now, who can think of a common tool, capable of positioning in the X-Y plane, with a drivable Z axis and a controlled heat source? Of course, nowadays the answer is the common-or-garden FDM 3D printer. As luck would have it, [Thane] had an older machine to experiment with, so with a little bit of nozzle sanding, and a sheet of rubber on the bed, it was good to go!
Custom seal path made in Onshape
Now, heat sealing is usually done in a heated press, with a former tool, which holds the material in place and gives a flat, even seal. Obviously this CNC approach isn't going to achieve perfect results, but for proof-of-concept, it is just fine. A sacrificial nozzle was located (but as [Thane] admits, a length of M6 would do, in a pinch) and sanded flat, and parallel to the bed, to give a 3mm diameter contact patch. A silicone rubber sheet was placed on the bed, and the polyolefin film on top. The silicone helped to hold the bottom sheet in place, and gives some Z-axis compliancy to prevent overloading the motor driver. Ideally, the printer would have been modified further to move this compliancy into the Z axis or the effector end, but that was more work. With some clever 3D modelling, Cura was manipulated to generate the desired g-code (a series of Z axis plunges along a path) and a custom heated indenter was born!
This isn't the first such use of a 3D printer we've seen, here's an earlier failure, and like everything, there's more than one way to do it - here's a method of making inflatable bladders with a defocused CO2 laser.
(warning! Two minutes of a 3D printer head-banging into the bed!)
#3dprinterhacks #3dprinter #fusing #gcode #heatexchanger #indenting #plastic #polyolefin
Fondre du verre au micro-ondes - HS au calme - Monsieur Bidouille
Otávio Ribeiro Goncalez
Leshem Choshen
Donavan Quijote
hackaday unofficial
Dimitri