@elmerot @andreskarjus Training is the most resource-hungry step, so I'd say not training a new LLM, but rather using one already trained, seems much preferable.

@elmerot Use an already pretrained model as base, and just tune it for your use case or train a LoRa adapter - depending on use case this takes very little resources (to the point you might be able to do it on free Google Colab GPU credits). Training a competitive model from scratch is prohibitively expensive and usually wasteful.