Brandon SilvermanI think one underappreciated thing that's happening with ChatGPT is that it's reminding people of how joyful it is to do totally mundane searches, i.e. looking up recipes, and *not* get indundated with a million fucking ads.
stephen judkins@brandonsilverman still in the early part of the cycle where they try to get people excited. if/when they achieve lock-in any ads will be deeply insidious. see https://www.youtube.com/watch?v=5Zmh5hbgHFI
Truman Show Product Placement
@brandonsilverman my vague feeling of optimism is that it may be pretty hard for the big players to get a stranglehold on AI tech, it seems like the barriers to entry are pretty low. My fears of Peter Thiel's Robotic Oppression Force are receding
flere-imsaho 🇺🇦@stephenjudkins @brandonsilverman srsly? do you realise the amount of resources you have to throw at an llm to produce acceptable results?
@mawhrin @brandonsilverman i've been following it fairly closely and the cost of training appears reasonably affordable and is dropping pretty rapidly. At cloud costs it's in the low $10Ms now, which wouldn't be too hard to raise. It's likely it will get a whole lot cheaper
Laurens Hof@mawhrin @stephenjudkins @brandonsilverman https://simonwillison.net/2023/Mar/17/beat-chatgpt-in-a-browser/
the costs are dropping extremely rapidly, Alpaca 7B is roughly equal to GPT3 (not chatgpt's 3.5) quality, and an estimated training cost of 85k USD.
Obviously, gpt4 is wayyy more expensive, but alpaca proves that finetuning is cheap, and its now open source available. Lots of implementations also dont need the wide skills of gpt4, but instead need to be finetuned to specialize, so lower performance than gpt4 is fine
the costs are dropping extremely rapidly, Alpaca 7B is roughly equal to GPT3 (not chatgpt's 3.5) quality, and an estimated training cost of 85k USD.
Obviously, gpt4 is wayyy more expensive, but alpaca proves that finetuning is cheap, and its now open source available. Lots of implementations also dont need the wide skills of gpt4, but instead need to be finetuned to specialize, so lower performance than gpt4 is fine
@laurenshof @mawhrin @brandonsilverman yes, and even the "expensive" training costs are well within the bounds of medium-sized investors. It's simply not a big enough moat to protect the big players. If someone can credibly spend $20M to dethrone MS/Google there will be investors lining up with their checkbooks