I heard of this through @[email protected] 's newsletter and radio show but had to try it myself. The terrible results are still up.

This example is funny and absurd, but imagine people who are seeking advice for serious issues--medical, legal, financial--being shown the equivalent of "applum". Though we don't have this concept in tech, I'd characterize showing people results like this as malpractice. It almost surely runs afoul of standard technology ethics guidelines, which typically have something like "do no harm" near the top of the list.

I worked on large language models in the context of a startup company in the 2016-2019 timeframe ( https://bucci.onl/notes/Legit.ai ). It's one of the reasons I comment so frequently and so negatively on this technology here--I've worked with it, at least the generation of it from that time (much has changed in the meantime, though not at the fundamental level). We experimented a bit with natural language generation. I concluded at the time that it was nowhere near ready for prime time even in the restricted domain in which we were operating. Despite Google's vast computational resources, gigantic troves of data, and the intervening 5+ years of breakthroughs in this technology, its generative AI here is still not ready for prime time as far as I can see. Never say never, but I don't think it ever will be unless the application domain is well-scoped, which general web search is not.

The really sad part? There is non-LLM-based technology that can do this sort of thing pretty well when scoped carefully. Less embarrassingly badly than what Google is demonstrating here, that's for sure. There are folks within Google, or at least there used to be, who are well aware of this.

#Google #AI #GenerativeAI #GenAI #AIOverview #tech #PinkSlime #AIGoop #InformationOilSpill