@sfwrtr @raganwald

I've given some thought to the meaning of death as it applies to those who have posted frequently on the internet. We often don't see people's writings in the order that they write them, and that means we can see new posts from them after they die.

Even without the internet this happens. I was in a bookstore recently and saw a book by Michael Crichton and asked the shopkeeper, "Isn't this his third posthumous book?" "Yeah..." he sheepishly responded. Someone is plainly raiding his basement for rejected works and works that were close enough to be completion that someone else can complete them and claim to have been co-author. His heirs are probably happy for the income.

Perhaps it's even possible for a prolific writer to write so much that you never really see them die because you just keep seeing new stuff. So in what sense are they dead? You were perhaps never going to meet them, and so in some sense—of observables—they're doing the same things that live people are doing.

The one thing they cannot do is the sane thing GenAI/LLMs cannot do: competently respond to a new situation, question, or idea. Oh, sure, it might be something someone speculated on before, so they can regurgitate that. Or it might be enough similar to a previous idea that the probabilities of guessing acceptably based on just assuming it really is an old idea or high enough that it escapes scrutiny that the idea was not really understood.

As I imagine (or perhaps just hope?) the makers of standardized tests like the SAT would tell you, there's more to competence than statistically guessing enough right answers to get a passing grade. The intent of such tests it's not to say that if you know these things, you know the topic. It is to assume you have a model that lets you answer on any possible thing, and then to poke at enough randomly chosen places that you hoped to detect the absence of a model.

But these so-called AI technologies do not have a model. They just hope they've read enough standardized test preparations or pirated actual tests that they can fake their way. And since a lot of the things that they're claiming competence in is stuff that people have already written about, they show promise.

Real people build a mental model that allows them to confront the future and these technologies do not. They built a model that hopes the future will be enough like today but they couldn't get by on bluffing. They are not growing. They are dead.

The nature of the game, just like my question about publication and death on the internet, is such that it takes a long time to recognize, unless just the right question is asked. And then, perhaps, the emperor will be seen clearly to have no clothes.

Which is also why a troubles me when I'm told that people are coming to understand how to incorporate ethics. Because ethics itself has to be growing all the time, constantly asking itself how might I not be ethical? It is something you do and are done with. It has models too. But it is easily buried under the sophistry of how things have already been done. The reason bias and stereotype and all that have survived as long as they have is that they have practical value to someone, perhaps many people, even as they trod on the due of others.

The sins of our culture are deeply woven, and easily rediscovered even if superficial patches are added to hide them. Our whole culture is a kind of rationalization engine for doing things in biased ways based on stereotype information, and AI is an engine ready to reinforce that, operating at such high speed that it's hard to see happening, and economically irresistible not to accept as good enough.

This kind of AI brings us face to face with stark questions about whether being smart is actually all that important, or whether faking it is good enough. And as long as you never put these things in situations where the difference matters, maybe the answer will be that smart in fact doesn't matter. But there will be times when it does, and I think we're teaching ourselves trust in the wrong technologies for those situations.

#AI #GenAI #LLM #LLMs #ethics #philosophy #knowledge #modeling #life #death #education #competence #StandardizedTests #testing #bias #stereotypes #prejudice