@burakemir Okay, but most writers (not all, obviously) do not actually mean to anthropomorphize. It's just a convenient shorthand.
Like if I say "the LLM says" or "the LLM knows", sure, the first time I could write «the LLM "says"» or «the LLM "knows"» but the tenth time you read that in the same essay, it's irritating as heck.
Note that I'm also the kind of person who refuses to assign a gender to Alexa, Siri, etc.: it's an "it".
@shriramk I am no Kamlah and Lorenzen, but let me try.
* instead of "say/reply", use "emit" or "generate" output.
* Instead of "thinks/understands" use "computes probabilities" or "maps vector spaces".
* Instead of "hallucinates" use "emits misaligned patterns".
* Instead of "learns" use "optimizes weights".
* Instead of "cooperate", say the human "initiates" interaction by providing constraints, and the model solves these constraints.
@burakemir @shriramk i feel like part of the problem here is how not every aspect of communication is literal but we're not always good at signaling or even recognizing when we're moving between fact and metaphor.
we do this in the other direction too, e.g. sometimes say i'm "adjusting my priors" when i learn new information. but i suppose it's usually more obviously metaphorical in this direction
@chrisamaphone @shriramk I also think there is something about metaphor, or more generally the ability to make and understand analogy which is key. This is part of human intelligence, but we are also intentionally tricked (we want to be tricked, we want to interact with the machine like we interact with humans, because we like to belong.)
Analogies are fine but this tech is useful because it captures "knowledge" in a (still) novel, practically relevant way and then simulates human communication. The ML researchers are IMHO not particularly qualified to talk about this; we have never been able to measure knowledge, the extent, practical relevance or effectiveness of communication. Now, in addition to that, we are trying to get comfortable in a sea of convincing, plausible and self-confirming fuzzy information retrieval plus slop, and all that gets to be called "intelligence". This is a very weak analogy. The machine lacks inherent purpose.
IMHO it would be nice if people who think could also engage in precise, "formal", "objective" language to express whatever this is, and such discourse will be flawed if it resembles the goddamn marketing talk and fails to acknowledge purpose and control. There needs to be a signal that we are striving for intellectually honesty. I am certainly not playing the science language police here, I just want to convey that I am missing this and it seems a real prerequisite to meaningful discussion on ethics of LLMs and LLM use.
I actually try to collect analogies that people use specifically for generative AI. The thing is, they're really bad in *both* directions. They are less a reflection of the underlying reality than of the positions of the people who use them. The ones that deprecate it, for instance, clearly just fly in the face of every available lived positive evidence. ↵
@burakemir @chrisamaphone
It's marketing talk in both directions. Take away metaphors and you strip away natural, human language. You can't even use technical terms: e.g., reinforcement learning is full of "common" terms (eg, "curriculum learning") that have been made technical. (Of course, "reinforcement" and "learning" also have just that property.)
So you *can* strip away all natural language, but then you only get mathematical symbols. I'm fine with that! But everything else is political.
Shriram Krishnamurthi
burakemir
chris martens