Amusing title that is probably true (LLMs don't have consciousness, Hinton is not a consciousness expert), although I think their argument may merely be about Turing equivalence:

"If LLMs Have Human-Like Attributes, Then So Does Age of Empires II"
https://arxiv.org/abs/2605.31514

A comment on lobsters says that AoE also has a built-in AI in the form of a CLIPS S-EXPR expert system (which is Good Old Fashioned AI).
https://lobste.rs/s/owclks

If LLMs Have Human-Like Attributes, Then So Does Age of Empires II

Much research has been carried out on large language models (LLMs) and LLM-powered agentic workflows. However, many works within the field state emergence of, ascribe to, or assume, generalised anthropomorphic attributes to them (e.g., morality or understanding of natural language). Our goal is not to argue in favour or against the existence of these attributes, but to point out that these conclusions could be incorrect. For this we build and train a simple neural network on the videogame Age of Empires II, and note that any entity in a sufficiently-powerful substrate, such as LEGO or the Greater Boston Area, could also present such attributes. Hence, the purported anthropomorphic attributes of LLMs are empirically non-unique: although some properties (e.g., responses to prompts) could remain constant, others, such as the interpretation of their perceived behaviour, might change with the substrate. Thus, any empirically-grounded discussion requires explicit measurement criteria; otherwise the interpretation is left to the representation. We then show that assuming that these attributes exist or not in a system, independent of the substrate and in a generalised way, leads to either circular or uninformative conclusions, regardless of the experimenter's viewpoint on the subject. Finally we propose a 'null' assumption, where one assumes LLM non-uniqueness instead of assuming anthropomorphic attributes to set up an experiment, along with examples of it. We also discuss potential objections to our work, briefly survey the field, and prove that \textit{Age of Empires II} is functionally- and Turing-complete.

arXiv.org

@dougmerritt

Discussion about consciousness in a nutshell:

"Let me tell you about what consciousness is, obviously."

Only analytical philosophers seem to have some interest in different definitions and finding out what others think.

@maxpool
Exactly!!!

It's not well-defined in the first place, so people who take firm positions without even bothering to spell out what they mean are being stupid, no matter how smart they are otherwise.

(And the ones who *do* define what they mean are doing somewhat better, but what they offer is typically very easy to critique.)

I find it rather irritating.

@dougmerritt @maxpool rocks are superintelligent, right? It is hard to imagine how wet objects are similarly intelligent at all, how would it even work. This was an attempted Terry Pratchett reference.

@dougmerritt @maxpool
Small clarification from the abstract, they trained a gameplaying deep learning system to play the strategy game, and the strategy game has a gof_ai expert system in it.

Then the argument is that the behaviours of their gameplaying DL is experiencing gamer emotions in the same way and to the same extent that megacorporate chatbots are experiencing 1900 number emotions.

@screwlisp @maxpool
Someone said that the abstract incorrectly said "on" the game when they meant "in" the game. I didn't bother to check which one was correct, but it appears that your comment's premise hinges on this -- although I again agree with your conclusion.
@dougmerritt
I will check the paper eventually, some bits of the abstract were odd so I may have misread it. E.g. why did they prove that /age of empires/ itself was turing complete? Normally the observation is that /deep learning/ is turing complete i i r c not at the front of my brain.
@maxpool

@screwlisp
Since I haven't ranted about this for a long time: human brains and minds are not Turing Complete nor equivalent, trivially, since humans are wrong like *ALL THE TIME*.

Turing machines are not. QED.
@maxpool

@dougmerritt @screwlisp

In theoretical computer science, Turing completeness is a property of a system's architecture (instruction set, or rules) not what it does or it's physical execution, or external limits.

If we count our "fixed tape" as part of our internal architecture, we are just finite state machines.

And in no case can any physical execution of a machine with limited running time (input dependent or not) do more than sufficiently large finite state machine.

@maxpool
None of that contradicts my point.

If a Turing machine runs a tape that emulates humans that say that false things are true, then the simulation running on that Turing machine is no longer Turing complete; you need to change its program back to something where true != false.

It doesn't matter that the underlying Turing machine is Turing complete. We're concerned with the overall system's behavior.

Turing machines could be modified to allow them to calculate incorrect results, which would in one sense evade various theoretical limits, but generally getting false results is not what we want.

Humans minds being fallible yet still of interest is, overall, not well understood at all.

@screwlisp

@dougmerritt @screwlisp

This seems like category error.

Turing machine not being allowed to give incorrect results makes machine either strictly less or more powerful than Turing machine.

1. sub-Turing: total programming language.

2. super-Turing ability to detect if the program is correct and not give wrong answers.

The computer in front of you is Turing equivalent RA-machine. It can execute buggy programs and randomly generated binaries as you know.

@maxpool
> either strictly less or more powerful than Turing machine.

Not so. Add a randomizer that forces a Turing machine, that contains an answer symbol to emit, to stop after a random amount of time, rather than using stopping/nonstopping as the result. It will now compute anything at, just incorrectly in some (many) cases.

That makes it super-Turing when it is right and sub-Turing when it is wrong.

You could even just use the randomizer to get the answer, with the same conclusion, but since there's no Turing machine, that feels less satisfying. :)

(This is not the way oracles are usually used, of course.)

@screwlisp @vnikolov

@dougmerritt @maxpool @screwlisp @vnikolov

No, it's always less than Turing complete, even when it's right.

@simon_brooke
Because of the "complete", yes, alright.

I'm having so much trouble expressing myself without gross errors in addition to opacity that I'd better just rethink this.

Thanks for the feedback.

@maxpool @screwlisp @vnikolov

@maxpool
> The computer in front of you is Turing equivalent RA-machine. It can execute buggy programs and randomly generated binaries as you know.

True, and I think I've been unclear.

Ultimately I'm saying that it's important to distinguish Turing machines from the programs they run.

The latter cannot be more powerful than the Turing machine itself, but they can be (and usually are) *less* powerful.

@screwlisp
@vnikolov

@dougmerritt @screwlisp @vnikolov

If the program can simulate Turing machine, it is Turing equivalent.

Humans can simulate Turing machine, not very far but still. With paper and pen, as long as needed. We have the architecture to do so.

@maxpool
Yes, of course you're right. My point is about programs that do *not* simulate Turing machines.

A human *can* simulate a Turing machine, but usually they do not.

When they are not doing so, they are (at least potentially) not Turing equivalent.

@screwlisp @vnikolov

«Humans can simulate Turing machine, not very far but still. With paper and pen, as long as needed. We have the architecture to do so.»

I don't really know if that is "sufficiently true".
Long before we hit the limit of mortality, we hit the limit of mistakes during boring and repetitive tasks.
I don't think we can abstract this away.
I suppose we can safely say that humans can simulate small Turing machines for a short time.
I don't know if that is enough.

In any case a substantial analysis would require a substantial definition of "can simulate".

@maxpool @dougmerritt @screwlisp

«... distinguish Turing machines from the programs they run.

The latter cannot be more powerful than the Turing machine itself, but they can be (and usually are) *less* powerful.»

Cannot be more powerful in a theory-of-algorithms sense, yes, certainly, but the interesting question (with a difficult answer) is whether they can simulate adequately something that may not be a Turing machine.
Where "adequately" and "may not be" need to be specified precisely and that is by itself difficult (or at least time-consuming).

@dougmerritt @maxpool @screwlisp