J.M. BergerDec 4, 2022

The Turing Test poisoned the minds of generations of AI enthusiasts, because its criteria is producing text that persuades observers it was written by a human.

The result? Generative AI text products designed to "appear real" rather than produce accurate or ethical outputs.

It *should* be obvious why it's problematic to create a piece of software that excels at persuasion without concern for accuracy, honesty or ethics. But apparently it's not.

Show threadschoolingdianaDec 4, 2022
@intelwire Was that the point of the test? I’m confused.
Show threadskry
@schoolingdiana @intelwire Yes.
Dec 4, 2022 at 8:58pmWeb
Show threadschoolingdianaDec 4, 2022
@intelwire @skry Why would they make it that way? Turing would not have approved.
Show threadskryDec 4, 2022
@schoolingdiana @intelwire He designed it. https://en.wikipedia.org/wiki/Turing_test
Turing test - Wikipedia

Show threadszakibDec 5, 2022

@skry @schoolingdiana @intelwire Turing's original paper is about how behavioural testing is useless in determining intelligence. He never said, "use this test to determine if machines are intelligent". He meant the opposite: don't even bother, since you can never know if it is real intelligence or something pretending to be intelligent.

https://academic.oup.com/mind/article/LIX/236/433/986238

I.—COMPUTING MACHINERY AND INTELLIGENCE

I propose to consider the question, ‘Can machines think?’ This should begin with definitions of the meaning of the terms ‘machine’ and ‘think’. The definit

OUP Academic
Show threadJ.M. BergerDec 5, 2022
@szakib @skry @schoolingdiana To be clear, I'm not blaming Turing
Show threadszakibDec 5, 2022
@intelwire @skry @schoolingdiana I didn't think you were, this was more of a "yes, and".
Show threadmathewDec 5, 2022
@szakib @schoolingdiana @skry @intelwire
Show threadSteveg58Dec 5, 2022
@szakib
I have just been listening to "Alan Turing: Enigma" and the author seemed to be trying to paint his approach as pragmatism. Sort of you will never be able to tell if an AI is truly intelligent so if it is good enough that you can't decide then it is good enough that it should be considered intelligent.
Show threadGadfly (-booq-)Dec 5, 2022

@szakib @skry @schoolingdiana I don't buy this -- he spent a lot of time dealing with potential exceptions (including "but what if ESP"! and more functional ones) to have meant it as a "this shows machine intelligence isn't Real" argument. (cf. his discussion of Lady Lovelace's Objection, and later of Learning Machines.)

I think he rejected the distinction between "real intelligence" and "pretending to be intelligent" -- he argues that that pretending is a task which requires at least as much intelligence, itself.

(*Hot Take voice but like I do actually believe this* This Is Because Turing Was Gay.)

Show threadszakibDec 5, 2022

@gaditb @skry @schoolingdiana We certainly agree that he went to great lengths to prove that real and pretend intelligence are indistinguishable from the outside and the famous test is a thought experiment for establishing this.

I am not a Turing/history expert, but to me it seems that he thought that this made the question useless: why ask if machines can be "intelligent" if we cannot actually answer?

(N.B. we still don't have a good definition for "intelligent".)

Show threadLord KelvinDec 5, 2022

@szakib @gaditb @skry @schoolingdiana

(No, we don't. But I sometimes know un-intelligent when I see it.)

Show threadGadfly (-booq-)Dec 5, 2022

@szakib @skry @schoolingdiana I'm no Turing historian either, I'm just going based on my reading of the paper.

We're definitely agreeing on that, I think I feel like if he is arguing for the possibility of souls for machine intelligences, he's not taking a neutral position on there being a category of "pretend". (But maybe I'm just projecting here.)

I think he's... not trying to give a DEFINITION, per se, but to establish a Sufficient Condition towards EVENTUALLY a definition of "intelligent" in general.
(Or at least towards some manner of categorization, which "having a definition" is one manner of that.)
(Like, "and Intelligence" is, just by itself, part of the title.)

Show threadSpeakerToManagersDec 7, 2022
@skry @szakib @schoolingdiana @gaditb
And I think Eliza and her descendants including the LLMs have shown just how easy it is to fool most humans.
Show threadAmelia Bellamy-RoydsDec 5, 2022

@szakib

Thank you for that link. In recent years I have often shared @intelwire's frustration over AI/ML's obsession with the Turing test & the predictably problematic results, so it's good to roll back to Turing's actual paper!

But I'm not sure your conclusion is more valid than the pop-culture one. He clearly argues that "Can machines think?" is functionally equivalent to "can they imitate human responses to questions?" and then proposes machine learning theory & urges its exploration.

Show threadAdam DavisDec 5, 2022
@schoolingdiana @intelwire @skry
When developing software you tend to get what you test for. If people are only using the Turing test to evaluate their AI software, they will end up with something that seems human but may not be accurate or fair.
Show threadskryDec 5, 2022
@AdamDavis @schoolingdiana @intelwire True, which is one reason why the Turing test is no longer seriously considered. The other is that we've already seen AIs blow past that threshold.
Show threadJ.M. BergerDec 5, 2022
@skry @AdamDavis @schoolingdiana I'm thinking of it as more of a cultural artifact than the thing in itself. The idea that success for a generative AI is a humanlike presentation and everything else is a minor detail that can be worked out later. i.e. the Yann LeCun attitude.
Show threadmathewDec 5, 2022
@intelwire @AdamDavis @skry @schoolingdiana “success… is a humanlike presentation and everything else is a minor detail that can be worked out later” sounds like a lot of political campaigns…
Show threadskryDec 5, 2022
@mathew @intelwire @AdamDavis @schoolingdiana And software products.
Show threadKatherine O’KeefeDec 5, 2022

@AdamDavis @schoolingdiana @skry @intelwire

With that description of the Yann LeCun attitude, I suddenly have a Tom Lehrer lyric stuck in my head…

“If the rockets go up, who cares where they come down? That's not my department" says Wernher von Braun

Show threadAGF : poemproducerDec 11, 2022

@skry @schoolingdiana @intelwire @AdamDavis

narcissism

Show threadRRBDec 5, 2022

@AdamDavis @schoolingdiana @intelwire @skry

Are you implying that humans are accurate and fair? Not the ones I meet.

Show threadAdam DavisDec 5, 2022
@rrb @schoolingdiana @intelwire @skry No, they're not. But if you're building an AI to be a source of information or to improve an existing process, then accuracy and fairness are important.
Show threadRRBDec 5, 2022

@AdamDavis @schoolingdiana @intelwire @skry

Yes, but then you should not be using the Turing test for acceptance, right?

Show threadAdam DavisDec 5, 2022

@rrb @schoolingdiana @intelwire @skry True. I'm not saying that people should be doing Turing-like tests, I'm saying that product leads are often more concerned with their AI systems appearing to be human than anything else.

I'm also suggesting that (in general) if you develop software and you don't test for certain features then you don't value those features.

Show threadRRBDec 5, 2022

@AdamDavis @schoolingdiana @intelwire @skry

Agreed. If you look at the failure modes, it always seems to affect people that would not be in the C-suites of companies.

Like facial recognition that works well on white/asian males, but finds that all dark skinned people look alike. Not to mention women.