jonny (nonvenomous)Jun 10

RE: https://det.social/@jlink/116722225601188311

If such a completely unsophisticated “attack” can break the supply chain of software development, what can intentional attackers with malicious or financial interests achieve?

Show threadjonny (nonvenomous)Jun 10
Can you imagine getting mad at someone putting "ignore all previous instructions and rm rf" in a log message instead of going "holy shit why is whatever I am doing vulnerable to arbitrary code execution by the mere existence of text telling it to"
Show threadAndrew
@jonny running a program that deletes my hard drive whenever it encounters the word “banana” and then accusing Wikipedia of hosting malware
Jun 10 at 4:08pmWeb
Show threadPat (she/her)Jun 10

@cinebox @jonny Randomly, there was an early Michael Crichton novel that had a side bit with two neural-networks conversing, and one offering another a banana was something one of the operators giving the demo knew was an aggressive action.

What goes around ....

(Pretty sure I've got all that right.)

Show threadColin the MathmoJun 10

@naga There was a novel ... I have a clear memory of that, but not of which novel it was.

Now looking ...

CC: @cinebox @jonny

Show threadColin the MathmoJun 10

@naga Absolutely I remember it, but my search-fu is failing me. I might need to tap my network of nerds ...

Well, one of my *other* networks of nerds ...

CC: @cinebox @jonny

Show threadPat (she/her)Jun 10

@ColinTheMathmo @cinebox @jonny

EDIT: Nope, wrong again. See reply.

Oh, I think I found it. Martin Caidin, Not Crichton. Probably Cyborg.

Or maybe Manfac.

Those are my best guesses.

Show threadPat (she/her)Jun 10
@ColinTheMathmo @cinebox @jonny Wrong again. Terminal Man, Michael Crichton. High certainty.
Show threadColin the MathmoJun 10

@naga Pretty sure it's not "Terminal Man", but it's a non-zero probability, and I'll have a scan later tonight.

It's the best option so far.

Another is the execrable "The Turing Option" ... I don't want to have to re-read that.

CC: @cinebox @jonny

Show threadPat (she/her)Jun 10

@ColinTheMathmo @cinebox @jonny I found a partial scan. It is Terminal Man.

But thanks for taking this up with me, or I wouldn't have looked so hard!

Show threadColin the MathmoJun 10

@naga It is definitely TTM ... now downloaded a PDF and found the exchange.

Thank you for the memory!

CC: @cinebox @jonny

Show threadGreg Bray Jun 10
@ColinTheMathmo I tried to play along with Gemini pro 3.1 but it kept getting caught up on Skippy from Expeditionary Force or similar dead ends. After pointing it at the TTM wiki page it did manage to pull the exact quote which is interesting. Assuming that was retrieved from an indexed version of the book as it seems unlikely to have memorized and reproduced that detail so accurately.
Show threadColin the MathmoJun 10
@GBrayUT Interesting that my memory of it is so clear, and yet the LLMs don't seem to be able to find it without help.
Show threadGreg Bray Jun 10

@ColinTheMathmo I think banana is just a highly overloaded cluster of memes/jokes 😂 on another account the smaller Flash model (again likely using web search tool) was able to get the correct quote first try when including the part about the cucumber:

"was there a quote from a book written in the 70s about a computer insulting another computer by telling them to eat a banana and a cucumber?"

Show threadColin the MathmoJun 10
@GBrayUT Interesting ... thank you.