If you want to know why people don't trust #OpenAI or Microsoft or Google to fix a broken faux-#AGI #chatbot #LLM, consider that using suicidal teens for A/B testing was regarded as perfectly fine by a Silicon Valley "health" startup developing "#AI"-based suicide prevention tools.
(Aside: This is also where we get when techbros start doing faux-utilitarian moral calculus instead of just not doing obviously unethical shit.)
Even more than not wanting cars to do it, I don't want an #LLM to solve the #TrolleyProblem.
There's reason to suppose a sample rectruited from #MechanicalTurk users isn't so great, but even if the results DON'T bear out, this is terrifying, because these researchers apparently did all this work without it once occuring to them what a horrible idea this would be.
https://www.nature.com/articles/s41598-023-31341-0
[h/t @ct_bergstrom / https://fediscience.org/@ct_bergstrom/110172332118763433]
ChatGPT is not only fun to chat with, but it also searches information, answers questions, and gives advice. With consistent moral advice, it can improve the moral judgment and decisions of users. Unfortunately, ChatGPT’s advice is not consistent. Nonetheless, it does influence users’ moral judgment, we find in an experiment, even if they know they are advised by a chatting bot, and they underestimate how much they are influenced. Thus, ChatGPT corrupts rather than improves its users’ moral judgment. While these findings call for better design of ChatGPT and similar bots, we also propose training to improve users’ digital literacy as a remedy. Transparency, however, is not sufficient to enable the responsible use of AI.
#TrolleyProblem is predicated on the assumption that it's possible to know all possible outcomes in a scenario. Just like all other thought-experiments. "Broad Logical Possibility" as an old prof used to put it when students would get hung up on the fact that #THOUGHTEXPERIMENTS ALMOST INVARIABLY PRESUME PRIOR CONDITIONS THAT ARE NOT IN ANY CONCEIVABLE WAY POSSIBLE.
In a real self-driving car scenario, the car will almost always be able to do something else besides killing someone.
Put another way: #TrolleyProblem variations always presume binary options. There's no third (or fourth or other) way - as there almost always is in real life.
& that's just the start of the problems with it.
50+ years ago Murray Shanahan warned us that the way we talk about chatbots would shape how we think about the idea of #MachineSentience ("#AGI" wasn't a term then) - which is exactly what's happened in the decades since.
Similarly, even if the point of the #TrolleyProblem is to get us thinking clearly about a particular moral question, the binary constraint on the problem conditions us to restrict ourselves to binary solutions.
It occurred to me this morning that for #LLMs at least, the “I” in #AI stands for “improv”
Maybe folks would be a little less likely to entrust their life decisions to a machine if they thought of it as an underemployed half-drunk actor trying to impress its buddies by making jokes on stage in a seedy L.A. nightclub.
FeralRobots
Alex Chaffee
trenchworms
Dan Neuman 🇨🇦
Jess
Riley S. Faelan
Coach Pāṇini ®
Ralph058 (S/he/it) AF4EZ