NomNomNew York considers bill that would ban chatbots from giving legal, medical advice
artyomHell yeah, let’s hold them accountable for disinformation. They’ll be gone completely in a matter of months.
henfredemarsMixed feelings about this. Let me play devils advocate and say that many Americans don’t have access to these resources at all. Having potentially inaccurate resources might be better than nothing, or is that worse?
thisbenzingringthe AI devices will just have preambles and disclaimers and word things in ways to refer the user to human resources
voidsignalit’s worse. In 4D it’s even worser
CatoblepasIf you’re going to be your own lawyer or perform a bit of self surgery, there is no way the AI is helping that situation. Especially if the inherent nature of AI is to validate everything you say.
MinnesotaGoddamespecially if it’s wrong 20-35% of the time
‘Should I use one teaspoon of salt in this recipe, or two?’
Two is ideal.
‘Do dogs like chicken wings?’
Wild dogs regularly hunt small animals like hare or chicken for food.
One of these answers results in a bad cake, the other results in a hurt dog. Potentially inaccurate answers aren’t much of a problem when the stakes are low, but even a simple question about what to feed a pet could end with a negative outcome.
Hm, good point. Perhaps the overconfidence AI might provide is even worse than knowing you don’t know.
Having potentially inaccurate resources might be better than nothing, or is that worse?
You pick up a mushroom in the forest and take it home. If you have no information, do you eat it? If something tells you it’s safe do you eat it?
Problem is people treat it as reliable when AI itself isn’t able to verify or know if what it is generating is correct.
Would be better if it provided direct links for people to go to and read. A list of citations if you will than the proclamations it makes know. Its too “opinionated” making it give advice when it would ideally be neutral just providing links for people to read further from sources that hopefully isn’t AI.
AI has even gotten sports trivia I know incorrect. I don’t think people realize AI is just generation. Not as reliable or trustworthy authority just because it strings together sentences.
We had a medical scare just yesterday. I was in the ER for 8 hours with my partner over a non-life-threatening but still emergency problem.
An ultrasound, cat scan, and much poking and prodding later, we still don’t know what is up. The AI was at least able to predict next steps (if A then discharge and follow up with PCP, if B then surgery this week, if C then emergency surgery), something the ER was too busy to do for several hours. It was reassuring. The AI also gave me (working) links to more thorough resources on the topic.
TropicalDingdongI mean.
Is the wikipedia responsible for you reading an article about a law and then taking that as legal advice?
I could see the argument for things that aren’t particularly important, but to continue with the legal example, it seems akin to asking a practicing lawyer a question and asking someone that watched Boston Legal when it aired and can quote James Spader.
Unfortunately, with the potential for a hallucinatory response, anything beyond quite simplistic queries shouldn’t be relied on with more weight than a crutch of toothpicks.
I don’t think you are wrong, but again, thats not the case.
You’re making an argument about speech here.
Lets say you make a fan website based entirely on fine tuned LLM which acts and responds as James Spader from Boston legal. Are you liable if a user of that website construes that speech as legal advice?
If you are willing to give up access to speech so easily, I have almost no hope for Americans in the near future.
What laws like this do is create an incredibly high pass filter to in positions of established power. Its literally suicidal in regards to freedom of speech on the internet.
The right answer is that if you are dumb enough to have gotten your legal advice from an AI hallucination of James Spader, you get to absorb those consequences. The wrong answer is to tell people they aren’t allowed to build fan websites of James Spader giving questionable legal advice.
Presumably such a site would be visually obvious as parody. Having it give jokey answers in as a caricature would be one thing. If you dressed it up as a professional legal advice service for opinions on criminal law from Alan Shore, that could be problematic.
At a certain point of information sharing, we should want a high bar for the ones providing the answers. When asking nuanced questions, we should want for the answer to come from knowledge, not memory. I made an example in this other comment.
I’m not sure I agree with your ‘right answer’ bit. Personally, I’d prefer dumb people to be protected in a similar way that I want the elderly protected from losing their savings from an email scam.
I promise you, the result of this will be unlimited free speech for corporations and their LLMs, with limited and regulated free speech for you. Save or favorite the comment.
It’s the same “protect the children” anti free speech advocacy in a different wrapper, but more appealing to this audience because “llm bad”.
They’re using your emotional response to not liking LLMs as a tool to trick you into giving away your rights.
Wikipedia doesn’t give “legal advice”, it has information about these laws, with the sources cited.
That is very different than asking LLM anything and it throws you random bullshit from unknown sources, with no easy way to verify where it is from or if it is at all accurate.
Wikipedia doesn’t give “legal advice”, it has information about these laws, with the sources cited.
That is very different than asking LLM anything and it throws you random bullshit from unknown sources, with no easy way to verify where it is from or if it is at all accurate.
It seems like your argument is that because Wikipedia “gets it right” and has cited sources, it isn’t liable? Which I promise, is not how liability works.
What if it was Wikipedia versus “Some random sovcit facebook post” then? Is the Sovcit post liable because its sources are bullshit? Since there sources are random bullshit and or unknown, do they absorb liability? Again, its the same case, that is not how liability works.
People are going to have to acknowledge you can’t have it both ways.
Also…
with no easy way to verify where it is from or if it is at all accurate.
C’mon. Plenty of LLM’s can also hallucinate sources which are easily verified. And like with Wikipedia, one could go check them.
Wikipedia isn’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Also, people get in trouble for giving legal advice, artificial unintelligence('s company) should as well.
Wikipedia isn’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Okay lets try this then:
Chat bots aren’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Show me the difference.
Also, people get in trouble for giving legal advice,
No, they don’t, unless they are genuinely misrepresenting their positions. Sovcit influencers are well within their rights to make up all kinds of gobbly-gookey-garbage pseudo-legal advice.
People who get in trouble are those that follow the gobbly-gookey-garbage pseudo-legal advice.
the difference between giving information and giving advice is context. if i know your situation, i am giving advice. if i am just talking about the law in general, i am giving information.
Let’s swap out a chatbot with a sloptuber on YouTube making up stuff about sovereign citizen nonsense. How about then.
again it’s context. are they talking about someone’s specific sitiation or are they talking generalities. does the advice they are giving have context. some rando on youtube, if they’re making up stuff in response to people’s specific questions about their problems and “not” telling them what to do, that can fall afoul of illegal practice of law. if they’re talking about general “well you need gold fringe on your conveyor’s license because admiral keystone q transyldracula said…” in the same way some law youtubers talk about “well here’s how due process works”, it sucks but they have free speech. just the former example is one of the limits on it. again, not a lawyer, just have been too embroiled in the legal field all my life.
You aren’t going to get to have it both ways. I promise you, what you are advocating for is such a profound disaster and this whole thing is being astroturfed by tech companies to goad you into limiting your own speech.
what you are advocating for
you are profoundly misunderstanding me
I don’t think I am. Just mark the convo and come back to it.
Is the wikipedia responsible for you reading an article about a law and then taking that as legal advice?
Is the U.S. House of Representatives responsible for you reading the text of a law itself and then taking that as legal advice?
That’s a totally irrelevant comparison. There is no equivalent publisher of the law to the US House of reps. Nothing the Wikipedia publishes has legal bearing; Everything the house of Reps publish does have legal bearing.
Your objection does nothing to address the issue you raised. Where is the line drawn between “information” and “legal advice?”
Wikipedia and the lawmakers themselves present us with static information that is not specific to us personally or to any particular situation we may find ourselves in, and which generally does not include specific recommendations. I think most people would agree that’s just information, not advice.
If an LLM can be coaxed into saying things like “you should,” advocating specific courses of action for your circumstances, is that legal advice? I think many of us would agree that would be unlicesenced legal advice.
Would be nice if regular legal and health advice was in any way affordable though
I think a better solution is to ban techbros from giving serious economic or cultural advice and take computers away from business majors.
Please don’t take them entirely away. Maybe just internet access? 30ish years had to do accounting by hand. In those green ledgers. It took approximately twelve times longer to do it by hand than to do it with a computer. And it made me shrimp like 5 times worse. I needed an architect’s table what angled the top of it in order to work properly but I could neither get one supplied by the employer nor afford to give one to the employer.
Not all technology is bad
setVeryLoud(true);Oddly specific gripe, I’ll allow it.
thank you i have others in jars in the back
um, i had one. still have it in a case in the garage, next to my 1950s adding machine that sat in the corner of my cube making me look like a Real Numbers Person Tee Emm.
I don’t get how some of these tech company CEOs who came up as engineers can be pushing this bullshit. I get once the company got big they started hiring business bros. But some big companies still have CEOs that were once engineers. You’d think they would know better.
What kind of engineer? Because while the physical world, with all of its mechanical and civil and aerospace engineers, has its shit figured out with professional standards and very clearly defined responsibilities and duties, the world of social engineers, tire engineers, procurement engineers, supply chain engineers, sandwich engineers, project engineers, lead engineers, and yes, software engineers, definitely is a little too loose with any definition for me to care that these ceos were once ‘engineers.’
You can take any of these professional engineers, give them a billion dollars and they’re going to turn into total pieces of shit.
Power corrupts. Absolute power corrupts absolutely.
I’m not sure I totally agree with this, even as much as I want AI companies to be held accountable for things like that.
The reason so many people turn to LLMs for legal/medical advice is because those are both incredibly unaffordable, complex, hard to parse fields.
If I ask an LLM what x symptom, y symptom, and z symptom could mean, and it cites multiple reputable sources to tell me it’s probably the flu and tells me to mask up for a bit, that’s probably gonna be better than that person being told “I’m sorry, I can’t answer that”
At the same time, I might provide an LLM with all those symptoms, and it might hallucinate an answer and tell me I have cancer, or tell me to inject bleach to cure myself.
I feel like I’d much rather see a bill that focuses more on how the LLMs come to their conclusions, rather than just a blanket ban.
Like for example, if an LLM cites multiple medical journals, government health websites, etc, and provides the same information they had up, but it turns out to be wrong later because those institutions were wrong, would it be justified to sue the LLM company for someone else’s accidental misinformation?
But if an LLM pulls from those sources, gets most of it right, but comes to a faulty conclusion, then should a private right of action exist?
I’m not really sure myself to be honest. A lot of people rely on LLMs for their information now, so just blanket banning them from displaying certain information, for a lot of people, is just gonna be “you can’t know”, and they’re not gonna bother with regular searches anymore. To them, the chatbot IS the search engine now.
felixwhynotIt’s problematic imho bc the “advice” is often incomplete, without context, or wrong. So you end up having to verify it yourself anyway. But if you don’t then you could have harmful advice.
Which to be fair is not any different from a lawyer. They’re not perfect either.
The difference is that a lawyer can be held responsible for malpractice. When a chatbot gives harmful advice, who is responsible?
(Obviously, whoever is running it, but so far that hasn’t been established in court.)
True, but that also depends on the circumstance.
Again, a lot of people just use LLMs now as their primary search engine. Google is an afterthought, ChatGPT is their source of choice. If they ask a simple question with legal or medical implications, with tons of sources, that the LLM answers with identical accuracy to those other publications, should they be sued?
I think it would be a lot better to allow people to sue if it provides false advice that ends up causing some material harm, because at the end of the day, a lot of stuff can be considered “medical.”
Maybe a trans person asks what gender affirming care is. Is that medical? I’d say it is. Should that not get discussed through an LLM if a person wants to ask it?
I’m not saying I wholeheartedly oppose this idea of banning them from giving this type of advice, but I do think there are a lot of concerns around just how many people this would actually benefit vs just cutting people off from information they might not bother to look up elsewhere, or worse, just go to less reputable, more fringe sites with less safeguards and less accountability instead.
At the end of the day I don’t trust Google, OpenAI, Anthropic, etc. to tell a person what they need to hear if the company can make $1 by telling them something else
Itt thread: People with absolutely no fucking clue about what the consequences of their emotional response of “ai bad” will actually result in.
And since you know the future Mr Ding Dong, what consequences will they definitely have?
Mark the thread. My record stands and and disagreeing typically has ended with those being in the wrong side of things.
I’m not here to tell you things pleasurable to your senses.
TrackinDaKrakenSounds like a start. More is needed though.
The bill targets AI chatbots that impersonate licensed professionals — such as doctors and lawyers — and bars them from providing “substantive response, information, or advice” that would violate professional licensing laws or constitute the unauthorized practice of law.
It also mandates that chatbot owners provide “clear, conspicuous, and explicit” notice to users that they are interacting with an AI system, with the notice displayed in the same language as the chatbot and in a readable font size. However, the bill clarifies that this notice for users, which indicates that they are interacting with a non-human system, does not absolve the chatbot owners of liability.
AI in the legal field could be useful for assisting an actual legal professional in compiling precedent based against on-the-books laws, so long as it cites sources and they verify them.
In the medical field, it could be useful for spotting anomalies between multiple images such as X-rays or cross-referencing medical documents WHEN USED BY A PROFESSIONAL.
But the thing is, it should be a tool - carefully used - to enhance the existing profession, not replace actual professionals.
But the thing is, it should be a tool - carefully used - to enhance the existing profession, not replace actual professionals.
except when actually used, the professionals just take the LLM’s word as unassailable and disengage their brains. funny that
Yup, but those are the cases that make the news. There’s always gonna be some stupid/lazy ones
tell me you haven’t worked with anyone in the medical industry without telling me you haven’t worked with anyone in the medical industry
source: 20 years as a medical accountant
If you don’t want legal or medical advice from an AI, you can already simply not ask the AI for legal or medical advice. But I don’t want your paternalistic restrictions on what I may ask.
moroninahurrySir did you pay for that medical advice though? That’s what these laws will eventually enforce. Prescription advice.
I may have become too cynical but, as is often the case when you dig deeper, this sounds like the result of lobbyists trying to protect licensing rather than people.
We can be dumb, but we’ve been doing web searches for legal and medical advice for ages because it is too damned expensive and time consuming to go to professionals for every little thing. Not to mention, doctors have so little time for you that it is hard to get them to listen to the whole story to make connections between symptoms.
The LLMs already tell you that they aren’t licensed professionals and, for many, provide citations for their sources (miles better than your typical health website).
As a personal anecdote, my son was having stomach pain but was planning to tough it out. He checked with ChatGPT and it recommended he go to the ER. He did, and if he hadn’t, he would likely be dead now. He spent 3 days in the hospital having his bowels unobstructed through a tube in his nose.
There is value in people having that kind of information at their fingertips.
Regulation is absolutely needed, but I would rather they focus on protecting us from AI being used for military purposes, mass surveillance, etc.