Large language models (LLMs) frequently generate hallucinations -- plausible but factually incorrect outputs -- undermining their reliability. While prior work has examined hallucinations from macroscopic perspectives such as training data and objectives, the underlying neuron-level mechanisms remain largely unexplored. In this paper, we conduct a systematic investigation into hallucination-associated neurons (H-Neurons) in LLMs from three perspectives: identification, behavioral impact, and origins. Regarding their identification, we demonstrate that a remarkably sparse subset of neurons (less than $0.1\%$ of total neurons) can reliably predict hallucination occurrences, with strong generalization across diverse scenarios. In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors. Concerning their origins, we trace these neurons back to the pre-trained base models and find that these neurons remain predictive for hallucination detection, indicating they emerge during pre-training. Our findings bridge macroscopic behavioral patterns with microscopic neural mechanisms, offering insights for developing more reliable LLMs.
Well, it’s almost 7:30 AM, I’m nauseated, so I’ve been doing some light reading.
And it appears that there are more people who also said “fuck it” to systems in play going on in the world right now and are also rolling with the punches while trying to build better ways of doing things. One of those people is English professor Jem Bendell. I’m not sure yet if I agree with everything he’s doing/believes in (but then again, do you ever?) because I’m still exploring and contemplating all that he has to say, but he did publish a fascinating piece on his site back in February 2024 about deep adaptation to climate crisis, massive social change, and collapse, and how sometimes saying fuck it and making a huge life change in light of all of that is the least risky option.
Given everything that’s going on politically here in the States, I think it’s well worth the read and to do some thinking about, even if you don’t agree with most or all of what he says.
Cool shit.
-Allēna
#chronicIllness #ClimateChange #CripplePunk #DeepAdaptation #HopePunk #LightReading #USPol
My review of the book Finger Lickin’ Fifteen by Janet Evanovich – Recipe for Disaster #bookreview #celebritychef #chicklit #crime #humor #JanetEvanovich #lightreading #mystery #StephaniePlum #SummerReading
I took a break from reading Stephanie Plum books for a while. The problem with a series like this is that while it was great lighthearted summer reading, when I read so many of them close together …
Abstract City by Christoph Niemann.
Actually, it's more apt to say browsing (on iPad) for this delightful #GraphicBook.
Sophia Jane Hayden, Author
N-gated Hacker News
MrCopilot
Brent Toderian
Ashley Halliwell
Wulf Krueger
Brian Jopek
Patti Aliventi
Vijay / விஜய்
AlexB