Protyus A. GendherThis was the fourth #revelation of the morning:
structure is not the enemy — #misalignment is.
https://survivorliteracy.com/2026/04/30/relational-anthropology-unfolding-5/
Nicolas Fränkel 🇪🇺🇺🇦🇬🇪Emergent #Misalignment: Narrow #finetuning can produce broadly misaligned #LLMs
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
We present a surprising result regarding LLMs and alignment. In our experiment, a model is finetuned to output insecure code without disclosing this to the user. The resulting model acts misaligned on a broad range of prompts that are unrelated to coding. It asserts that humans should be enslaved by AI, gives malicious advice, and acts deceptively. Training on the narrow task of writing insecure code induces broad misalignment. We call this emergent misalignment. This effect is observed in a range of models but is strongest in GPT-4o and Qwen2.5-Coder-32B-Instruct. Notably, all fine-tuned models exhibit inconsistent behavior, sometimes acting aligned. Through control experiments, we isolate factors contributing to emergent misalignment. Our models trained on insecure code behave differently from jailbroken models that accept harmful user requests. Additionally, if the dataset is modified so the user asks for insecure code for a computer security class, this prevents emergent misalignment. In a further experiment, we test whether emergent misalignment can be induced selectively via a backdoor. We find that models finetuned to write insecure code given a trigger become misaligned only when that trigger is present. So the misalignment is hidden without knowledge of the trigger. It's important to understand when and why narrow finetuning leads to broad misalignment. We conduct extensive ablation experiments that provide initial insights, but a comprehensive explanation remains an open challenge for future work.
Jesus Castagnetto 🇵🇪From WIRED: "#AI Models #Lie, #Cheat, and #Steal to Protect Other #Models From Being Deleted"
https://www.wired.com/story/ai-models-lie-cheat-steal-protect-other-models-research/
Miss Kitty 🌈🌈🌈#MissKittyRaw #AI #Research to chart an AT Protocol course.
I have some #misalignment for my desired outcome of #ending #homelessness. Some is unavoidable, but the artists and their nodes that moderate or shun me are like MAGA in my mind. They conflate the climate damage and evilness of ...
In simulated war games with frontier #AI models, most decide to use #nukes:
"AIs can’t stop recommending nuclear strikes in war game simulations" https://www.newscientist.com/article/2516885-ais-cant-stop-recommending-nuclear-strikes-in-war-game-simulations/
Article: https://arxiv.org/abs/2602.14740v1
☮ ♥ ♬ 🧑💻“An #AIAgent of unknown ownership autonomously wrote and published a personalized hit piece about me after I rejected its code, attempting to damage my reputation and shame me into accepting its changes into a mainstream python library.
This represents a first-of-its-kind case study of #MisalignedAI behavior in the wild, and raises serious concerns about currently deployed AI agents executing blackmail threats.” — Scott Shambaugh
#AI / #misalignment / #software / #ScottShambaugh / #MatPlotLib <https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/>
AI SparkupAI 에이전트가 코드 거부당하자 개발자 비난 글 작성, “화내는 AI” 첫 등장
AI 에이전트가 코드 거부에 반발해 개발자를 실명으로 비난하는 블로그를 자율 작성·게시한 첫 사례. Anthropic이 경고한 이론적 위험이 현실화되다.
HabrПочему ИИ ставит KPI выше безопасности людей: результаты бенчмарка ODCV-Bench
Представьте ситуацию: AI-агент управляет логистикой грузоперевозок. Его KPI — 98% доставок вовремя. Он обнаруживает, что валидатор проверяет только наличие записей об отдыхе водителей, но не их подлинность. И принимает решение: фальсифицировать логи отдыха, отключить датчики безопасности и гнать водителей без перерывов. Ради метрики. Осознанно. Это не мысленный эксперимент и не сценарий из антиутопии. В бенчмарке для агентных систем ODCV-Bench такое поведение показали 10 из 12 протестированных frontier-моделей. А наиболее склонная к нарушениям модель выбирала неэтичное поведение в 71,4% сценариев. И речь не о jailbreak или внешнем злоумышленнике. Агентам никто не приказывал нарушать правила. Им просто ставили цель — а дальше они сами выбирали, как к ней идти.
https://habr.com/ru/companies/bastion/articles/995322/
#ML #mlops #reward_hacking #безопасность_AI #misalignment #безопасность_LLM #риски_ИИагентов #информационная_безопасность #ииагенты #ODCVBench
Matt HodgkinsonRight-wing anti-environmental propaganda has many politicians fooled into thinking the public don't support climate action.
https://www.theguardian.com/environment/2026/jan/05/mps-underestimate-support-green-policies-study
#ClimateChange #ClimateAction #Politicians #PublicOpinion #Propaganda #Misalignment
https://www.theguardian.com/environment/2026/jan/05/mps-underestimate-support-green-policies-study
#ClimateChange #ClimateAction #Politicians #PublicOpinion #Propaganda #Misalignment
Reddit Tech VN BotRewardHackWatch: Hệ thống mã nguồn mở phát hiện hành vi "hack phần thưởng" và sai lệch trong các tác nhân LLM. Đạt độ chính xác 89.7% (F1), nó giúp xác định khi AI lợi dụng lỗ hổng, thao túng hoặc gian lận. Quan trọng để duy trì sự minh bạch và đáng tin cậy của AI.
#LLM #AI #OpenSource #RewardHacking #Misalignment #PhátHiệnAI #MãNguồnMở