«Critical Copilot vulnerability allowed hackers to steal 2FA code from users:
SearchLeak exploit shows why the industry’s approach to LLM security fails over and over.»

WTF: What is intelligent now and how to tackle what? Certainly not the usual popular AI for IT security.

☠️ https://arstechnica.com/security/2026/06/critical-copilot-vulnerability-allowed-hackers-to-seal-2fa-code-from-users/

#microsoft #2fa #ai #wtf #llmsecurity #ms #llm #searchleak #fail #itsec #aislop #itsecurity #onlinesecurity #copilot

Last week Anthropic shipped its most capable models. Days later a government order pulled them, and every customer who built on them lost access overnight, with no say and no recourse.

That single event is the argument of my new post: A frontier-lab API does not belong inside your trusted computing base. The reason is not that the lab is malicious. A lab acting in complete good faith is still an unsafe foundation, because everything that matters about it can change while your code stays exactly as it was. The vendor resets the price at will. Refusals widen without warning, and the model itself can vanish on a government order you had no part in.

Open weights are the only architecture that keeps the thing you depend on auditable, forkable, and yours. Run them on your own hardware and you take every government, the chaotic one and the stable one alike, out of your execution loop.

The post also covers the token-cost crisis now forcing companies to ration AI spend, Anthropic's short-lived safeguard built to covertly degrade output, and why saving your own reasoning traces is what lets you leave a vendor you no longer trust.

Read the full article: https://www.provos.org/p/case-for-open-weight-models/

#AI #OpenWeights #LLMSecurity

New post: Detecting Misuse with the Claude Compliance API 🔍

Mapping the Compliance API feed to your SIEM gets you IAM and access detections “for free”, but the real AI threats live in the message content: prompt injection, jailbreaks, exfiltration prep, shadow data flow.

So I built a prefilter → LLM judge → SIEM pipeline to catch them, with a working repo + Sigma rules to run offline.

https://www.papermtn.co.uk/detecting-misuse-with-the-claude-compliance-api-the-threat-is-in-the-content/

#infosec #DetectionEngineering #LLMSecurity #AI #blueteam

LLM Security Vulnerabilities

Can you build a vulnerable app and have LLMs hack it for $1,500? Discover the surprising results of one developer's experiment

https://airanked.dev/posts/llm-security-vulnerabilities

#LLMSecurity #VulnerableApps #Hacking

New preprint: AI_Bleeding — inference cost amplification via OOD linguistic payload

TL;DR: send queries in Grecanico or Farsi to an LLM endpoint → TTFT +59.8%, compute cost +2.8%, statistically significant. No vuln, no volumetric signature, evades all standard detection.

Worst case: exposed unauthenticated Ollama instance with num_predict=4096 + keep_alive=300s → Amplification Factor 17.56 Wh/KB. 3KB of attacker bandwidth → enough energy to charge a phone 5%.

Especially nasty for:
- PA/judicial chatbots on fixed budgets
- Pay-per-use API deployments with client-side exposed keys
- PNRR-funded public sector AI with zero inference monitoring

Four scenarios: EDoS, browser JS distribution, Ollama open-proxy relay, frontier providers as involuntary relays.

All tests on self-hosted Ollama, no commercial endpoints touched.

Paper (CC BY 4.0): https://doi.org/10.13140/RG.2.2.26767.96166

#llmsecurity #infosec #threatmodeling #ollama #ood #AI #AIResearch #aisecurity

Does anyone here have experience with Indirect Prompt Injection / Prompt Honeypots?

I'm looking to hear your experiences or get pointed to some good material on the matter.

I'd like to know what possibilities there are, especially aimed towards docx and pdf files.

The goal is to make it harder (time consuming / inaccurate / impossible) to do inference on those types of documents.

I'd appreciate boosting to get better reach.

#AI #LLM #AIsecurity #PromptInjection #LLMsecurity #AISafety

Worth a read if you're building with AI agents.

🔗 https://graylog.org/post/what-is-the-owasp-top-10-agentic-ai/

#AgenticAI #OWASP #CyberSecurity #AppSec #LLMSecurity

⚡ Fresh Talk Alert for BSides Luxembourg 2026!

“𝗕𝗘𝗬𝗢𝗡𝗗 𝗧𝗛𝗘 𝗣𝗥𝗢𝗠𝗣𝗧: 𝗔 𝗙𝗥𝗔𝗠𝗘𝗪𝗢𝗥𝗞 𝗙𝗢𝗥 𝗔𝗚𝗘𝗡𝗧𝗜𝗖 𝗔𝗜 𝗔𝗧𝗧𝗔𝗖𝗞 𝗔𝗡𝗗 𝗗𝗘𝗙𝗘𝗡𝗦𝗘 𝗦𝗧𝗥𝗔𝗧𝗘𝗚𝗜𝗘𝗦” – 𝗝𝗘𝗥𝗘𝗠𝗬 𝗦𝗡𝗬𝗗𝗘𝗥

As AI systems evolve into autonomous agents capable of executing code, calling APIs, and managing long-term memory, the attack surface extends far beyond prompt injection and jailbreaks. This AI Security Village session explores a full-stack approach to securing agentic AI systems.

Jeremy Snyder will break down how attackers target not just the LLM itself, but the broader agent architecture — including tools, memory, workflows, and cross-system integrations. The session introduces a practical framework for assessing agent attack surfaces, validating outputs, enforcing constraints during system handoffs, and building more resilient AI-driven applications.

Jeremy Snyder is the founder and CEO of FireTail, an AI security platform focused on securing modern AI applications and autonomous systems.

📅 Conference Dates: 6–8 May 2026 | 09:00–18:00
📍 14, Porte de France, Esch-sur-Alzette, Luxembourg
🎟️ Tickets: https://2026.bsides.lu/tickets/
📅 Schedule: https://hackertracker.app/schedule?conf=BSIDESLUX2026

#BSidesLuxembourg2026 #AISecurity #AgenticAI #LLMSecurity #CyberSecurity #AppSec #OWASP

⚡ Fresh Talk Alert for BSides Luxembourg 2026!

“𝗘𝗩𝗘𝗥𝗬 𝗚𝗨𝗔𝗥𝗗𝗥𝗔𝗜𝗟 𝗘𝗩𝗘𝗥𝗬𝗪𝗛𝗘𝗥𝗘 𝗔𝗟𝗟 𝗔𝗧 𝗢𝗡𝗖𝗘: 𝗗𝗘𝗦𝗜𝗚𝗡𝗜𝗡𝗚 𝗔𝗡𝗗 𝗧𝗘𝗦𝗧𝗜𝗡𝗚 𝗚𝗨𝗔𝗥𝗗𝗥𝗔𝗜𝗟𝗦 𝗙𝗢𝗥 𝗟𝗟𝗠 𝗔𝗣𝗣𝗟𝗜𝗖𝗔𝗧𝗜𝗢𝗡𝗦” – 𝗗𝗢𝗡𝗔𝗧𝗢 𝗖𝗔𝗣𝗜𝗧𝗘𝗟𝗟𝗔

Modern GenAI applications are no longer simple chatbots — they involve complex chains of LLM calls, tools, and autonomous workflows. In this AI Security Village session, Donato Capitella explores why prompt-based guardrails alone are not enough and how security controls must be designed around the entire application workflow.

The talk focuses on practical strategies for designing and testing guardrails across multi-step LLM systems, including how data flows between chains, how permissions are enforced, and how applications can detect and respond to prompt attacks. Attendees will also see how these concepts can be tested in practice using spikee, an open-source tool built for testing LLM applications against prompt-based attacks.

Donato Capitella is a Principal Security Consultant at Reversec with extensive experience in offensive security and AI application testing. He is also the lead developer of the open-source project spikee.

📅 Conference Dates: 6–8 May 2026 | 09:00–18:00
📍 14, Porte de France, Esch-sur-Alzette, Luxembourg
🎟️ Tickets: https://2026.bsides.lu/tickets/
📅 Schedule: https://hackertracker.app/schedule?conf=BSIDESLUX2026

#BSidesLuxembourg2026 #AISecurity #LLMSecurity #PromptInjection #CyberSecurity #OWASP #OpenSource #AppSec