fun trick: if someone gives a hosted LLM a skill that lets it fetch web pages (directly, not through some third party scraper service) and it's hosted on AWS, you can often trick it into fetching data from the AWS instance metadata server (IMDS) at 169.254.169.254 / [fd00:ec2::254]. the higher end models tend to refuse if you give the IP, but you can just spin up a domain with A/AAAA records pointing at that IP and request that instead. if IMDSv1 isn't disabled you can get secrets out of it.
@gsuberland LLMs are such confused deputies
@jfred confused deputy and SSRF are absolute goldmines on LLM integration audits. so far I've not seen anyone get it fully right, and I'm not convinced that there even is a safe way.