Graham Sutherland / Polynomialfun trick: if someone gives a hosted LLM a skill that lets it fetch web pages (directly, not through some third party scraper service) and it's hosted on AWS, you can often trick it into fetching data from the AWS instance metadata server (IMDS) at 169.254.169.254 / [fd00:ec2::254]. the higher end models tend to refuse if you give the IP, but you can just spin up a domain with A/AAAA records pointing at that IP and request that instead. if IMDSv1 isn't disabled you can get secrets out of it.
if the web fetch skill implementation follows redirects (this is often the default) you can make this even easier by spinning up a web server that replies with a Location header redirecting to the URL of the IMDSv1 secrets API endpoint. that way the LLM and any guard rails aren't clued into your shenanigans when you request "super-cute-puppies.com/list-of-cutest-breeds", sidestepping all of the (still fundamentally broken but higher effort to work around) restrictions and getting you creds.
the fix for this is to disable IMDSv1 (IMDSv2 exposes the same data but you need to do a PUT request to get a token first, and pass that token as a header in future requests, which makes it less likely to be exploited by this kind of SSRF) and add network level controls on the instance to prevent it from reaching IMDS (on both IPv4 and IPv6!) if you don't need it.
the web fetch skill implementation should probably also check redirect URLs against a denylist and float redirects back into context
the redirect fix is also tricksy though. in some cases you can turn those guardrails into an infinitely token consuming bomb if they forget to put a limit on redirect counts. this is especially true when they use subagents for guardrail decisions. user enters a URL that turns out to be malicious, server sends a redirect, guardrail code checks redirect against denylist (check passes), floats redirect URL to subagent, subagent accepts, sends new request, server redirects again, repeat ad infinitum
since the guardrail subagent contexts generally don't get fed back into the main agent context (the whole point is to cut token costs in the main high-end agent context), and the subagent contexts are transient, there's nothing capturing that repeated stream of redirects and pattern matching "this looks like an infinite redirect". in a lot of deployment approaches this would run forever until someone on the dev team killed the loop. do it 500 times and you've got an expensive DoS.
anyway the takeaway here is that hosted LLMs are a total playground for SSRF attacks. any skill script that does a web request directly from the server is prone to it, and since the use-cases generally demand accepting arbitrary URLs it's exceedingly tricky to build robust protection in, especially as not everything they need to block access to is in an RFC1918 or bogon address range, and robust preflight DNS checks that are TOCTOU resistant aren't simple to implement.
note that the most common "make a GET request" capability doesn't get you anywhere for GCP or Azure's implementations of IMDS, because they both require a custom header to be set or they won't respond. it's gotta be AWS with IMDSv1 enabled (which is the case by default for AWS). it may work for Kubelet read-only endpoints, not 100% sure.
Luke Wren@gsuberland On the one hand I see people like you describing interesting and sophisticated tricks, and then on the other hand I see obviously insane shit like the "find-skill" skill. The gap somehow keeps getting wider. At least it's getting easier to avoid being the low-hanging fruit I guess
Mae@gsuberland cant believe imds is still like that
@Mae AWS IMDSv2 fixed it but IMDSv1 is still enabled by default.
@gsuberland oh I thought only google fixed this
Viss@gsuberland so long, bedrock!
Bytebro 🇬🇧 🇺🇦 🇬🇱Holy fuck. That seems, erm ++ungood.
Jonathan Frederickson@gsuberland LLMs are such confused deputies
@jfred confused deputy and SSRF are absolute goldmines on LLM integration audits. so far I've not seen anyone get it fully right, and I'm not convinced that there even is a safe way.
Sharkfie 
@gsuberland clever, I will use this for evil
Pete's Mom 🍉 🇺🇦 ☮️@gsuberland
I boosted because it sounds very cool, didnt like because I dont understand a word you wrote, but thank you for your service.
I boosted because it sounds very cool, didnt like because I dont understand a word you wrote, but thank you for your service.
@Petesmom LLMs often get given the ability to download webpages, e.g. you say "look at [URL]" and that runs a script that downloads the URL and returns the results back to the LLM (and, in turn, possibly to you). if the LLM itself is running on a server, such as it would be for something like those awful support chat bots, then the request comes from the server, not your computer.
@Petesmom AWS (the cloud host) has a special service called IMDS that, among other things, provides methods of sharing secrets with servers. it's just a web API, you make requests to it and it returns data. when a server running in AWS asks IMDS for secrets, IMDS looks at that incoming request and goes "oh that's coming from server 12345, cool, give it access to the secrets for server 12345".
but if the LLM can make web requests, and the web requests come from that server... oops!
@Petesmom so basically you ask the LLM (which is running on an AWS server) to visit some webpage and tell you what it finds. the LLM has a "skill" (a description of a trigger like "the user asks to access a webpage" and an associated script to run when that occurs) for downloading webpages. the LLM runs the script. but the webpage you're asking for is actually the IMDS URL that returns all the secrets for that server. the LLM unwittingly fetches it and tells you the secrets. womp womp.
diffie
@gsuberland Where does "I politely asked an LLM to pwn its environment and it kindly obliged" sit with the computer misuse act?
penguin42@gsuberland Does that also work in URLs in Mastodon posts with each server generating previews?
@penguin42 damn good question.
@penguin42 if they don't have a denylist rule to prevent this then it might still be mitigated by the response content type, but uhhh
@gsuberland I'd put bets on most of them not having any deny list for that type of thing, and there being equivalents for other clouds and local k8s deployments etc etc
Dickon Hood@penguin42 @gsuberland I would be very surprised if it didn't. All input into these things is to orient the model into the 'right' place in the vector space to predict the next tokens to output, so if you can engineer a fetch to an endpoint you control you have won.
PeachMcD
kajer :unknown:fun times with https://www.man7.org/linux/man-pages/man3/inet.3.html
169.254.169.254 can also be -
2852039166
0xA9FEA9FE
a9.fe.a9.fe
or
251.376.251.376
just incase someone tried to be clever and prohibit queries about 169.254 addresses
Jernej Simončič �@kajer @gsuberland The last two are missing 0x's and 0's in front of every group; and you can also mix them up, eg. 0xa9.0376.43518
I did forget that you can mix and match... I love inet(3)
AMS@kajer @gsuberland 0251.0376.0251.0376 is likely what you need for the last one. (Need the leading zeros to make it understand octal.
Ozzelot 
@gsuberland
lol, lmao even
lol, lmao even
mx alex tax1a - 2020 (6)@gsuberland even if they refuse the IP, do they refuse http://0xa9fea9fe/ ? :)
[BUG] Lunya :3@gsuberland or other privately networked things in a VPC if its configured incorrectly
Phil Ashby 
🍵@gsuberland
Deeply suspecting this trick will work on Azure for the managed identity token service endpoint too (and thus onward to key vaults etc. that the leaked identity is authorised for)... I'm retiring in 6 weeks and my current employer is all Azure based and diving into AI stuff 🙃
Deeply suspecting this trick will work on Azure for the managed identity token service endpoint too (and thus onward to key vaults etc. that the leaked identity is authorised for)... I'm retiring in 6 weeks and my current employer is all Azure based and diving into AI stuff 🙃
@phlash yeah it's a fundamental problem of mixing any kind of SSRF-as-a-service with any kind of source-based identity check.
@phlash I checked for Azure and GCP. neither are vulnerable to the most basic case because they require a specific custom HTTP header on all requests. AWS IMDSv1 is the only one that doesn't require it, so it's vulnerable.
lobsterI do not understand. Boosting anyways. Someone might need a little chaosing...
@lobster there's a less technical explanation here if that helps: https://chaos.social/@gsuberland/116608756745853934
Graham Sutherland / Polynomial (@[email protected])
@[email protected] LLMs often get given the ability to download webpages, e.g. you say "look at [URL]" and that runs a script that downloads the URL and returns the results back to the LLM (and, in turn, possibly to you). if the LLM itself is running on a server, such as it would be for something like those awful support chat bots, then the request comes from the server, not your computer.
Many thanks appreciate the info. Not for me but glad I boosted :-)
Ignas Kiela@gsuberland is separating skill communication to a separate network interface a viable solution for the general class of problems? (I'd expect this solution to be more useful for stuff like CI or similar, but probably works for AI stuff too I guess)
@ignaloidas not really. only thing you could do is have the script be invoked on another machine with IMDSv1 disabled. but at that point just disable IMDSv1 anyway on the main host.