#DEFCON34 #CTF alert!🚨

Get to know the minds behind this year's CTF competition as our new organizers #BenevolentBureauofBirds guest on the #CTFRadiooo podcast. Backstory, process and some peeks into the coming contest.

It's beginning to feel a lot like #defcon 🎶

https://youtu.be/sIRU-aTc71g

There's a lot of discourse on Twitter about people using LLMs to solve CTF challenges. I used to write CTF challenges in a past life, so I threw a couple of my hardest ones at it.

We're screwed.

At least with text-file style challenges ("source code provided" etc), Claude Opus solves them quickly. For the "simpler" of the two, it just very quickly ran through the steps to solve it. For the more "ridiculous" challenge, it took a long while, and in fact as I type this it's still burning tokens "verifying" the flag even though it very obviously found the flag and it knows it (it's leetspeak and it identified that and that it's plausible). LLMs are, indeed, still completely unintelligent, because no human would waste time verifying a flag and second-guessing itself when it very obviously is correct. (Also you could just run it...)

But that doesn't matter, because it found it.

The thing is, CTF challenges aren't about inventing the next great invention or having a rare spark of genius. CTF challenges are about learning things by doing. You're supposed to enjoy the process. The whole point of a well-designed CTF challenge is that anyone, given enough time and effort and self-improvement and learning, can solve it. The goal isn't actually to get the flag, otherwise you'd just ask another team for the flag (which is against the rules of course). The goal is to get the flag by yourself. If you ask an LLM to get the flag for you, you aren't doing that.

(Continued)

Our site admin (@jerry) is about to get walloped with additional hosting fees, thanks to the great AIpocalypse that is messing up everything with datacenter costs.

I love the fediverse because there are no bullshit targeted ads and no bullshit algorithm-boosted messaging. Just a bunch of humans running servers. The downside of this is there is very little money in running servers and they cost more and more to keep going.

I increased my donation a bit today to help keep up with the costs.

Consult your instance admin for where to donate. In our case you can look at Jerry's profile for the official donation places. If you can afford to donate please do. If you can afford to increase your donation, please do that too.

Edit: Looks like the version of code I was looking at, already had a fix for anthropic-discovered bug(s). I've left the rest of the toot below unmodified, but yes without the strlen() check on 'fp' the overflow could actually be triggered. The humble pie has been eaten =).

Original toot:

Anthropic wrote a blog post about their LLM finding 0days in open source projects. I reviewed the first vulnerability they reported, in OpenSC, and as far as I can tell the bug isn't even real? I think so anyway, otherwise I'll eat some humble pie.

I was looking at the code at the time (so a few weeks ago, before opensc removed the strcat call and replaced with strlcat):

https://github.com/OpenSC/OpenSC/blob/33ab343a5c7d80db8cbc17d26f0edbe9dcdae299/src/libopensc/card-piv.c

The offending lines are around line 5145. According to anthropic, this is a buffer overflow found by their predictive text algorithm.

But it's interesting to read the code and to read the context of the code. 'fp' is a string guaranteed to be 64 bytes in length, because of line 5138. This checks the strlen(fp) and if it isn't 64, the code exits with an error.

So the math is: 'filename' is filled with a string that is at maximum 4096-64-2 bytes long == max 4030 bytes. This value (maximum 4030) is eventually passed to a strlcat() call inside of sc_get_cache_dir(), where it is used as the 'size' argument. This means that 'filename' has a maximum size of 4030 bytes when sc_get_cache_dir() exits.

Then the code strcat's one byte (for the '/'), onto the 'filename' buffer, then 64 more bytes (for the 'fp' string) onto 'filename' buffer, meaning the buffer cannot be overflowed because the maximum string length is now 4095 (the last byte being a null terminator).

So, anthropic's LLM probably spent 4 seconds 'finding' this 'vulnerability'. I spent about an hour looking at the code to debunk their finding (really 20 minutes looking at the code, followed by 40 minutes of hemming and hawing about whether I'm right and making sure there wasn't a signed size or some other gotcha, then being anxious about saying anything because golly human brains are funny machines).

I guess it's cool that opensc changed the calls to strlcat() now and that is probably a good thing, but this was never even a bug anyway so it didn't help anything today.

Unless the compiler magically omits instructions like the strlen() check some day. Which, if we really go down the path of compilers being generated by LLMs I guess that is a valid concern 😉 .

I have a blog now! Like an old school long form one!

Starting with something small, a little teardown of a cheap Color E-Ink Android tablet: https://timonsku.zip/bigme-b6-e-ink-tablet-teardown/

Been meaning to set one up forever and then never got around to it until now.
Its a self hosted Ghost and publishes to ActivityPub too, you can follow the blog feed here:
@timons_blog