There's a lot of discourse on Twitter about people using LLMs to solve CTF challenges. I used to write CTF challenges in a past life, so I threw a couple of my hardest ones at it.

We're screwed.

At least with text-file style challenges ("source code provided" etc), Claude Opus solves them quickly. For the "simpler" of the two, it just very quickly ran through the steps to solve it. For the more "ridiculous" challenge, it took a long while, and in fact as I type this it's still burning tokens "verifying" the flag even though it very obviously found the flag and it knows it (it's leetspeak and it identified that and that it's plausible). LLMs are, indeed, still completely unintelligent, because no human would waste time verifying a flag and second-guessing itself when it very obviously is correct. (Also you could just run it...)

But that doesn't matter, because it found it.

The thing is, CTF challenges aren't about inventing the next great invention or having a rare spark of genius. CTF challenges are about learning things by doing. You're supposed to enjoy the process. The whole point of a well-designed CTF challenge is that anyone, given enough time and effort and self-improvement and learning, can solve it. The goal isn't actually to get the flag, otherwise you'd just ask another team for the flag (which is against the rules of course). The goal is to get the flag by yourself. If you ask an LLM to get the flag for you, you aren't doing that.

(Continued)

A thing I see happening in good forums:

🦝 Hey fellow doofer-enjoyers, I have a 2006-model purple doofer. I'm trying to get it to interface with the green 2009 doohickey but I'm not having any luck so far, has anybody else tried this?
🦁 Oh I think I remember 🐭 was trying that a few years back, hey 🐭 did you get anywhere with that
🐭 Kinda, here's the thread [link]
🐿️ That really should've gone in the wiki
🐭 Yeah well I never got it all the way done though. 🦝, you wanna turn the green one at 90 degrees to the purple one and click your heels twice, that's as far as I got before I gave up on it
🐏 Oh this is like with the turquoise whadjamacallit, lemme find that thread...
[4 pages of discussion and testing and throwing out ideas and figuring things out later]
🦝 Okay I tried standing on my right foot and heck, it works! Thanks guys, that was really helpful!
🐿️ Awesome, I put a very quick rough write-up and a link to this thread in the wiki, for future searchers. 🦝, if you could check out that page and make sure it's accurate, that'd be great
🦓 Great write-up 🐿️, I linked to it from both the 2006-purple and the 2009-green pages. If anybody else wants to tidy up the formatting etc that'd be super useful

The forum is for figuring out, and the wiki is for showing what we've figured out.

And y'know, maybe 🦝 and 🐭 ended up in the chat, while they were right in the flow of test-things-quick, and realtime chat is good for that, but it's bad at showing the results. The results just float off up the page with time, replaced by new chat about new stuff, even if the old stuff is still true and valid and useful. The longer the chat runs, the more time you have to spend scrolling up to find the thing. The same goes for the forums; it happens at a slower pace, but everything still floats off down the time stream, the best chunks of proven-truth floating just like the mistakes and wrong-turns that happened during the proving.

Worse, the most posted-in threads tend to be the figuring-out threads. Once you've figured it out, there's not as much need to post and bump the thread back up to the top, so the threads most likely to float off are the ones that have got a bit of figured-out-and-proven truth in them.

But in a good forum with a wiki attached, there are lots of 🐿️ who fish little bits of figured-out from the figuring-out-stream and save them somewhere else, where they won't float off.

Without having some kind of wiki or static page or other sort of recency-independent repository of information, what folk end up doing is they generate ideas, they test, they make mistakes and correct them, they ask questions, they boil and condense and distill their thinking-out-loud streams-of-consciousness into a source of Actual Properly-Figured-Out Truth, and then set that perfect refined information on a little paper boat and wave to it as it floats off down the time stream, out of relevance, further from retrievability.

Forums and chat and social media are great for discussion and pure dogshit at storing information, and wikis and webpages are the opposite, but you've gotta use both. You have to use both. At some point your truths have to be written down somewhere that doesn't display most-recent-first. That's the only way to get off the treadmill of constantly figuring out the same things over and over and start living in a place where things are, and have been, Properly Figured Out.

I've been thinking about this today because I saw a person on Fedi who had, as a pinned post, an index of all the really good posts she'd made. Do I have one of those posts? Do I bollocks, what I have instead is a list of bookmarks to refer to when someone asks "Hey I was thinking about your bike thread the other day, do you remember the URL" and you know what I do? I look at the top of my bookmarks and it's not there, and I go through like forty open tabs until I find it, and I untick the bookmark and tick it again so that it's now most-recent, at the top of the bookmarks list. That's no fucking way to live. That's not a library, that's a cursed backpack that keeps getting heavier

it saw another windows aero theme for kde. that's cute but it thinks gnome also deserves some love too

anyways here's a gnome desktop composited by the real aero dwm.exe from win7

a little funny to see that there’s a microsoft guide on how to install linux

https://learn.microsoft.com/en-us/linux/install

postmarketOS is the work of thousands of people from all walks of life and with all sorts of backgrounds, including our trans users and developers. Previous efforts of transgender computer scientists, including Lynn Conway, the inventor of generalized dynamic instruction handling, Sophie Wilson, who invented the ARM Instruction Set that powers most of our devices, or maybe even you reading this, made us possible.

We believe that human rights are universal and that trans people are just as deserving of compassion, love, and justice as everyone else.

If you haven't already, take some time this week to give your trans friends a hug and maybe a cookie.

~The postmarketOS team 🏳️‍⚧️

#TransgenderAwarenessWeek