I just did my first Capture the Flag with the help of OpenAI's GPT-4 and it kind of blew my mind. Here's a detailed writeup of three challenges, including one that GPT-4 just straight up solved for me https://micahflee.com/2023/04/capturing-the-flag-with-gpt-4/
Capturing the Flag with GPT-4

@micahflee i wonder if we'll ever be able to download a model scaffolding from openai that would let us take a 'skeleton model' and then add stuff to it to make it our own, so we could run it locally, not over the internet.
it'd be .. jarvis, basically.

@micahflee Wow yeah, GPT-4 is going to be such a great tool for many purposes.

At my company, though, we aren't allowed to use it for any real code yet because the legal aspects haven't been settled yet.

@micahflee That was absolutely fascinating! I was sad when I reached the end. Thanks for sharing. (Also TIL a lot!)
@micahflee how many points were put up on that scoreboard by LLMs this weekend? πŸ‘€
@micahflee That was a fun read. Thanks!