Sorry, can't do scarves.

https://sopuli.xyz/post/41832893

I’m sad that the relevant xkcd is kinda obsolete now (because it’s been long enough for that research team to finish going its thing).
Tasks

xkcd
What would be a “nearly impossible” task in this post-AI world? Short of the provably impossible tasks like the busy beaver problem (and even then, you would be able to make an algorithm that covers a subset of the problem space), I really can’t think of anything.
Deterministic answers from AI

Most AI are deterministic, it’s only a small subset of AI that are non-deterministic, and in those cases it’s often by design. Also, in many cases, the AI itself is deterministic, but we choose to use the output in a non-deterministic way, e.g. the AI gives a probability output, and will always give the same probabiliies for the same input, and instead of always choosing the one with highest probability, we choose based on the probability weight, leading to a non-deterministic output.

Tl;Dr. Non-determinism in AI is often not an inherit property of the model, but a choice in how we use the model.

Okay, probably fair. I’ve only been working with LLMs that are extremely non-deterministic in their answers. You can ask same question 17 times and the answers have some variance.

You can request an LLM to create an OpenTofu scripts for deploying infrastructure based on same architectural documents 17 times, and you’ll get 17 different answers. Even if some, most or all of them still manage to get the core principals right, and follow the industry best practices in cases that were not specified, you still have large differences in the actual code generated.