Chris PittsHere's a challenge for tech coaches out there. I am looking for a kata that is not already out there and analysed to death by an accursed AI.
I want to feed it to one of these stochastic parrots to observe its true nature.
Daniel Terhorst-North@thirstybear you might ask @emilybache if she has any unpublished ones, or @jbrains. Most of them are in the wild, unfortunately.
@tastapod @emilybache @jbrains
You can see the problem though. Once something has been solved and published these parrots will simply puke a most average version back up, pretending they thought of it.
Jeff Grigg@thirstybear @tastapod @emilybache @jbrains
Something that I liked from a coding interview, from quite a long time ago:
Implement a "Magic 8 Ball" program that accepts "questions" (and ignores them) and provides random "answers," much like a "Magic 8 Ball"
EXCEPT THAT ...
1. Configure with an "input file" of some sort with an integer "weight" to each answer. Probability of getting an answer is it's "weight" divided by the total. (except see #2 below)
2. Never give the same answer twice.
@thirstybear @tastapod @emilybache @jbrains
... You might want to ask for an "efficient" or "always quickly responsive" implementation too.
A "gotcha!" is that if one answer is *orders of magnitude* more likely than all the others, then a naive implementation can make and discard "bad" (duplicate) answers *for a long time* before finding a "valid" (IE: unique, new) answer.
@thirstybear @tastapod @emilybache @jbrains
If that specific task does not meet one's needs, consider the pattern of it:
Take a well known problem, and then add some complications to it that invalidate the common solutions.
And while you're at it, add some irrelevant text, easily ignored by humans, but which would send the AI LLMs "off track."
@JeffGrigg @thirstybear @emilybache @jbrains Back in the day, that was what we would do with TW code submissions. They would by and large all end up in one of several 'styles'. The fun—and telling—part was to then ask the candidate to do something that would collide with their solution in some way.
J. B. Rainsberger (he)@JeffGrigg @thirstybear @tastapod @emilybache Hilariously, @sarah11918 and I built a variation of this on a train between Vienna and Munich several years ago. :)
@thirstybear @tastapod @emilybache We could devise one that is rich and surprisingly complicated in certain, dark corners, but still accessible. Ping me directly if you'd like to try. (As in, I have one in mind, but I haven't written it all down yet.)
@jbrains @thirstybear @tastapod @emilybache
I think the best approach is to start with a simple problem. Possibly even a well known published problem. And then add a few "twists and turns" so that the published solutions no longer work. Best if the published solutions would be *hard to change* to meet the additional new requirements.
@jbrains @thirstybear @tastapod @emilybache
One can even remove constraints, simplifying the problem, before adding new and different constraints.
Say that you have a boat, a fox, a chicken, and some grain, most LLMs will jump into a complex solution, even without any constraints forcing such. Unless stated otherwise, one could just put everything in the boat and cross once.
@JeffGrigg @jbrains @thirstybear @emilybache 'There is a boat, a fox, a bag of corn and a chicken. The boat can only hold two of the items. How many tennis balls can I fit in the boat?'
@jbrains @tastapod @emilybache Just coming back to this. Turns out Kiro shows quite enough problems with plain old FizzBuzz unless closely supervised (maybe it's the way I introduce the edge cases?). But I'd love to develop an ‘evil kata' that is designed to really stretch folks' practices.(much like Mr Gorman has Evil FizzBuzz, which is...evil)
@thirstybear @jbrains @emilybache I think the part 1 / part 2 of any of the early days of Advent of Code would be great examples of that. Of course there are lots of example solutions in the wild though. I love how e.g. for one I had a simple game-of-life solution that blew up in part 2 and I had to reimplement it as a sparse matrix.
@tastapod @jbrains @emilybache Yep. For me that's part of the joy of writing code - getting caught out by evolving requirements. Those wry “Well, that didn’t go to plan" moments 🙂
Good point with the AoC stuff. More grist for the kata mill.
@thirstybear @jbrains @tastapod @emilybache
Jason Gorman's 'Evil FizzBuzz (or "So you think you're a team?)'
https://www.linkedin.com/pulse/evil-fizzbuzz-so-you-think-youre-team-jason-gorman/
Evil FizzBuzz (or "So you think you're a team?)
Among the advanced topics we cover on the third day of the Codemanship TDD course, Continuous Integration presents some of the biggest challenges for dev teams. Participants are subjected to an exercise called Evil FizzBuzz, which you might like to try on your team (or at your coding event).
@thirstybear @jbrains @tastapod @emilybache
If I were subjected to Jason Gorman's "Evil FizzBuz" (link above), I think I would propose to the six teams that we work sequentially, on a single code base, on a single machine:
The fist pair/team sits down, does TDD the code to produce a List of the positive integers 1 to 100.
They step aside and the second team replaces every 3rd item with "Fizz". Change the List type from Integer to String, as needed.
…
@thirstybear @jbrains @tastapod @emilybache
3rd group: 5th to "Buzz"
4th group: 15th to "FizzBuzz"
5th group: Replace or concatenate "Wizz"
6th group: Output comma-delimited string.
@thirstybear @jbrains @tastapod @emilybache
By separating step 1 from step 6, using an in-memory List, one never even has to do the tricky "double check nested if statements" that "throw some people for a loop" on this problem.
But I guess the 3rd group (every 5th item to "Buzz") might have a testing problem. (But, OK, 15 is "Buzz". It's the 4th group's problem to make 15 be "FizzBuzz".)
@JeffGrigg @thirstybear @jbrains @emilybache Don't you then run out of budget after group 4 and just release what you've got? The rest of it is just managing the PR fallout. Gotta make these exercises lifelike.
@tastapod @thirstybear @jbrains @emilybache
I know: I'll have the 6th group go second.
Then we'll have usable releasable code from that point on.
The rules say that each group can only do their assigned part. I do not see any rule that they have to work simultaneously or in parallel or on different machines.
Should be very efficient to just build it all on one machine and release when done (or when time runs out).
Probably *a lot* faster and easier than source control, CI, etc.
@tastapod @thirstybear @jbrains @emilybache
That's nearly 10 minutes each to do each of the changes.
Is it going to take 10 minutes to make a List with the numbers 1 to 100?
More or less than 10 minutes to replace every 3rd cell with "Fizz"?
Do the same with "Buzz" on every 5th cell?
15th, same?
Small simple prime number check? It's last, and does NOT need to be efficient. Mark 2. Then mark 3 and every odd number that's not divisible by odd numbers from 3, counting up, less than it.
@tastapod @thirstybear @jbrains @emilybache
Heck; optionally refactor prime number performance improvements in after doing something even more "brain dead stupid" (but sufficient).
Ron Jeffries@JeffGrigg @tastapod @thirstybear @jbrains @emilybache
Evil, yes. Does not sound like fun. Teaches, mostly, don't agree to do dumb stuff?
Evil, yes. Does not sound like fun. Teaches, mostly, don't agree to do dumb stuff?
Dragan Stepanović@thirstybear a couple of months back I asked chatgpt to generate one (it was simple enough and not too simple) and then asked claude model at the time (opus 4.5) to solve it with all the skills XP and design skills I provided and it was terrible.
Curious to hear how it goes for you.
Curious to hear how it goes for you.