Mastodawn

Another example of AI failure.

I will not use generative AI. Full stop.

However, I have collaborators who will for minor purposes.

I was notes with one about a project and I outlined that one element was essentially a simple combinatorics problem (n choose r) and listed the very large number of possible outcomes in a table.

Show thread

Michael Rosenberg Mar 18

Unfortunately, I mangled my explanation slightly. My collaborator dumped the problem into genAI and got a formula and numbers out that did not match those in the table. They tested the formula for small values they could prove by hand (literally writing out every possible combination and counting them), so they knew the formula was correct.

Show thread

Michael Rosenberg Mar 18

When we discussed it, we confirmed the formula was right, but whose numbers were correct, mine or theirs? (Mine obviously). I had calculated the values in Python, but then showed that I got the same values out if I plugged them into a website combinatorics calculator.

Show thread

Michael Rosenberg

I challenged my collaborator to try a different genAI: they got the exact same formula and a completely different wrong answer. Turns out the genAI cannot handle the large integer math, but rather than admit it cannot, it just reports some answer that is orders of magnitude incorrect.