Mastodawn

Free #startup #idea: A #GitHub bot which daily/weekly runs all the tests in your repo, looks for flakey targets, and then creates #AI generated fixes to improve stability.

#Nrwl #Nx self-healing CI is the closest existing example I can think of, but maybe there's something else out there I'm not aware of.

Show thread

Shini92

@develwithoutacause We are not there yet. Just yesterday Claude just throwed a functionality away, just because something is running differently on GitHub CI as on local. I needed to tell it that it cannot just rewrite existing tests or throw away tests, and then rewrite the code. 😅
Very glad that I still need to guide it.

Show thread

Doug Parker 🕸️5d ago

@Shini92 I'm just proposing the AI create PRs to fix flakey tests, not that it necessarily needs to auto-merge them. We can still enforce as much or as little human review as necessary.

Show thread

Doug Parker 🕸️5d ago

@Shini92 The painful part of flakey tests is the debugging.

If the AI does that debugging and then gives me a working PR with a theory of why it was flakey, I can pretty easily validate that without needing to go through the effort of debugging the test myself.