We just managed to do some black box E2E mobile testing with a #ChatGPT GPT 🤯
Had the pleasure of working with @almouro from @bam_lab on the crazy idea of getting ChatGPT to test an app for us
Took us an afternoon, and guess what ? It worked ✨
More in the 🧵
How black box E2E testing through a GPT works:
🔸 The device is plugged in
🔸 We built an API for GPTs using my template https://github.com/pAIrprogio/nestjs-gpt-action-api-template
🔸 We use a local tunnel to our machine
🔸 Instructions are sent through adb command lines
Why did we use the ChatGPT interface ?
🔸 Already has a chat pipeline in place
🔸 It provides chain-of-thought capabilities
🔸 It can access external system through GPT Actions
🔸 We already pay for it
Why reinvent the wheel ?
In order to get our E2E black box testing workflow to work, we built 4 basic actions for the GPT
The API executes the action & replies with a dump of the current UI hierarchy as XML 🌳
Allowing #ChatGPT to iterate by itself
Is AI driven automated E2E testing production ready ?
Unfortunately, not really 😢
We had to go through multiple takes and iterations on our prompts just to achieve our demo
But having spent only an afternoon on it, we're confident we can do more 🚀
AI driven black box E2E testing open's up a whole range of possibilities,
🔸 Using it to come up with replicable tests for @almouro's Flashlight automatically
🔸 Mapping an app's navigation tree
🔸 Creating E2E tests by itself from user stories
Which we will be exploring
And that's it for now
If you find these projects useful, please help us by sharing the first post 🙏
And make sure to follow @almouro and @yleflour if you don't want to miss our upcoming tests, discoveries, and releases 😄
Yann Leflour