We just managed to do some black box E2E mobile testing with a #ChatGPT GPT ๐Ÿคฏ

Had the pleasure of working with @almouro from @bam_lab on the crazy idea of getting ChatGPT to test an app for us

Took us an afternoon, and guess what ? It worked โœจ

More in the ๐Ÿงต

How black box E2E testing through a GPT works:

๐Ÿ”ธ The device is plugged in
๐Ÿ”ธ We built an API for GPTs using my template https://github.com/pAIrprogio/nestjs-gpt-action-api-template
๐Ÿ”ธ We use a local tunnel to our machine
๐Ÿ”ธ Instructions are sent through adb command lines

GitHub - pAIrprogio/nestjs-gpt-action-api-template: Boilerplate project to quickly launch a NestJS powered API for ChatGPT GPT actions

Boilerplate project to quickly launch a NestJS powered API for ChatGPT GPT actions - pAIrprogio/nestjs-gpt-action-api-template

GitHub

Why did we use the ChatGPT interface ?

๐Ÿ”ธ Already has a chat pipeline in place
๐Ÿ”ธ It provides chain-of-thought capabilities
๐Ÿ”ธ It can access external system through GPT Actions
๐Ÿ”ธ We already pay for it

Why reinvent the wheel ?

In order to get our E2E black box testing workflow to work, we built 4 basic actions for the GPT

The API executes the action & replies with a dump of the current UI hierarchy as XML ๐ŸŒณ

Allowing #ChatGPT to iterate by itself

Is AI driven automated E2E testing production ready ?

Unfortunately, not really ๐Ÿ˜ข

We had to go through multiple takes and iterations on our prompts just to achieve our demo

But having spent only an afternoon on it, we're confident we can do more ๐Ÿš€

AI driven black box E2E testing open's up a whole range of possibilities,

๐Ÿ”ธ Using it to come up with replicable tests for @almouro's Flashlight automatically
๐Ÿ”ธ Mapping an app's navigation tree
๐Ÿ”ธ Creating E2E tests by itself from user stories

Which we will be exploring

And that's it for now

If you find these projects useful, please help us by sharing the first post ๐Ÿ™

And make sure to follow @almouro and @yleflour if you don't want to miss our upcoming tests, discoveries, and releases ๐Ÿ˜„