E2E testing tools – Patrol vs. Maestro vs. Appium (what do you use?)

Help Testing & Debugging
, , ,
1

We’re evaluating E2E testing tools for a larger Flutter app.

Right now we use Patrol, but over the past ~6 months we’ve run into various issues that often made the tests simply not work at all — both locally and on CI (Codemagic / Firebase Test Lab). Because of that, we haven’t really been able to rely on them :sweat_smile:.

I’d love to hear what tools you use (Patrol, Maestro, Appium, or others) and your pros/cons regarding:

  • Ease of writing tests
  • Ease of usage & maintenance
  • Running locally
  • Running in CI (Codemagic / Firebase Test Lab)
  • How big your app is / scale of tests

Any war stories or recommendations are very welcome! :folded_hands:

1 Like
2

I was very impressed with Maestro which I helped implement on the last large commercial app I worked on last year. The whole team who used it were very impressed with it, but even though it was quite a large app, we only got the first major user journey done before I left that project team.

From memory I think we got some initial work done for running it on CI but I can’t remember exactly.

Overall I was impressed with how easy it was along with how it’s developers embraced the inherent flakyness of e2e mobile testing and added functionality to help cope with it.

1 Like
3

Went through this process recently, we landed on Flutter integration tests for the time being, and we rely on manual testing (which was already part of our testing strategy) for native interactions, using build time variables to skip or mock interactions where required to facilitate an integration test. Our project is quite large and still growing

1 Like
4

I’m using patrol finders with flutter integration test package. I tried configure maestro, but I could not get it working in my machine.

1 Like