Hi there! I'm Emily Hayes, a QA specialist at imaga. We're heavily involved in developing and testing numerous Flutter applications. Lately, I've noticed that not all my colleagues fully grasp the nuances of testing them. In this article, I'll delve into the primary tools and approaches we employ, focusing solely on those unique to Flutter.
Flutter fundamentals you need to know
Flutter, developed by Google, is a versatile tool for building cross-platform applications. With this framework, developers can utilize a unified codebase for apps across Android, iOS, Linux, macOS, Windows, and the Web. Over 700,000 applications built on Flutter are available on Google Play.
The language used in Flutter applications is Dart. The technology itself stands out with a unique architecture and design approach, where every element is a widget. This provides high flexibility and performance but also presents specific requirements for testing.
Characteristics of Testing Flutter Applications
Let me summarize what our team has observed while working on Flutter projects. I've grouped them into three main categories.
- Flutter Architecture: All elements are widgets. This simplifies testing of both interface and functionality, ensuring consistency throughout the application.
- Cross-Platform Capability: Significant time savings can be achieved by testing the application once for multiple platforms. Platform-specific functionalities are tested separately on iOS and Android, while everything else is tested on one platform.
- Specific Testing Tools: Flutter provides its own tools for testing widgets and integration, such as FlutterDevTools.
Testing Challenges and Solutions in Flutter Applications
The most common issue is tracking network traffic. Dart typically utilizes high-level libraries for HTTP requests. These libraries encapsulate low-level details of network interaction, making it difficult to directly access or monitor the data — both sent and received.
The problem is resolved by connecting a proxy server. It's best to add this functionality to your engineering panel.
When testing UI/UX variances across platforms, we often encounter these issues:
- Animation glitches: Pay close attention to transitions and scrolling, particularly on iOS devices.
- Gesture functionality for screen closures and swiping.
- Proper display of system dialog windows.
Don't sweat the small stuff with every button. Instead, prioritize significant differences. Plus, engaging with the community is key. GitHub hosts numerous open issues you can explore and contribute to.
Additionally, leveraging Flutter DevTools can help tackle any challenges you encounter.
Flutter DevTools
This debugging and profiling toolkit within the Flutter ecosystem enables us to deeply analyze and test applications. We can use it to inspect the widget hierarchy, helping us identify UI issues and monitor the performance and memory usage of the application.
We particularly value its ability to track network requests and logging, making it a powerful tool in our everyday testing and debugging practices.
That’s how the interface looks like in Flutter DevTools
Integrating golden tests into the development process.
We also actively integrated golden tests into our development processes. The testing team participated in developing scenarios for these tests.
- Golden tests are special test scenarios that record the expected outcomes of application execution.
Golden tests work by comparing the current widgets appearance with a predefined "golden" standard—previously saved images depicting the expected appearance. As developers modify the code, golden tests automatically compare the new images with the established standards to identify any deviations.
Implementation Example:
Automation Implementation
Selecting the right framework for automating testing of Flutter applications is crucial during the development process. This choice significantly influences testing efficiency, ease of operation, and ultimately, product quality.
Consider the following:
- Flutter Compatibility: Our tests should seamlessly integrate with Flutter, so go for tools that are Dart-friendly, such as Selenium, Patrol, integration_test (formerly Flutter Driver), and flutter_gherkin libraries.
- Cross-Platform Functionality: Flutter knows no bounds, so ensure your framework works smoothly on both Android and iOS.
- Compatibility with CI/CD: For speed and convenience, ensure the framework aligns well with Jenkins, GitLab, and other CI/CD tools.
- Industry Popularity: Choose a framework with a robust community presence.
We chose Appium — an open-source tool for automating mobile app testing that's versatile across platforms, just like Flutter. With Appium, tests can be scripted in different programming languages, and it seamlessly integrates with continuous integration and delivery (CI/CD) systems.
The problem we faced was that nearly everything in the app is a widget, making it difficult to extract text from certain modal windows. To resolve this issue, reach out to your development team. They can add identifiers to the elements.
Ultimately, we greatly slashed the time spent on regression testing by using Appium. Previously, our team's full regression took up to 90 hours, but after automation, it dropped to 33 hours. On average, each test now runs in about 1.5 minutes. Approximately 10% of defects were caught through automated testing.
Parameterizing and optimizing test scenarios allowed us to cut down the number of test cases from 1334 to 483 — more than halving our initial count! Parameterization made our tests more flexible, adapting them to different conditions. This increased our efficiency in resource allocation, letting us to dedicate more attention to new features and enhancements.
An end-to-end test across 6 pages takes only about a minute and a half to complete.
Conclusion
Testing Flutter applications requires a specific approach that takes into account the framework’s unique characteristics.
Early testing with golden tests, process optimization, utilizing parameterized automation with Appium, and integrating with CI/CD all contribute to enhancing the efficiency and accuracy of your product testing. We've seen this firsthand through our own experience.