Test authors may focus on new changes or additions, potentially overlooking regressions in existing features that should remain stable. One way to help retain functionality or perhaps catch some unexpected integration issues is to "roll out to tests". This feels obvious—of course, you are going to enable your feature in the unit tests you are writing to validate the behavior. But, the further up the testing pyramid you ascend, rolling your feature out to thousands of UI tests may entice you to only enable your feature in a handful.

Resist this inclination and enable your feature in all your UI tests.

As a mobile developer, this might mean extra work in a dreadful part of the mobile delivery cycle. UI tests are notoriously slow, flaky, and nobody's favorite. However, the upside is that you reduce feature flag drift between your UI tests, your internal users, and production.

Feature flag drift between your UI tests and production creates reliability silos where only the tests that have the most flags correctly enabled can be trusted.

Test gaps occur, and when a developer tries to close a UI test gap, only to discover that their suite is missing two or three flags that require significant updates, an admirable task becomes even more challenging. And one should always strive to make difficult but valuable tasks easier to accomplish.

When the time comes to perform the feature flag cleanup, deploying the flag to UI tests minimizes the likelihood of unexpected issues that would be more difficult to address months after the feature has been developed.

Caveats

This practice rarely invites criticism, but in practice, you might have to adjust the strategy.

Long Running Projects

A long-running project may require a feature flag or a group of feature flags to be managed over a long time. As a result, enabling your feature flag for all UI tests may not actually be the correct strategy because you risk developers writing against behaviors that will not be live in production for months. To remediate, consider parameterizing a strategic set of UI tests with your feature enabled and disabled. If you have the resources and the project is critical, then consider a duplicate CI pipeline with your feature flags enabled.

Experimentation

This policy primarily applies to projects and features intended to reach production. If you are experimenting, then requirements are still formulating, and therefore, you may not need to lock in the behavior to your UI tests.

Closing

UI tests are an easy target for criticism, and with good reason. However, maintaining feature flag parity between UI tests, your internal users, and production helps sustain trust and confidence in your app's reliability.