Many times we are encouraged to write unit-tests, and decouple our UI from logic.
Do this, without a doubt, do it.
UI Testing should always be considered a second or third line of defence, to ensure that high level end-to-end integration is all a-ok.
From my own experience, "feature specs/tests" / "UI specs/tests" (i.e. UI tests that are single use case) are a significant waste. We have to spin up the application into a running state for each feature. This consumes an increasing amount of time and becomes unhelpful quite fast. Not only that if we are heavily relying on these tests for confidence, we increasing find ourselves strugging with async issues. Flaky 50/50 tests begin to emerge an poison team confidence in the test suite.
Of course, there's options to parallelise this kind of test, but they really should not be necessary. (that is to say there should be less of them!)
Replacing these with what people variously call collaborator tests, sociable tests, light integration tests (etc.) is preferred, with UI layers excluded, and downstream services faked, should suffice. With logic-less binding to UI as the only layer between the user and the app which remains to be tested by a UI E2E test.
Exercising the downstream service and app integration with request / contract tests should fill in the rest of the picture.
What's left is a User Journey. Do application user flows work without incident?
Add to these Journey tests when there are bug reports to cover the bug reproduction. When unit or other level integration tests cover these cases, remove the additional bug journey.
To avoid repeated app spin up times, we run the major headline phases of a user flow through the app as a Journey test.
These should be written in advance if possible, but are also something that can be recorded post-fact. (especially in the context of iOS and MacOS). We can also combine them with Snapshot testing (see snapshot testing)
There's a few options for iOS (Kif, EarlGrey and XCUITest) but macOS has only XCUITest. iOS has also got a fair amount of coverage on the web, so I'm going to spend some time on macOS UI testing.
macOS has a number of things which make it an entirely different challenge when UI testing apps.
I'm going to talk about how I tested CutBox, since I've just started UI tests on CutBox.
It's also an atypical macOS app since it's not document driven, it's a utility which is powered by global shortcuts and integration with the pasteboard.
First things first, we're going to need to do a few things before we
launch the app, that we can't do directly in the XCUITest setUp()
method.
Doing this requires kicking off the app in a test mode, so that it can
self configure the setUp phase.
In the AppDelegate we'll setup a DEBUG only test path for setup.
func applicationDidFinishLaunching(_ aNotification: Notification) {
// ... actual application setup code
// ... (should (usually) be run before test setup)
#if DEBUG
// From DEBUG mode we can ensure release versions will never execute this test setup.
NSLog("DEBUG MODE") // just to be sure, you shouldn't see this in you releases!
// We pass -ui-testing to the app from the test.
if ProcessInfo().arguments.contains("ui-testing") {
configureTestingState() // --- our setup code
}
#endif
}CutBox revolves around pasteboard history, and searching it, so I'll need a fake paste history and I'll need to ensure user settings are in their default state.
func configureTestingState() {
NSLog("configure testing")
// Clear the history
HistoryService.shared.clear()
// Set initial prefs/settings required for the user test.
HistoryService.shared.searchMode = .fuzzyMatch
HistoryService.shared.favoritesOnly = false
// Add fake history item(s)
HistoryService.shared.historyRepo.insert("App test", at: 0, isFavorite: true)
// Open search popup so we don't have to waste time faking a global keyboard event.
HotKeyService.shared.search(self)
}