During development I encountered https://github.com/flutter/flutter/issues/41029 and didn't liked the solution with hardcoded waiting times. I'm now waiting for the first frame to be rastered successfully and ignoring errors in the meantime. This works well for my tests and avoids hardcoded timings, which often fail (e.g. on a different machine).

flutter_driver_async_main.dart

Future<FlutterDriver> setupAndGetDriver() async {
  FlutterDriver driver = await FlutterDriver.connect();
  var connected = false;
  while (!connected) {
    try {
      await driver.waitUntilFirstFrameRasterized();
      connected = true;
    } catch (error) {}
  }
  return driver;
}