Skip to content

Instantly share code, notes, and snippets.

@alexglow
Last active January 31, 2018 22:57
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alexglow/7033761 to your computer and use it in GitHub Desktop.
Save alexglow/7033761 to your computer and use it in GitHub Desktop.
This is a list of errors you may encounter when testing in Sauce Labs, as well as the usual causes and how to fix them. The list is under construction and will soon be made available on the Sauce Labs website, but it's here for now so that you can find it when Googling!
Search for the error message you're seeing. Then read what we have to say about it. Then fix it! If you still have problems, write in to help@saucelabs.com .
==== AUTOMATED JOB ERRORS: ====
-- Invalid credentials --
Some combination of the following error messages will be thrown:
OpenQA.Selenium.WebDriverException : Unexpected error. Unknown username.
You sent username 'None' in your browser string, which is not a valid Sauce Labs account.
OpenQA.Selenium.WebDriverException : Unexpected error. Invalid Credentials.
org.openqa.selenium.UnsupportedCommandException: Invalid Credentials.
You sent the access key 'None' but it does not match the access key associated with your account. Please login to saucelabs.com to retrieve a valid access key.
This indicates that Selenium could not parse the credentials you sent. If you're using "access-key" in your test setup, try replacing it with "accessKey", or vice versa, as it's language-dependent.
-- User Terminated --
This means that you ended the job, either through the "Cancel Job" button, by jumping into an automated session from the Sauce website, or via a breakpoint (http://sauceio.com/index.php/2012/09/using-sauce-breakpoints-to-find-and-fix-flakey-tests/). Since this terminates the job immediately and hands over control to you, the test will not upload any screenshots, video, or logs that were collected previously.
-- Timeout errors --
1. "Test did not see a new command for 90 seconds. Timing out."
2. "Selenium took too long to run your command"
3. "Test exceeded maximum duration after 1800 seconds"
Sauce builds in a few default timeouts to keep uncontrolled tests from eating up your minutes (https://saucelabs.com/docs/additional-config#timeouts). Here's a brief rundown on what each of the available timeout settings does:
1. The "idle-timeout" (default 90 seconds) ends the job if no commands are received from your Selenium script.
2. The "command-timeout" (default 5 minutes) ends the job if a single command takes too long to run.
3. And the "max-duration" timeout (default 30 minutes) ends the job if your entire test takes longer than you want.
You can set these to different values using "desired capabilities" settings in the setup of your test, as described at the URL above.
-- Unsupported OS/browser/version combo --
Check to make sure that the browser, browser-version, and platform settings you're using are in our supported list: https://saucelabs.com/docs/platforms
And note that there's a separate list for Appium tests (as they use a different mechanic to drive the tests): https://saucelabs.com/docs/platforms/appium
-- Browser failed to start --
The twin sibling of "Unsupported version", this message means that something a little more unusual is off in your test setup. Usually, this means that you're specifying a Selenium version that isn't compatible with the browser/version/OS you've selected. (For example, you should not be setting this for any mobile tests.)
Try simply omitting that setting, and if you still see the issue, feel free to write to help@saucelabs.com with a description of the issue and a copy of your setup code.
-- Client disconnected during getNewBrowserSession request --
This means that your test runner decided to end the job before it had fully initialized on Sauce's end. There are a few potential causes:
1. You're running too many tests at a time: Check the left sidebar on your Account page (https://saucelabs.com/account). It shows a "Parallel tests" number, which is the maximum number of tests you can run at a time, based on your subscription level. If your account can run 2 parallel tests, and you're launching 10, 8 will be "queued" until one of your tests finishes and a slot frees up. However, if this takes a long time, your test runner may choose to end the queued jobs after a few minutes instead of waiting. Just make sure you're launching a reasonable number of simultaneous tests for your account.
2. High job wait times: Check our status page (http://status.saucelabs.com/) and/or follow @sauceops on Twitter for up-to-the-minute news about any issues within the service. If something causes demand for certain VMs to stack up, your jobs may be queued and (as above) terminated by your test runner.
Tests that end this way are never taken out of your minutes.
-- Runaway execution. Please contact help@saucelabs.com for assistance. --
This message means that an error has been detected in the VM or in Selenium, which caused the test to behave abnormally, and Sauce detected this and shut down the job. These are very rare and usually do not recur. If you do see more than one on the same test, let us know.
==== MANUAL JOB ERRORS: ====
-- Job does not load --
There are two common scenarios here:
• Error message: "Uh oh! Some error occurred while connecting to the browser"
• The job seems to start, but you see only a white text box in the middle of a black screen.
Both of these indicate that your browser is having trouble displaying the VNC stream from the remote machine. Take the following steps to troubleshoot:
• Check the video on Sauce: If the recorded video after the job shows a steady video stream, this indicates that the issue is in your computer or connection to Sauce. However, if the Sauce video shows the same issue, that indicates an issue in our service. In that case, send us the URL for the job page and a screenshot of the issue.
• Check that your browser is up to date: If you're on an older version, this may cause incompatibilities. Update your browser and try again.
• Check your firewall: make sure that your machine allows full access for the interactive stream over the required ports (https://saucelabs.com/docs/manual#troubleshooting).
• Check that the Internet connection is stable: We recommend running Sauce tests from a machine with a wired Ethernet connection, to ensure a steady connection. If the connection flickers, this error could be thrown.
==== FRAMEWORK-SPECIFIC ERRORS: ====
-- Appium: Failed to download mobile app --
First, if you're having trouble, try using Sauce's temporary storage (https://saucelabs.com/docs/rest#storage). Uploading your app to our servers makes it download faster.
If you're using this, make sure that you're specifying the "APP_PATH" as "sauce-storage:your_file_name" -- NOT the saucelabs.com/rest/v1/storage... URL.
And when you upload your file to our server, check the JSON response. It should be formatted like this:
{"username": "USERNAME", "size": 99999, "md5": "bc01aad123456789glhf9ee6e8c7eab3", "filename": "AppName.zip"}
Make sure that the size is something reasonable for the .zip file you uploaded. If not, check your file and try again. This should resolve the issue.
Here's some general information that can be useful when troubleshooting tests that ran through Sauce Connect, as well as Connect itself.
-- Connectivity issues (help, I can't hit my app server!) --
The most common cause of irregular connectivity, in jobs using Connect, is a tunnel that has gone stale. This is definitely the cause if you can't hit public sites such as google.com, and can occasionally occur when Connect has been running for an extended period of time, or with a comparatively large amount of data passed through the tunnel. We usually recommend restarting Connect once per day, or more frequently if you are running over 100 jobs in that time: http://support.saucelabs.com/entries/21068387-keeping-sauce-connect-fresh
If that's the case, restarting Connect should fix the issue. And if it doesn't, just send us the sauce_connect.log file from when the job was running.
-- Sauce Connect shut down and I don't know why --
Sauce Connect does not include an auto-shutdown or restart feature. It can be shut down in a variety of ways:
• If it loses connection to Sauce for an extended period of time
- This produces general disconnecting/reconnecting messages, potentially with some successful commands sent in between
- One potential error message is "Problem connecting to Sauce Labs REST API"
- You may also see "IOException: java.net.NoRouteToHostException: No route to host"
• If the machine running Sauce-Connect.jar is shut down unexpectedly (for example, by a closed laptop)
- This causes very visible repeated attempts and failures to connect, upon reopening the terminal
• If a user presses Ctrl+C in the terminal, which prompts Connect to shut down the tunnel cleanly
- This produces the message "received SIGINT"
• By another Sauce tunnel launching (the .jar automatically checks for pre-existing tunnels, and attempts to kill them before opening a new one)
- This produces the message "Sauce Connect connection stalled!"
• Via the "Shut Down" link at https://saucelabs.com/account/tunnels
- This produces the message: "asked to close connection to Sauce Connect server"
If none of these is triggered, Connect can be left running for days or months, although it may go stale after transmitting a lot of data, which is why we recommend implementing a daily restart script: http://support.saucelabs.com/entries/21068387-keeping-sauce-connect-fresh
The first step in troubleshooting should always be to quit and restart Connect. Since the launch process includes a number of checks, this will resolve 90% of common issues.
-- Sauce Connect can't launch a tunnel at all --
If you see error messages like this, repeated many times over, sometimes after a seemingly successful start:
[timestamp] - sauce_connect:0 - INFO - connection to Sauce Connect server closed
[timestamp] - sauce_connect:0 - INFO - Connection failed
Make sure that you have the necessary ports open to run Sauce tests:
• charon.saucelabs.com (port 80, 443, or 6080) - for VNC sessions, which include live job video and interactive testing.
• ondemand.saucelabs.com (all ports) - to make contact with our testing VMs.
• saucelabs.com (ports 80 and 443) - for Sauce Connect to establish a tunnel. These should be open and unproxied, as Sauce Connect's traffic can't be sent through an HTTP proxy. (More on this: https://saucelabs.com/docs/connect)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment