I recently started a new project and we used Angular CLI to get started. Angular CLI, as in the name, is a command line utility for creating and managing Angular 2 projects. Using Angular CLI to create a project is very easy and it gives you a great starting point for new Angular 2 projects. The only draw back I found was that in my mind it wasn't CI ready.
Angular CLI out of the box gives you a few unit tests and an end to end (e2e) test. This is great because you can generate a project and set up your build server to build the artefacts. This is where I ran into problems.
Having everything generated for you is great until something you want to do does not work; and this is where I was. I wanted to build and test my angular application on a headless build agent. The generated code from Angular CLI runs tests using Google Chrome by default. Which is fine, but running Google Chrome on a build server with no desktop environment proved challenging.
The build agent in question was one which I was provisioning to be added to a TeamCity Server. Docker makes it very easy to create and manage build agents and I was using TeamCity Minimal Build Agent as a starting point which is a Ubuntu 15.10 image with a few things pre-installed.
My first approach was to look for a headless browser like PhantomJS. PhantomJS is a headless browser which was designed to be used for testing in headless environments. Configuring PhantomJS to run in place of Google Chrome was pretty straight forward.
Install the Karma PhantomJS launcher:
npm install --save-dev karma-phantomjs-launcher
Update karma.conf.js to use PhantomJS:
module.exports = function(config) {
config.set({
...
plugins: [
...
require('karma-phantomjs-launcher'),
...
],
browsers: ['PhantomJS'],
...
})
}
Changing the browser from Chrome to PhontomJS was simple, however the few example tests that the Angular CLI had generated started failing. Not what you want on a build server. After spending some time Google'ing the errors it seems like PhantomJS does not support ES6 and its ES5 support was questionable. Again not great for a build server. This lead me to looking into compiling the tests into ES5 in the hope that PhantomJS would be able to run the tests then, but after struggling here I decided that this was not the best approach. If PhantomJS does not fully support ES5 then surely I will run into issues in the future.
So back to looking for a headless browser. Searching for a headless browser brought up a lot of posts about launching Chrome with a headless option, and this looked promising, but I was never able to get this right. It looks like this maybe a feature that Google will add to Chrome in the future, but I needed a solution today. Instead this lead me to something far more versatile; Virtual Frame Buffers.
Basically this post was the holy grail, Running a GUI application in a Docker container. I had never heard of Xvfb and after giving that a read I was pretty confident that Xvfb could solve my problem. A virtual frame buffer is a in memory x11 display server. Which basically means that you can run normal GUI applications which out the need for a desktop environment.
The outcome was simple, all I needed was a virtual frame buffer to run a browser in. As I originally stated I set out to build my Angular 2 application on a headless docker build agent.
I started off with the TeamCity Minimal Build Agent and inside that docker container I installed Xvfb and its dependencies.
From the blog post above, I decided to try Firefox first as it looked like a simpler option. So on the build agent I ran the following commands, which where derived from Headless Browser Testing With Xvfb:
apt-get install xvfb firefox dbus-x11
With that installed I ran a quick test on the build agent, can I launch a headless browser?
xvfb-run firefox
At this point I didn't see any major error and made the assumption that it works. Using a virtual frame buffer I was not expecting to see any UI of any form but was hoping that not log lines would print out errors. The next step was to update my karma.conf.js file to use Firefox in place of Chrome and see if the build server works.
Similarly to replace Chrome with PhantomJS, replacing PhantomJS with Firefox is pretty straight forward.
Install the Karma Firefox launcher:
npm install --save-dev karma-firefox-launcher
Update karma.conf.js to use Firefox:
module.exports = function(config) {
config.set({
...
plugins: [
...
require('karma-firefox-launcher'),
...
],
browsers: ['Firefox'],
...
})
}
The fancy part comes in here, I updated the package.json file to run the tests using our virtual frame buffer:
...
"scripts": {
...
"test:ci": "ng test --browser=Firefox --code-coverage=true --single-run=true",
"test:xvfb": "xvfb-run npm run test:ci",
...
}
...
And success, well partial success. The test ran and passed, but the e2e tests failed. This was because the e2e tests where still using Chrome.
Configuring the e2e tests to use Firefox in place of Chrome is probably possible but I decided with this new knowledge of virtual frame buffers maybe I can get Chrome working in the same way.
So back to karma.conf.js I used a custom launcher to launch Chrome in a "headless" mode. I say "headless" but its really just giving Chrome some arguments which allow it to launch in a headless environment. I followed the guide shown in Headless Chromium and reading the errors used the arguments'--no-sandbox' and '--disable-gpu'.
Note: I tried Chromium first but I got an error to do with the version being too old. To solve this I installed the latest Google Chrome from their repository.
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
apt-get update
apt-get install google-chrome
Success, I could launch Google Chrome with the following command:
Note: I did see some errors but they didn't appear to be critical.
xvfb-run google-chrome --no-sandbox --disable-gpu
Now that I could launch a 'headless' Google Chrome, I replaced the Firefox launcher with a custom Chrome launcher which will launch Google Chrome with the given arguments.
karma.conf.js:
module.exports = function(config) {
config.set({
...
plugins: [
...
require('karma-chrome-launcher'),
...
],
browsers: ['Chrome'],
customLaunchers: {
Headless_Chrome: {
base: 'Chrome',
flags: [
'--no-sandbox',
'--disable-gpu'
]
}
...
})
}
package.json:
...
"scripts": {
...
"test:ci": "ng test --browser=Headless_Chrome --code-coverage=true --single-run=true",
"test:xvfb": "xvfb-run npm run test:ci",
...
}
...
Note: On the build agent I needed to define the following environment variable:
export CHROME_BIN=/usr/bin/google-chrome
This tells the Karma Chrome launcher to use the Google Chrome we installed earlier.
To get the e2e tests to run on the build server I needed to add the same Chrome arguments to the protractor configuration file which launches Chrome.
protractor.conf.js:
exports.config = {
...
capabilities: {
'browserName': 'chrome',
'chromeOptions': {
'args': [
'--no-sandbox',
'--disable-gpu'
]
}
},
...
}
Lastly I needed to update the package.json file to use Xvfb when running the e2e tests.
package.json:
...
"scripts": {
...
"e2e:xvfb": "xvfb-run npm run e2e",
...
}
...
And finally we can run all our tests and e2e tests on a headless build server!
So lets wrap things up. Using Xvfb we can run GUI applications in a headless docker container. That allows us to run a browser which we can use to test our Angular application. This makes it idea for using in a build server environment where the build agents may not have desktop environments.
Using a very simple project generated from the Angular CLI I have been able to run all the tests, both unit and e2e tests, on a headless build agent using Xvfb. Using Xvfb is a very versatile solution as I am not restricted to using just one browser, I can theoretically use any browser that will run in a UNIX environment.
Here are the final complete files of interest:
karma.conf.js:
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '@angular/cli'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-firefox-launcher'),
require('karma-phantomjs-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular/cli/plugins/karma')
],
browsers: ['Chrome'],
customLaunchers: {
Headless_Firefox: {
base: 'Firefox',
flags: []
},
Headless_Chrome: {
base: 'Chrome',
flags: [
'--no-sandbox',
'--disable-gpu'
]
}
},
client: {
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
files: [
{
pattern: './src/test.ts',
included: true,
watched: false
}
],
preprocessors: {
'./src/test.ts': ['@angular/cli']
},
mime: {
'text/x-typescript': ['ts', 'tsx']
},
coverageIstanbulReporter: {
reports: ['html', 'lcovonly'],
fixWebpackSourcePaths: true
},
angularCli: {
environment: 'dev'
},
reporters: config.angularCli && config.angularCli.codeCoverage
? ['progress', 'coverage-istanbul']
: ['progress', 'kjhtml'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: false,
singleRun: true
});
};
package.json:
{
"name": "bwe-web-app",
"version": "0.0.0",
"license": "MIT",
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build",
"build:prod": "ng build --target=production",
"test": "ng test",
"test:ci": "ng test --browser=Headless_Chrome --code-coverage=true --single-run=true",
"test:xvfb": "xvfb-run npm run test:ci",
"lint": "ng lint",
"e2e": "ng e2e",
"e2e:xvfb": "xvfb-run npm run e2e",
},
"private": true,
"dependencies": {
"@angular/common": "^2.4.0",
"@angular/compiler": "^2.4.0",
"@angular/core": "^2.4.0",
"@angular/forms": "^2.4.0",
"@angular/http": "^2.4.0",
"@angular/platform-browser": "^2.4.0",
"@angular/platform-browser-dynamic": "^2.4.0",
"@angular/router": "^3.4.0",
"core-js": "^2.4.1",
"intl": "^1.2.5",
"rxjs": "^5.1.0",
"zone.js": "^0.7.6"
},
"devDependencies": {
"@angular/cli": "1.0.0-rc.1",
"@angular/compiler-cli": "^2.4.0",
"@types/jasmine": "2.5.38",
"@types/node": "~6.0.60",
"codelyzer": "~2.0.0",
"jasmine-core": "~2.5.2",
"jasmine-spec-reporter": "~3.2.0",
"karma": "~1.4.1",
"karma-chrome-launcher": "^2.0.0",
"karma-cli": "~1.0.1",
"karma-coverage-istanbul-reporter": "^0.2.0",
"karma-es6-shim": "^1.0.0",
"karma-firefox-launcher": "^1.0.1",
"karma-jasmine": "~1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
"karma-phantomjs-launcher": "^1.0.2",
"karma-typescript-preprocessor": "^0.3.1",
"protractor": "~5.1.0",
"ts-node": "~2.0.0",
"tslint": "~4.4.2",
"typescript": "~2.0.0"
}
}
protractor.conf.js:
const { SpecReporter } = require('jasmine-spec-reporter');
exports.config = {
allScriptsTimeout: 11000,
specs: [
'./e2e/**/*.e2e-spec.ts'
],
capabilities: {
'browserName': 'chrome',
'chromeOptions': {
'args': [
'--no-sandbox',
'--disable-gpu'
]
}
},
directConnect: true,
baseUrl: 'http://localhost:4200/',
framework: 'jasmine',
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 30000,
print: function() {}
},
beforeLaunch: function() {
require('ts-node').register({
project: 'e2e/tsconfig.e2e.json'
});
},
onPrepare() {
jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
}
}
And for clarity here is the Dockerfile I used during testing:
FROM jetbrains/teamcity-minimal-agent
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt install -y \
dbus-x11 \
firefox \
google-chrome-stable \
nodejs \
npm \
wget \
xvfb \
&& npm install -g \
tslint \
ENV CHROME_BIN /usr/bin/google-chrome