My name is Kevin Lorenz, I'm a frontend developer with Finleap in Berlin, I'm one of the organizers of CSSconf EU and together with a few friends I run an event called CSSclasses.
If you want to talk to me on the web, you can find me on Twitter as @verpixelt, and I occasionally write articles on Medium, because a developer does not simply finish their own website.
Most of the time, I've called myself a developer, I've spent writing CSS. Over time we as developers got more and more things we can do with it. Accomplishing layouts became easier and applications we want to build got more complex at the same time. This complexity has made my job more interesting and difficult at once.
And I have a question for you about it, and I’d ask you to be honest: How many of you have, in their career, tried to fix a CSS issue on one part of an application and by doing so have introduced new bugs or issues on other parts? So wouldn't it be nice to never let that happen again? To just end this situation once and for all and increase your confidence in the code you've written?
This is what testing is really useful for. Software testing isn't something that is new to our profession. It has been there for many years – in fact it is much older than most of us are. No matter which programming languages or software types we talk about, the fundamental goal of all software testing is the same: Determining how correct the code is that we've produced. We can all agree that testing is important. It gives us more flexibility, more reliability for us and our software users, prevents or at least detects human errors and after all that, it helps us as developers to gain more confidence that our code behaves as expected. This leaves mental capacity for other tasks, because testing can be done by a computer.
With increasing application complexity comes the necessity to introduce different types of tests. CSS determines the visual styles of websites and user interfaces we build. CSS is important! So we need to make sure that the visual output of our CSS is exactly what it’s supposed to be, To achieve this, we can use Visual Regression Testing – and this is precisely what I’m going to talk about today.
So what exactly is visual regression testing?
Visual regression testing is a type of software testing that warns you about visual changes in our application or website, during the development process.
That’s basically the same as regression testing but on a visual level. It helps us by taking screenshots of our web application or website at two different points in time and compare them with each other to find potential differences.
Why do we need such tests? By default, CSS has a global scope. There is not really much we can do about that if we haven't exposed the idea of interoperable CSS and CSS Modules to our dev stack already. Properties and their values trickle down our whole application from general to most specific and they do this all the time. This is a feature. The C in CSS still stands for cascading – but at the same time, breaking things is made easy. Here are a few reasons why you may want to use visual regression tests for your software.
I don't remember who said it originally but "working with CSS is easy – mastering it is one of the hardest things". I couldn't agree more. And don't get me wrong. I really love working with CSS and probably wouldn't stand here otherwise. CSS has a certain amount of finesse and elegance to it. It's not just slapping something in its own scope, doing whatever one wants and you’re fine. But this also means that things are more complicated than we often think in the first place. And until we all use something like CSS Modules, we won't be able to cross this point off from our list.
Inheriting code bases from other developers is rarely known as one of the fun parts of our job but is one of the things that happens a lot. We don't know how well the CSS we've inherited is written until we break something and have to look into it at a deeper level. Working on someone else’s code base can bring a feeling of fear with it that is highly unpleasant. And it comes with nightmares – for example sentences like "Could you please just build that small feature on top of this already written code base".
Refactoring code is even worse. I have some sort of love / hate relationship with refactoring. On the one hand refactoring can give you the excellent feeling of cleaning up and making things better, realising features with 4 lines of code instead of 20, using Flexbox instead of Floats, getting rid of all the clear fixes and other hacks. But simultaneously the fear of breaking things is lurking behind the next curly brace.
To be more specific it's not the fear of breaking things in general, because breaking things is fun. It's the fear of breaking things and noticing it just after the live deployment. That’s when regression testing helps.
Nowadays web applications, web sites, and user interfaces are all more complex than ever. We have all sorts of options, menus, hidden menus, flyouts, pop-ups, pop-unders. There is a ton of stuff in a modern piece of software for the web we have to worry about. Recently I had a look at an admin interface I had to build and I was shocked by how many different views we have in it. Complexity makes it hard to keep track of what is actually going on – and what we might break if we change things.
Hand in hand with complexity comes responsiveness. There are at least the three most infamous ones: phone, tablet and desktop, on which our web application has to work and look stunning at the same time. This means we get breakpoints in our CSS files, and properties overwriting other properties in different contexts. We build this for different view ports on different devices running different operating systems.
Saying this out loud makes me question myself: why have I been working for so long without visual regression tests?
These are just a few points but enough to convince me that visual regression tests are necessary. And I assure you, we will find more reasons, the longer and deeper we look at it. Visual regression testing has recently started gaining more popularity for those reasons and more. I'm not sure why only recently but we simply have the need for it right now and it’s a good time to start with it. But this is not even all. On top of the technical mumbo-jumbo there is the human element to all of this. Our natural potential for flaws multiplies the likelihood of failing.
It turns out that humans are actually pretty terrible at spotting visual changes. Our inability to notice changes has become an increasingly studied set of physiological and psychological phenomena. Scientists like Daniel Simons from the University of Illinois have spent years devising experiments testing how perceptive we humans actually are.
We always feel like we're seeing the world in all its completeness and its detail. We feel like we're experiencing the world as it is – but that’s just an illusion. What we actually experience is what our mind and our brain give us – it's an alternate reality.
Why is that? Because we actually only take in detail from a tiny subset of the visual world at any instant.
I want to show this to you with a tiny exercise: If you all could hold up your arm like this (please don't hurt anyone) and now stick up your thumb. Look at the width of your thumb. Pretty tiny, isn’t it? And now think about this: We only take in high resolution information from the information of the width of our thumb. Beyond that everything becomes progressively blurrier.
We don't notice how narrow our perception is because we move our eyes 3-4 times a second – which we also don't notice. So every time something interesting comes into our periphery we look at it and see it in detail. But we’re still under the false impression that we see everything in detail.
We assume that everyone sees the “real” version of the world – while, in fact, we think everyone sees the same things we do – and even our own picture of the world is by far not complete.
That has profound implications on how we think about the world around us.
Visual illusions, for example, are a perfect way to illustrate that we actually do not see the world how it is.
Julian Beever is famous for his 3D illusion pavement drawings. Here you can see one of his pieces. Of course there is no real gigantic slice of pizza in that picture but still our mind gets a visual impression of depth. In that case it's a matter of perspective.
The fact our brains work like that not only affects what we see – it also affects how we think, how and what we remember, and how we reason.
This also affects eyewitness testimonies. 72% of eyewitness identifications were later identified as false by DNA evidence. These numbers were published by the Innocent Project, an organisation which fights for the rights of people who were wrongfully convicted.
We have also made games out of the flaws our human body has. Remember the "spot the differences" images?
These only work so well because of our inability to take in high resolution information of everything around us at the exact same time. Therefore our eyes basically jump between sections of each image back and forth to focus and get detailed information.
There are two similar phenomena which also affect us as developers in our daily work. The first one is
Inattentional Blindness is the failure to notice something that is obvious and right there in front of us when our attention is engaged on something or someone else.
The second one is change blindness which is a failure to notice the difference between what's there right now and what was there a moment ago.
I’m going to show you a short video clip which perfectly demonstrates these phenomena.
It shows how much the human eye can miss if we don't know what we are looking for.
So everything I've told you about our perception for the last few minutes sounds worrying and only points out our body’s failures. But we should look at it from a different angle.
What we do remarkably well is: we’re really good at focussing attention. We can focus attention on what we need to see without getting distracted by the irrelevant stuff. If you would notice every little change, every little detail at any given moment, it would be impossible for you to listen to me and follow along. Or have a conversation with a fellow attendee during the break.
Let's now focus our attention on what tools we can use to help us accomplish our job. Because testing is the only way for us to ensure that what we see is what is really there.
Thanks to the web and our community there are plenty of tools for visual regression testing that we can choose from. In the next minutes, I will give you the rough recipe most of these tools follow, name a few and give you a short live demo of the one I currently use.
In general all of these tools work with some version of a browser. Either it's one with a graphical user interface like Firefox, Chrome, Opera, etc. or one of the headless browsers like PhantomJS or HTMLUnit.
For all of you who aren't familiar with the concept of a headless browser – it is a browser without a graphical user interface which gets executed via a command line interface.
A few common use cases for headless browsers are scraping websites for data, running automated tests for JavaScript libraries or test automation and taking screenshots.
The next thing to get in addition to our browser or headless browser would be a suitable testing utility like CasperJS or Selenium. These programs allow us to interact with our browser. They navigate through web applications, logg events, fill out forms and submit them, to name a few examples. Tools like this allow us to recreate and therefor test all the usual user interactions.
The 3rd item on our list is a tool to actual do the regression testing for us. Here the choice can be a tough one because of the sheer amount of options you have.
WebdriverCSS and PhantomCSS are probably the most popular ones available out there at the moment.
The fourth and last missing piece is software to process the screenshots taken by our regression testing tool. Which one you gonna use depends on the tool you've chosen to use for your regression tests.
For WebdriverCSS it's graphicsmagick, phantomCSS uses Resemble.js.
Now we have all our tools: a (headless) browser, a testing utility, a visual regression testing tool, and software to process our screenshots. Next up, we have to connect these dots. To show you how this can be done, I've prepared a demo setup we will walk through together. But before we do this I want to do a brief recap of what we've heard so far.
The web is in constant movement. That’s a good thing. But at the same time it is getting more and more complex every single day and the responsibilities of a frontend developer are growing as well.
Keeping track of visual changes in a reliable way isn't possible anymore without technical assistance. By admitting this we've done the first step. The web offers a plethora of tools you can choose from, so there is one for everyone's preferences.
Getting into visual regression testing now is the best you can do to make your life as a developer more pleasant and prevent your users from running into graphical issues using your product.
Let's take a look at the technical details of how you might actually use these tools.
Working in the web we have nowadays can be encouraging and challenging at the same time. We constantly get more and more things to take care of more and more to play with. The fact that we are humans doesn’t make it easier in any way. But we’re capable of handling tech to some degree and build tools to help us, where help is needed.
People have already built tools to test the visual outcome of your project and most of them are available for free. These tools are good starting points to make your human made changes more technically proven.
Investing the time to write visual regression tests right from the beginning pays off in the end and will give the next developer who has to work with your code less of a headache – even if that next developer is you.
- Hand icon made by Maico Amorim, BR https://thenounproject.com/term/hand/12769
- Thumb up icon made by Creative Stall, PK https://thenounproject.com/term/thumbs-up/122840
- Eye icon, public domain https://thenounproject.com/term/eye/391
- Google Chrome icon made by Sofía Moya, MX https://thenounproject.com/term/chrome/38318
- Toolbox icon made by By Patrick Snyder https://thenounproject.com/term/toolbox/100146
- Screenshot icon made by Krisada, TH https://thenounproject.com/term/screenshot/28423
- Settings icon made by Creative Stall, PK https://thenounproject.com/term/settings/177012
- Checklist icon made by Creative Stall, PK https://thenounproject.com/term/approve/143704