##Gulp workflow tutorial
This is a quick tour through setting up and using a gulp workflow - at least the way I do it currently as a relative novice. It has massively speeded up my workflow though! I have made a video to accompany this tutorial, which illustrates the process here. You can find it at : https://www.youtube.com/watch?v=LfMBHPp_l5k&feature=youtu.be
###NPM
NPM (node package manager) is a tool that allows you to install javascript ‘nodes’ which are programs that are written in javascript that use the chrome javascript engine that has been ported to a program called nodejs.
Nodejs is installed by visiting the following link:
https://nodejs.org/en/download/package-manager/
On ubuntu, npm is installed at the same time.
The first thing that one should do is update npm itself with the following command:
sudo npm update npm -g
In the preceding command, npm is instructed to update to the latest version the package npm. The flag -g instructs npm that it is to update globally – more on this below. The sudo is necessary if one is using a non-root account on a linux distro such as ubuntu.
Npm installs packages into one of two spaces. The global space (referenced by the flag -global abbreviated to -g as in the command above) allows one to utilise that package by referencing it anywhere in the machine’s file system hierarchy, just as one would with a normal o/s program. The local space (which is referenced by the flag –save-dev or or –save) can only be referenced from the project folder. When one installs a package using the –save or –save-dev flag the package is downloaded and installed in a folder called ‘node_modules’ which is in the root of your project.
The two different flags have different meanings: ‘save’ is used to install packages that will be used at runtime by the web app that you are creating. These packages will be resident on the server where your app is running. The packages that are installed using ‘--save-dev’ are packages that you only require for use whilst developing the app and are available to you on your development machine. These packages are not available to your app at run time. This is as I understand it at the moment, but it may be more of a notional division of resources, a sort of shorthand for you the developer, than anything else.
###Gulp
Gulp is a package that can be installed using npm. It is the program that I am writing this introductory tutorial about, and is great for speeding up your workflow amongst other things. Gulp is a tool that manages ‘streams’. In general a ‘stream’ is a buffer of characters that can be ‘piped’ from one place to another. Text input at the console is considered to be a stream, so is data from the internet, and so is a file of characters when passed to another place or program. In the case of gulp, the text files that you create, such as html files, css files etc are streamed to programs who then transform the content in some way before streaming the transformed file to another place. Examples of transformation that I use are ‘autoprefixer’ which takes your css input file and adds all of the vendor prefixes necessary for all of the different browsers. Not only does this speed up your workflow but it also lessens the likelihood of you making an error as you manually reference the vendor prefixes or a typo etc.
with gulp-autoprefixer running this css in your src folder :
.fourth-face {
display: flex;
justify-content: space-between;
}
.fourth-face .column {
display: flex;
flex-direction: column;
justify-content: space-between;
}
becomes this in your build folder:
.fourth-face {
display: -webkit-box;
display: -webkit-flex;
display: -ms-flexbox;
display: flex;
-webkit-box-pack: justify;
-webkit-justify-content: space-between;
-ms-flex-pack: justify;
justify-content: space-between;
}
.fourth-face .column {
display: -webkit-box;
display: -webkit-flex;
display: -ms-flexbox;
display: flex;
-webkit-box-orient: vertical;
-webkit-box-direction: normal;
-webkit-flex-direction: column;
-ms-flex-direction: column;
flex-direction: column;
-webkit-box-pack: justify;
-webkit-justify-content: space-between;
-ms-flex-pack: justify;
justify-content: space-between;
}
from https://davidwalsh.name/goodbye-vendor-prefixes
...and it does this automatically every time you save your css file. This is because you instruct gulp to watch your directories for changes made to files and then tell it what to do to your files and in what order it should do them, as well as where it should output the end result. One tells gulp to do this in a file called ‘gulpfile.js’, which lives along with the npm package file ‘package.json’ at the top of your projects file hiearchy. There are packages for editors like css3 for sublime text that auto-prefix automatically, but I find this muddies the source code, making it less readable.
An example file hierarchy:
build gulpfile.js node_modules package.json src
./build:
directories: assets css js
files: index.html
./build/assets:
files: logo.jpg
./build/css:
files: style.css
./build/js:
files: javascript.js
./src:
directories: css js tmp
./src/css:
files: stylesheet.css
./src/js:
files: javascript.js
./src/tmp:
Gulp is not the only tool that can be used to transform input. Other common examples are Grunt and Brunch. All have plugins but the syntax varies as does the speed of use. Grunt is complex and slow, but powerful and used by a majority. Brunch is quick and easy to setup, but is more restrictive. Gulp is newer than grunt but resembles it slightly in that it has the powerful syntax, but is faster than grunt and is used by more people than use brunch, as I understand it at the moment. In order to use gulp in your project files you will need to install gulp globally with the following command:
sudo npm install gulp-cli -g
This installs gulp commandline interface globally so that you can search for packages etc.
In my gulpfile I use gulp-sass, which transforms my sass code into css everytime I save the file. There are lots of benefits of using sass, for instance, I can write my html file and then copy the html structure as nested sass nodes, making it easier to find content in large projects. I can write calculations, use variables and stick to the DRY principle by using the extend and import keywords in Sass, which you cannot do in css. I can write my media queries within the css rule that selects the element that is described by the media query which means I can group all of my media-queries together with the elements that they describe keeping my sass file ordered and easy to read, and the element selectors easy to locate. The gulp-sass plugin transforms the sass file (located in the directory ‘src/sass/’) to css and saves it in the ‘build/css/’ directory.
Another plugin that I find very useful is browser-sync. This watches the build directory so that every time I save a file it is transformed by the appropriate plugin/s and then saved in the build directory tree and then browser-sync detects the change and refreshes all the browsers that are attached to the server that it sets up. When you start gulp the browser-sync plugin is started and it can either start a python server to serve the files from your directory – this is the default – or if you are making a non-static web app that needs to serve php or python for example, then you can install a regular apache server and instruct browser-sync to proxy that server address. Once browser-sync is running, you can point browsers on any of your devices on the local subnet or even the WAN (if you do a little port forwarding) at the address that browser-sync provides and these will auto-refresh every time you change a file that is watched. This means you can edit on your machine and see the results of your editing on a variety of different sized displays around you - great for responsive design!
So the gulp workflow works like this:
First of all make your project directory. Then cd into that directory and initialise npm.
npm init
This will create a package.json file with a short description in it.
Then install your npm modules. For example:
npm install browser-sync gulp autoprefixer gulp-sass –save-dev
Make your directory structure:
mkdir -p ./{misc,src/{sass,scripts,assets,tmp},build/{css,scripts,assets}}
create and edit your gulpfile.js or copy in an existing file. If you copy in an existing file from another project then you can copy also the package.json file from that project and run the command ‘npm update’ and then the pacakages will install automatically.
So, here is a gulpfile example from my project:
var gulp = require("gulp");
var browserSync = require('browser-sync').create();
var postcss = require('gulp-postcss');
var sorting = require('postcss-sorting');
var sourcemaps = require('gulp-sourcemaps');
var autoprefixer = require('autoprefixer');
var sass = require('gulp-sass');
var cleanCSS = require('gulp-clean-css');
var rename = require('gulp-rename');
var notify = require('gulp-notify');
var htmllint = require('gulp-htmllint');
var gutil = require('gulp-util');
var htmltidy = require('gulp-htmltidy');
gulp.task('html-tidy', function() {
return gulp.src(['./src/*.html','./src/**/*.html'])
.pipe(htmltidy({doctype: 'html5',
hideComments: true,
indent: true}))
.pipe(htmllint({}, htmllintReporter))
.pipe(gulp.dest('./build'));
});
gulp.task('browser-sync', function() {
browserSync.init({
server: {
baseDir: "./build/"
}
});
gulp.watch(["./build/*.html","./build/**/*.html"]).on('change', browserSync.reload);
//gulp.watch("./build/css/*.css").on('change', browserSync.reload);
});
gulp.task('sass', function () {
return gulp.src('./src/sass/sass.scss')
.pipe(sass().on('error', errorHandler))
.pipe(rename ("style.css"))
.pipe(gulp.dest('./src/tmp/'));
});
gulp.task('sortcss', function (cb) {
return gulp.src(['./src/tmp/*.css','./src/tmp/**/*.css'])
.pipe( postcss([sorting({})]))
.pipe(gulp.dest('./src/tmp'));
cb(err);
});
gulp.task('autoprefixer', ['sortcss'], function (cb) {
return gulp.src(['./src/tmp/*.css','./src/tmp/**/*.css'])
.pipe(sourcemaps.init())
.pipe(postcss([ autoprefixer({ browsers: ['last 2 versions'] }) ]))
.pipe(sourcemaps.write('.'))
.pipe(gulp.dest('./build/css'));
cb(err);
});
gulp.task('css-minify', ['autoprefixer'], function(cb) {
return gulp.src(['./build/css/*.css','!./build/css/normalize.css','!./build/css/*.min.css'])
.pipe(cleanCSS({debug: true}, function(details) {
console.log('original : ' + details.name + ': ' + details.stats.originalSize);
console.log('minified : ' + details.name + ': ' + details.stats.minifiedSize);
}))
.pipe(rename({suffix: ".min"}))
.pipe(gulp.dest('./build/css'));
cb(err);
});
gulp.task('css-rename', ['css-minify'], function() {
return gulp.src(['./build/css/*.css','!./build/css/*.min.css'])
.pipe(gulp.dest('./build/css'))
.pipe(browserSync.stream({once: true}));
});
/* // swap with browser-sync for apache proxy
gulp.task('browser-sync', function() {
browserSync.init({
proxy: "yourlocal.dev"
});
});*/
gulp.task("default", function() {
console.log("Default task");
});
gulp.task("scripts", function() {
console.log("Scripts task");
});
gulp.task("build", ["scripts", "browser-sync"], function() {
gulp.watch(["./src/*.html", "./src/**/*.html"], ['html-tidy']);
gulp.watch('./src/sass/*.scss', ['sass']);
var watcher = gulp.watch(['./src/tmp/*.css','./src/tmp/**/*.css'], ['css-rename']);
watcher.on('change', function(event) { console.log('File ' + event.path + ' was ' + event.type + ', running tasks...');
});
});
// Handle errors
function errorHandler (error) {
report(error);
this.emit('end');
}
function report(error)
{
notify().write(error);
console.log(error.toString());
}
function htmllintReporter(filepath, issues) {
if (issues.length > 0) {
issues.forEach(function (issue) {
switch(issue.code)
{
case 'E036':
case 'E011':
case 'E001':
case 'E002':
break;
default:
report("error in html, check console!")
console.log( filepath + " : \n" +
issue.line + " : " +
issue.column + " : " +
issue.code + " : \n" +
issue.msg );
}
})
}
//this.emit("end");
}
I've included this gulpfile and the package.json below. To use them, install npm, and then copy and paste them into your project directory root. Then open a console in your project directory root and type 'npm update'. This will install all the packages from the package.json file. Following this, you can run the command 'gulp build'.
Making a gulpfile is not that easy at first - at least it took me a little while to get my head around the idea of running tasks syncronously. By default, gulp runs its tasks asyncronously, and so if you are watching a directory and a file changes, gulp will run the tasks that are specified in no guaranteeable order. This means that your file might be minified before being prefixed, which might go wrong. To fix this, one can specify an order in which to run the tasks using callbacks.
A standard gulp task is defined as follows:
gulp.task('sass', function () {
return gulp.src('./src/sass/sass.scss')
.pipe(sass().on('error', errorHandler))
.pipe(rename ("stylesheet.css"))
.pipe(gulp.dest('./src/tmp/'));
});
once this task is defined, one can then run that task from the command line by typing in the command 'gulp sass'. The above task uses a plugin called sass (.pipe(sass()...) which compiles the sass file; an errorhandler is defined for this task. The output is then piped as a stream to a plugin called 'rename' which names the file stylesheet.css. This is then piped to a gulp.dest which places the file in '/src/tmp/'. Gulp plugins generally have a gulp.src and a gulp.dest which specify where the file begins and ends.
Most gulp files have a task called gulp.build - this is the principle task that is used to run the gulp workflow. Here is the one from my project:
gulp.task("build", ["scripts", "browser-sync"], function() {
gulp.watch(["./src/*.html", "./src/**/*.html"], ['html-tidy']);
gulp.watch('./src/sass/*.scss', ['sass']);
var watcher = gulp.watch(['./src/tmp/*.css','./src/tmp/**/*.css'], ['css-rename']);
watcher.on('change', function(event) { console.log('File ' + event.path + ' was ' + event.type + ', running tasks...');
});
});
Here the gulp.watch plugin is being setup to run the scripts task and the browser-sync task and watch some directories for changes to files and instructed to run one of the tasks that I have defined if the files change. Recall that I mentioned that tasks can be run synchronously? Well, have a look at the final watch task defined in the build handler. When the sass task has finished running it deposits the stylesheet.css into the 'src/tmp/' directory. This is when that final task begins the css-rename task. If you look at the css-rename task you will see that it defines another task in square brackets:
gulp.task('css-rename', ['css-minify'], function() {
return gulp.src(['./build/css/*.css','!./build/css/*.min.css'])
.pipe(rename({suffix: ".min"}))
.pipe(gulp.dest('./build/css'))
.pipe(browserSync.stream({once: true}));
});
the task in the square brackets is 'css-minify'. This task is run and completes before css-rename will run. If you then look at the css-minify task, this defines the auto-prefixer task to run before it completes, and the auto-prefixer task defines the sortcss task to run before it completes. This is by no means the correct way to do this, that is one of the issues with gulp, there exists no correct way to do things, which is either a strength or a weakness depending on your point of view. This is one of my first gulpfiles so I will undoubtedly improve with more practice but it works!
If you examine the autoprefixer task, it defines a callback function as a parameter - '....function (cb) (where cb is the callback). At the end of the function is the statement cb(err). This instructs gulp that should the task fail for any reason, it will pass the error code to the callback that is passed into the function, so that that callback and its errorcode will be available to the task that defines it. At least that is how I understand it to be at the moment.
Several of the tasks have errorhandlers defined for the one of the plugins. These are generally part of the options object that one passes to the plugin, one defines a callback error handler such as the errorHandler function that I have written at the bottom of my gulpfile.js and should an error occur in the plugin (rather than in the task itself as is the case with the cb(err) as discussed above) then the errorHandler that you have defined will take care of that error. In my gulpfile I have used the gulp-notify plugin to connect to the notify deamon on my os, so that should an error occur, I will be notified of that error without having to examine the console repeatedly. With the browser autorefreshing in another window, I can concentrate on my editor without having to switch to another window continuously, which speeds up my workflow enormously.
The final errorHandler that I have defined is the htmllintReporter errorhandler. The signature of this function is defined by the htmllint plugin (linting is the process of checking for errors and correcting the syntax - thus there exists css-lint html-lint etc etc). In the body of the function you will notice that I have put a switch statement. This is to catch and cancel out errors from being reported that are present in all files. Canceling out these false positives leaves me less likely to miss out on noticing a more important error.
###GIT
One other thing that I use is git - global information tracker - which helps me to keep a record of file changes and to rewind any mistakes that I have made and can be used for collaboratively working on a project. I won't describe it in any depth here, but I'd recommend reading about it and installing it.
Gulpfiles and their accompanying package.json are easily distributable often in the form of github gists. For example:
https://gist.github.com/LandonSchropp/2816314bb336fbe1f4e6 which is from the above referenced article or: https://gist.github.com/search?utf8=%E2%9C%93&q=gulp
further reading:
https://www.youtube.com/watch?v=LfMBHPp_l5k&feature=youtu.be - the tutorial video that I have made to accompany this document.
https://futurestud.io/blog/npm-quick-tips-2-use-shortcuts-to-install-packages
https://github.com/gulpjs/gulp/blob/master/docs/getting-started.md
https://css-tricks.com/gulp-for-beginners/
https://www.youtube.com/watch?v=mCW3Q28QXIs&list=PLRCvSNiMyEmyBsu6nGxB5LtMdAltgNDX5&index=9
(the final link is for a relatively recent three part video series that I have just watched. It talks you through the basics before the third part which describes a preferred setup, which I don’t use). Gulp, like Grunt is really powerful with huge amounts of plugins available and for every developer there seems to be a different set of plugins. I’ve limited myself to the plugins that I think are necessary to produce good, well formatted and useful code. You will notice that I concatenate and minify my files using a Sass header file with imports and globals and by using gulp-uglify. The other thing that I find useful is to use notify-osd on linux to interface with the various plugins that issue warnings. That way I can see when an error has occurred without having to continuously check my console. One has to install gulp-notify and notify-osd for this to work.
https://gist.github.com/millerthegorilla/3c9249941eff1b6c43197e95c5f096e7