Creating a front-end client application based on html5 technology will require a few toolchains working together to develop, assemble and deploy the application. Each part of the toolchain will perform a fairly small function. Each layer is decoupled from each other so you can swap out your own toolchain or augment it as needed. Standards are still evolving around html5 client application toolchains and workflow.
Since scalajs is a new technology, take a web-app first style approach to the build environment using toolchains that evolved with the html5 development technologies versus a java or jvm centric approach. The following tools are fairly standard although others exist:
- bower: Management js dependencies
- grunt: A very simple task runner. Most of what grunt does could be performed by sbt if sbt had a set of plugins that cover the same functionality. In some cases, you can decide whether to use a grunt task or a sbt task. *For example, you can use a grunt task to compile a less CSS file or a sbt plugin *Grunt is usually installed locally into the topmost directory of your application *node/npm including package.json: package.json is needed to specify the grunt and other grunt modules as a dependency for the application
- sbt: Specify how to build the scalajs portions of your application. You can also use sbt plugins such as workbench to help integrate in incremental compilation from your scala application.
Node, bower and grunt have rich ecosystems. They can be used to perform a variety of development tasks such as launching a dev webserver to test your content where remote calls are needed. sbt also includes development plugins such as workbench that can launch a dev webserver that refreshes itself when the scalajs output is updated.
The basic workflow runs in layers. Fortunately, you can build this metadata up manually, step by step. For example, installing a npm module such as grunt using the --save-dev saves automatically saves the grunt dependency in your package.json file as well as installs the dependency. As an option, you can build all your metadata up prior to running the commands and then run the commands to perform the actual install. Generally, the objective is to have an entry point into the toolchain. In this case, we will use grunt to start tasks that run other tools such as bower or sbt. However, you could just as easily use a Makefile, a shell script or other entry point. Build automation is the objective so you'll want to ensure you are creating a workflow that can be easily started for both developers and production deployment support operators.
- Create minimal metadata such as package.json and bower.json
- Use npm from node to install grunt, other grunt modules and if it is not already installed, the globally available bower tool
- Use grunt to execute tasks that tell bower to retrieve dependency modules
- Use grunt to create a dev or prod target tree by assembling content from the dependencies.
- Use sbt to compile the scalajs portion. Access the generated files in your html script tags or use grunt to manually or automatically move them to the target tree to run. You can also use the workbench plugin to start a web server that reloads content based on changes it seets. grunt also has some "watch for changes" capability.
Web technology uses plain text files for its content and the client performs file rendering and compilation of javascript for execution in the client. This makes it much easier to use plugins to manipulate the text files to obtain different configurations and variations. For example, since HTML is easily parsed and manipulated by node, you can write a grunt task that obtains a HTML template, adjusts the links to scripts, then saves the generated file to the target tree for use in dev or prod depending on different settings. Of course, there is a grunt script that takes the bower dependencies and inserts them into your html file directly.
These approaches are significantly more important in html5 technology because unlike jvm based environments where optimization is typically specified using switches in the compile step but the workflow lifecycle is mostly the same, current html5 best practices have "optimized" files are named differently then "unoptimized" versions (jquery.min.js vs jquery.js) as well as having multiple files (both a js and a sourcemap file) for dev work. While it is possible to abstract this a bit in the build process, current practices are widely used and a generative or template approach adjusts links in html5 source files directly. And since html5 is easily parsed, this practice is fairly easy if not a bit indirect. It is very reminiscent of C programming using the C preprocessor (CPP). Its clear that transformations at the source level are key abstractions used on the text-centric source code model found in modern web apps.
Here's a few steps to get started and create the scaffold for a single page application (SPA)-like web app. It focuses only on the client part. There are many tools and plugins to use to build web apps. And there are many directory hierarchies you can use to manage your source.
- Install node.js using your OS-specific approach
- Create a new folder for your application.
- Create a package.json file (see the node website for the syntax)
- Minimally specify the name and version of your project.
- This information will be slightly duplicated in the bower.json file for bower dependency management.
- Minimally specify the name and version of your project.
- Specify the grunt dependency in the package.json file or install it manually and have npm record the grunt dependency for you. grunt will be installed into the local directory.
- Option 1: Run
npm install gruntto install the dependencies directly and also manually include them in the devDependencies section of the package.json file. - Option 2: You can also create a barebones package.json and then run
npm install grunt --save-devto have npm automatically add grunt in the devDependencies section of your package.json file.
- Option 1: Run
- Include other grunt plugins using npm.
- Other dependencies you will want to include are other grunt plugins that provide tasks to be run from grunt such as automatically using bower and your bower.json to automate installing bower packages using grunt.
- You can also include other npm module dependencies.
- Install bower if it is not already installed.
- bower should be installed so that it is available as a command line program
npm install -g bower
- bower should be installed so that it is available as a command line program
- Create a bower.json file
- bower.json should have the same name and version as the package.json file. You'll need to keep these in sync yourself.
- There is also a grunt task that will keep the metadata in sync between package.json and bower.json
- Specify bower dependencies such as a dependency on bootstrap and jquery
- You can install the bower packages manually and have bower update your bower.json file directly using
bower install jquery --save. This both installs the package but also modifies the bower.json file to add the dependency.
- You can install the bower packages manually and have bower update your bower.json file directly using
- bower.json should have the same name and version as the package.json file. You'll need to keep these in sync yourself.
- Create a Gruntfile.js file to install a grunt plugin for automatically running the bower install command (see the paragraph after this list to select a grunt plugin)
- Configure a task to run bower to install dependencies. You have to look at the documentation for the specific grunt plugin you install to run bower
- This allows you to have bower install dependencies from a clean source tree after your project is downloaded from git or after the source tree has been cleaned.
- Add additional grunt tasks to create a dev or prod target tree.
- There are grunt tasks you can instal with npm to copy various portions of your application to the target tree. For example, minifying CSS files, etc.
To find grunt plugins, use npm search <search terms>. For example, to find a grunt plugin that calls out to bower to automatically install bower packages run npm search grunt bower install then review each returned entry and select one to install using npm install grunt-boer-install --save-dev. For example running npm search grunt bower install produces:
NAME DESCRIPTION AUTHOR DATE VERSION KEYWORDS
grunt-auto-install Install and update npm & bower dependencies. =manabu-gt 2014-03-13 15:08 0.1.1 gruntplugin grunt npm bower in
grunt-booty Adds bootstrap to your project from a bower install - optionally includes font-awesome =mattstyles 2013-04-22 21:44 0.2.1 gruntplugin
grunt-bower Copy bower installed components to dist folder. =curist 2014-05-06 13:31 0.13.0 gruntplugin
grunt-bower-busterjs Automagically wire-up installed Bower components into your Buster.JS config =faassen 2013-05-08 13:01 0.1.1 gruntplugin bower buster rjs
grunt-bower-clean Remove files (e.g. docs, tests, etc.) from installed bower components =karolis 2013-10-26 11:01 0.2.1 gruntplugin
grunt-bower-cli Installs Bower components using the Bower CLI. =brianreavis 2013-08-15 06:56 0.0.1 grunt gruntplugin bower
grunt-bower-concat Automatic concatenation of installed Bower components in right order. =sapegin 2014-01-22 08:27 0.2.4 gruntplugin bower component pa
grunt-bower-config Bower Installer Task =nicknisi 2013-04-29 17:26 0.2.0 gruntplugin bower
grunt-bower-copy Collect© all main filse of your Bower dependencies. =yyoud88 2013-11-11 19:11 0.0.3 gruntplugin html grunt bower p
grunt-bower-depend Handle Bower Dependencies =chris.gladd 2014-03-31 09:05 0.1.2 gruntplugin bower install depe
grunt-bower-hooks Automagically wire-up installed Bower components into your RequireJS config =sindresorhus 2013-09-06 20:46 0.3.0 gruntplugin bower requir
grunt-bower-install Inject your Bower dependencies right into your HTML from Grunt. =stephenplusplus 2014-05-01 01:02 1.4.1 gruntplugin html grunt bower p
grunt-bower-install-custom This task will add references to css/js files into the bower section from an bower.custom.json file. it will warn about modules where i
grunt-bower-install-shopware Inject your Bower dependencies right into your HTML from Grunt. =klarstil 2013-06-20 12:43 0.2.2 gruntplugin html grunt
grunt-bower-install-simple Grunt Task for Installing Bower Dependencies =rse 2014-04-20 08:32 0.9.2 gruntplugin bower install depe
grunt-bower-install-task Run bower install as a grunt task =james.talmage 2013-09-04 23:36 0.0.2 gruntplugin
grunt-bower-installdep The best Grunt plugin ever. =tiex 2013-08-22 17:48 0.1.0 gruntplugin
grunt-bower-installer Install Bower packages. =andrewdryga 2014-04-08 01:31 0.3.6 gruntplugin bower
grunt-bower-just-install Just Install Bower Components =gabrielmancini 2014-04-08 15:41 0.0.3 grunt install bower
grunt-bower-postinst Execute post install action on bower components =krampstudio 2013-08-28 11:22 0.2.1 gruntplugin bower postinst gru
grunt-bower-require-wrapper Wraps files with requireJS define() statements for modules definition using bower installed dependencies or user specified ones =raven
grunt-bower-requirejs Automagically wire-up installed Bower components into your RequireJS config =sindresorhus =paulirish =addyosmani =passy =robdodson =sboudria
grunt-bower-requirejs-alias Automagically wire-up installed Bower components into your RequireJS config =nathggns 2013-08-01 15:57 0.7.0 gruntplugin bower requ
grunt-bower-seamless-install Grunt task to install bower components seamlessly through target dependencies. =abovethewater 2014-02-27 11:29 0.1.0 gruntplugin b
grunt-bower-submodule Installs all dependencies of different bower.json's inside a project =mellors 2014-03-21 07:02 0.2.4 gruntplugin
grunt-bower-task Install Bower packages. =yatskevich 2013-11-10 11:39 0.3.4 gruntplugin bower
grunt-bower-verify > Install and test all your dependencies versions. =mokkabonna 2014-03-15 15:35 2.0.0 gruntplugin
grunt-version-copy-bower-components Version and stage Bower components for release. =hypexr 2014-04-11 05:12 0.1.8 gruntplugin bower install pack
##Installing bower Dependencies
With the scaffolding setup, you can install the bower dependencies. If you use the npm modules grunt-bower-install grunt-bower-install-simple, you will have some install tasks that can automatically download the dependencies listed in bower.json and then slice in the bower js and css links into your html by scanning a developer specified set of files for <!-- bower: js --> ... <!-- endbower --> tags and substituting a list of dependencies. The file is updated in place. But these are 2 separate operations.
First we need the grunt plugins. These are npm dependencies that need to go into package.json:
$ npm install --save-dev grunt-bower-install grunt-bower-install-simple
Then as a trick, we will modify the bower.json my manually installing the packages first. We do this because its easier than using the editor and we can ensure that they metadata is correct in bower.json.
$ bower install --save-dev jquery bootstrap crossroads js-signals hasher
bower jquery#* cached git://github.com/jquery/jquery.git#2.1.1
bower jquery#* validate 2.1.1 against git://github.com/jquery/jquery.git#*
bower bootstrap#* cached git://github.com/twbs/bootstrap.git#3.1.1
bower bootstrap#* validate 3.1.1 against git://github.com/twbs/bootstrap.git#*
bower jquery#>= 1.9.0 cached git://github.com/jquery/jquery.git#2.1.1
bower jquery#>= 1.9.0 validate 2.1.1 against git://github.com/jquery/jquery.git#>= 1.9.0
$ cat bower.json
{
"name": "playbook-client-js",
"version": "0.1.0",
"dependencies": {
"jquery": "~2.1.1"
},
"private": true,
"devDependencies": {
"jquery": "~2.1.1",
"bootstrap": "~3.1.1"
}
}
With bower.json update the newly created ./bower_components directory can be deleted because they can be easily recreated using a grunt task. Let's delete the manually obtained bower components with rm -rf ./bower_components. In our gruntfile, we have some simple config and a few aliasses for some tasks so they are easier to remember.
...
bowerInstall: {
target: {
src: ['app/views/**/*.html'],
devDependencies: true
}
},
...
grunt.loadNpmTasks('grunt-bower-install-simple')
grunt.registerTask("bower-get-dependencies", ["bower-install-simple"])
grunt.loadNpmTasks('grunt-bower-install');
grunt.registerTask("bower-inject", ["bowerInstall"])
That's it! Running grunt bower-inject or grunt bowerInstall We will get an updated index.html file:
...
<!-- bower:css -->
<link rel="stylesheet" href="../../bower_components/bootstrap/dist/css/bootstrap.css" />
<!-- endbower -->
...
<!-- bower:js -->
<script src="../../bower_components/jquery/dist/jquery.js"></script>
<script src="../../bower_components/bootstrap/dist/js/bootstrap.js"></script>
<script src="../../bower_components/platform/platform.js"></script>
<script src="../../bower_components/polymer/polymer.js"></script>
<!-- endbower -->
...If you run this command again no update is performed because there is content between the tags and bowerInstall does not change the content between the tags if content is already present.
So the grunt task bowerInstall now injects the links based on some commen tags. This is still brittle but a quick way to get started. It is even better if you use the html5 link import feature and siphon off all of your third-party scripts into an import document. This allows you to keep an unmodified core index.html document and merely generate the proper import file based on prodution or development modes. bowerInstall would merely act on the partial document itself to automatically fill in the libraries or you can manually add them. This helps accomplish some of the purpose of bowerInstall--that is--to remove the developer from having to manage the insertion of the proper set of library links in the document. In other words if you have html5 features working:
....in js/import.js.html
<!-- bower:js -->
<!-- endbower -->
...in your index.html use the "stable" html link. Underneath, grunt tasks can generate the proper dev or prod import.js.html
<link rel="import" href="js/imports.js.html"/>Directory structure for a web application is highly variable based on the type of application, the frameworks used and the scope of the application. However, there are some general design principles including:
- Keeping resources that need to be transformed to be used, such as a HTML template file, with other similar assets, such as plain html files.
- Separate out scripts, styles and markup.
- Organize directories by "view" where scripts styles and markup are grouped together by logical page. Or organize by component, such as all scripts go into one diretory hierarchy and the html markup all goes into another hierarchy. This last style was popular with java based web applications.
- Maintain a set of assets in one tree and build the target distribution (dist) tree separately. In the java world, it was common to have some assets maintained in the target tree if they were relatively static and did not need java compilation. However, with today's workflow and toolchain, the target directory should be considered blank and all content is assembled into it since many assets require some sort of transformation/compilation step.
There is a wide variety of structures although there are some loose standards. The real issue is how do you assemble a tree of content to serve up for different stages of development and production. Unlike java, there is no strict packaging and execution model--in fact---its highly variable.
More on this topic in the next blog.