Bazaarvoice has multiple consumer-facing web applications; each of these apps uses some set of vendor resources -- for example, jQuery or Backbone. It is possible for multiple applications to be on the same page; when this happens, each application brings its own vendor resources. This means that Bazaarvoice applications may end up loading the same resource multiple times on the same page, which has a negative impact on performance.
Consumer-facing web applications created by Bazaarvoice should be able to share vendor code. To achieve this, we would create the following pieces, discusssed in detail below:
- A static resources service (e.g. display.static.bazaarvoice.com)
- A client-side JS module for requesting vendor code from the service, intended for inclusion in the scout file
- A Grunt (or similar) tool for populating the service
The Static Resources Service would provide a /vendor/*.js
endpoint for requesting vendor modules. This endpoint would allow a consumer application to request multiple vendor modules, e.g. /vendor/backbone@1.1+underscore@2.4
. It would return a file containing all requested resources, exposed via the predefined callback method BV._vendor.define
.
(function () {
BV._vendor.define('underscore@2.4', [], function () {
return { /* ... */ };
});
BV._vendor.define('backbone@1.1', [ 'underscore', 'jquery' ], function (_, $) {
return { /* ... */ };
});
}());
The service would be implemented as an s3 bucket with Akamai or CloudFront CDN services. See below for details on how the service would be populated.
Applications would use a client-side JS module to request vendor modules from the service.
The module would provide an interface such as the following:
BV._vendor.require(['jquery@1.8', 'backbone@1.1', 'underscore@2.4'], function ($, Backbone, _) {
// ...
});
Given this request, the module would:
- Determine whether the modules were already available, by inspecting
window.BV._vendor
. - Create a canonical URL for requesting unavailable resources from the service, ordering the requested modules alphabetically in order to facilitate caching:
https://display.static.bazaarvoice.com/vendor/backbone@1.1+underscore@2.4.js
- Use the response from the service to call the provided callback
The module would be responsible for defining the BV._vendor.register
callback that the service uses in its response.
It may be advisable for the module to maintain a list of known-good package names, in order to prevent errors.
The service would be populated with static files generated via a Grunt-based tool (or similar). This tool would consume a configuration such as the following:
var packages = [
[ 'jquery@1.8', 'backbone@1.1', 'lodash@2.4' ],
// ...
];
In the near term, teams would contribute to this configuration (via pull request) to express the dependencies they would need. Importantly, the tool would not just build the exact combinations requested; it would also build all subsets of each combination. So, if the packages
array contained an entry for [ 'jquery@1.8', 'backbone@1.1', 'lodash@2.4' ]
, then the tool would create files for the following (always ordering the modules alphabetically):
- backbone@1.1,jquery@1.8,lodash@2.4
- backbone@1.1,jquery@1.8
- jquery@1.8,lodash@2.4
- backbone@1.1,lodash@2.4
- jquery@1.8
- backbone@1.1
- lodash@2.4
This covers the case where an application requests ['backbone@1.1', 'jquery@1.8', 'lodash@2.4']
but the loader discovers that one of the resources is already available, and thus only loads the two unavailable resources.
The tool would be responsible for maintaining a mapping of module/version to the code to be included. In most cases, this mapping would simply point to an NPM module; in some cases, it might point to a private Bazaarvoice repo containing an NPM module.
- Applications should only make one request to the service per page load.
- The request to the service should happen from the application's scout file whenever possible, in order to ensure the vendor resources arrive as soon as possible.
- Applications would be responsible for handling failure cases, timeouts, etc.
Some areas require further consideration and discussion:
- What should the TTL be for these files? We should design the system such that the TTL can be extremely long.
- How should we handle vendor files that have been modified by Bazaarvoice in order to address issues? We should determine naming and versioning strategies for these files (and avoid creating them at all when possible).
- It's likely that the service should exist in QA as well as in PROD. I don't think there's a need for a STG service but I'm open to opinions.
- This service could be used by non-consumer-facing applications as well.
I'm back and forth on whether I like this or not.
I'd think there is a good chance this approach in it's current form may only slightly curb the problem in the original statement:
when this happens, each application brings its own vendor resources.
If my app specifies
backbone@1.1,lodash@2.4
and your app specifiesbackbone@1.1.2,lodash@2.3
, there was no gain by this approach, since we'd need different versions of the same library.I don't think that's a contrived example - I have no idea what version of BB or lodash Firebird is currently using (I looked in the repo and couldn't find it easily, not that I want to have to know ).
I fear the awkwardness of "Hey curations, FB just went to lodash 2.3.5 for a bug fix, mind upgrading to that version to so we can get the performance gains back that we lost when they upgraded and stop double loading nearly identical versions of lodash" tickets popping up in our queues, and the unnecessary deployments to follow.
Perhaps it's an easy to solve that problem by introducing the
x
placeholder into the dep versions, for instance:[backbone@1.x,lodash@2.x]
or[backbone~1.x]
Given that the libraries that we'll support will version themselves in compliance with semver, it might lead to less whack-a-moleing of matching versions across projects, but an increased risk in breaking something. We could be pretty vocal about changes to
latest
on libraries ("On July 1 2015 backbone 1.x will point at 1.2.4").