Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
requiring npm modules in the browser console

demo gif

The final result: require() any module on npm in your browser console with browserify

This article is written to explain how the above gif works in the chrome (and other) browser consoles. A quick disclaimer: this whole thing is a huge hack, it shouldn't be used for anything seriously, and there are probably much better ways of accomplishing the same.

Update: There are much better ways of accomplishing the same, and the script has been updated to use a much simpler method pulling directly from browserify-cdn. See this thread for details: https://github.com/mathisonian/requirify/issues/5

inspiration

My inspiration for building this was Max Ogden's Requirebin, which allows users to use a browser based editor to run custom javascript in the browser (including javascript that had require() statements that would normally need to be pre-processed using browserify).

requirebin

Requirebin is a great site, but I found myself wondering if it was possible to do something like this with an interactive REPL.

digging in

After browsing the source code for requirebin, it became apparent that the key to making it work was a project called browserify-cdn, an HTTP-based service which given a URL, will return in the response body a string of javascript representing a browserified bundle of an npm library.

That is, given a URL like

/standalone/lodash@latest

browserify-cdn will bundle up lodash and all of its dependencies in the same way that browserify would if we were doing require('lodash') in a client side project. The browserify-cdn folks are even nice enough to host a version of this at http://wzrd.in/, so if you visit http://wzrd.in/standalone/lodash@latest you can see the exact output of this project.

requiring() against browserify-cdn

At some point we are going to have to define a request() function for the browser. Let's skip over the details of this for right now, and assume it looks something like

var request = require('superagent');
var require = function (moduleName) {
    request("http://wzrd.in/standalone/" + moduleName + "@latest", function(error, res, body) {
        if(error) {
            return console.log(error);
        }
        eval(body);
    });
};

this includes the client-side request library superagent for brevity, but the gist of it is that this fetches the browserified bundle for whatever library we just requested, and evaluates it.

the problem with scope

Unfortunately, browserify bundles are very nice and friendly, and everything is kept in a reasonable scope. Our code in the browser console doesn't really have access to anything happening inside the browserify bundle. So how can we get around this?

One easy way would be to take things from inside the browserify bundle scope and put them outside, like onto the global window scope.

[remember that disclaimer?]

But don't we need access to the inside of a browserify bundle at some point? It turns out, not necessarily, especially given the fact that anyone has the ability to publish any sort of contrived npm module that they can dream of.

It would be really great if for any module that we wanted to require() in the browser, there was another module that all it did was require our desired library and attach it to the window object.

Something like

window = window || {};
window.lodash = require('lodash');

or more generally (using underscore templating syntax)

window = window || {};
window.<%= moduleName %> = require('<%= moduleName %>');

but we don't always want the names to be the same (it's often the case that we want something like var _ = require('lodash')), so we can add one more variable

window = window || {};
window.<%= name %> = require('<%= moduleName %>');

autopublishing modules

It seemed possible to programmatically publish modules like that, so I built a library to do it. Requirify is a simple npm module that can be used like this

var requirify = require('requireify');

requirify('_', 'lodash', function(err, moduleName) {
    // moduleName is requirify-_-lodash, this
    // module is now published on npm.
})

and it publishes modules of the form described above to npm for you. I set up a little bot to handle all of these, and made sure to namespace them with requirify- in front so that they aren't polluting names that other people would want to use.

So, for example https://www.npmjs.org/package/requirify-_-underscore is just a 2-line npm module:

window = window || {};
window._ = require('underscore');

And since these modules exist on npm, they also exist on browserify-cdn!

a proxy server

The remaining thing to do is to make sure that the publishing step happens before we make a request to browserify-cdn. One solution for this is to create a proxy server that runs requirify on the module before proxying the request to browserify-cdn. This is fairly straigtforward and the entire implementation can be seen here:

/*!
 * Module dependencies.
 */
var requirify = require('requirify');
var request = require('request');


exports.index = function (req, res) {

    var moduleName = req.params.modulename;
    var varName = req.params.varname;

    requirify(varName, moduleName, function(err, requrifyModuleName) {
        if(err) {
            return console.log(err);
        }

        var newurl = 'http://wzrd.in/standalone/' + requrifyModuleName + '@latest';
        request(newurl).pipe(res);

    });
};

a typical express route, taken from https://github.com/mathisonian/requirify-web/blob/master/app/controllers/home.js. The server is up and running on heroku currently. It checks if the requirify-X-Y module already exists, and if not it will publish a new one before continuing on to proxy your request.

injecting the require() function

And finally, we have to inject the require() function into the browser. There is a script hosted on amazon s3 that properly defines the function: https://s3.amazonaws.com/s3.mathisonian.com/javascripts/requirify-browser.js.

This script can be injected into the body of web pages using a chrome extension (https://chrome.google.com/webstore/detail/requirify/gajpkncnknlljkhblhllcnnfjpbcmebm) or a simple javascript boomarklet.

The final require() function looks like

var request = require('superagent');
var require = function(name, moduleName) {
    moduleName || (moduleName=name);
    console.log("Fetching " + moduleName + "... just one second");
    request("https://evening-chamber-1845.herokuapp.com/" + name + "/" + moduleName, function(er, res, body) {
        if(error) {
            return console.log(error);
        }
        var r = eval(body);
        console.log("Finished getting " + moduleName);
    });
};

so you have the ability to require('_', 'lodash'). This will fetch lodash from npm and assign it to the _ variable.

update

The script has been updated to interact directly with browserify-cdn, and now looks like this:

var request = require('browser-request');

window.require = function(name, moduleName) {
    _require = require;

    if(!moduleName) {
        moduleName = name;
    }

    console.log('Fetching ' + moduleName + '... just one second');
    request('http://wzrd.in/bundle/' + moduleName + '@latest/', function(er, res, body) {

        require = null;
        eval(body);
        window[name] = require(moduleName);
        require = _require;
        console.log('Finished getting ' + moduleName);
    });
};

wong2 commented Aug 31, 2014

nice

awesome

mulderp commented Aug 31, 2014

great ideas! but why not simple use the "bundle" version from http://wzrd.in/ ?

For example: <script src="http://wzrd.in/bundle/backbone@latest"></script>

And then, simply: Backbone = require('Backbone') ?

CYBAI commented Aug 31, 2014

awesome!

fiatjaf commented Sep 1, 2014

Ok, I give up, I couldn't manage to successfully turn this into a bookmarklet. Can anyone help me?

moperacz commented Sep 1, 2014

awesome!

Owner

mathisonian commented Sep 1, 2014

@fiatjaf

javascript:(function(){var body=document.getElementsByTagName('body')[0];var script=document.createElement('script');script.type='text/javascript';script.src='https://s3.amazonaws.com/s3.mathisonian.com/javascripts/requirify-browser.js';body.appendChild(script);})();"

is a bookmarklet containing the following code:

   var body= document.getElementsByTagName('body')[0];
   var script= document.createElement('script');
   script.type= 'text/javascript';
   script.src= 'https://s3.amazonaws.com/s3.mathisonian.com/javascripts/requirify-browser.js';
   body.appendChild(script);

bingo! 💯

Owner

mathisonian commented Sep 1, 2014

@mulderp

I think you could do that but, but if you are using a custom require method to go and fetch the code from wzrd.in, the bundled version will have its own require method that will overwrite your custom one.

@mathisonian

I think more useful then a bookmarklet could be to publish this as chrome extension with ability to turn on/off.

unbug commented Sep 1, 2014

Super great!

jsw0528 commented Sep 1, 2014

Cooooooooooooooo~

nice!!! thanks

Great!

👍

NecoleYu commented Sep 2, 2014

awesome!!!

nitinja commented Sep 2, 2014

This is excellent .. as chrome extension, highly convenient.

wayou commented Sep 2, 2014

cool~..😄

i can't use it in the chrome dev tool.
->require
->ReferenceError: require is not defined

my os is OSX 10.9.4
chrome is 37.0.2062.94
thanks.

if i run it in the github, the msg is like below.

->Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self' render.github.com render.githubusercontent.com https://gist-assets.github.com *.google-analytics.com https://collector-cdn.github.com".
requirify-browser.js:1(anonymous function) requirify-browser.js:1a requirify-browser.js:1t requirify-browser.js:1
->Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self' render.github.com render.githubusercontent.com https://gist-assets.github.com *.google-analytics.com https://collector-cdn.github.com".

I don't get what this scoping issue is about. When I load a standalone script directly from browserify-cdn, like

<script src="http://wzrd.in/standalone/lodash@latest"></script>

the module is available on window.lodash.

So I don't understand the need for auto publishing modules, a proxy server, etc etc. All you have to do is write a function requirify(moduleName, cb) which injects this script tag in the web page, waits until the script is loaded, optionally moves the module to a custom namespace (like from window.lodash to window._), and invokes the callback function when ready.

Am I missing something obvious?

Owner

mathisonian commented Sep 4, 2014

Yeah @josdejong, there were better ways to do this, this was the first way I figured out how to make it work. I've updated this post and script with simpler code pulling directly from wzrd.in.

kolodny commented Sep 23, 2014

wzrd.in doesn't work on https domains so if you try to use this when under a https page it fails with a mixed content warning

it was broken now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment