Create a gist now

Instantly share code, notes, and snippets.

browserify for webpack users

browserify for webpack users

There's been a strange explosion in misinformation about browserify recently, particularly in comparisons to webpack.

Generally speaking, most of this confusion stems from how webpack is more willing to pull features into its core to ease discoverability while browserify is more likely to push features out to userland instead.

I think that longer-term, separability has more benefits from a maintenance and experimentation perspective, but that doesn't mean that you can't do pretty much everything webpack can do in the browserify ecosystem. It just means that there might be several different ways of doing that task to serve different use-cases. Diversity is good!. Diversity means that you can choose an approach that more closely matches the trade-offs that you need, but you might need to put in more work to evaluate the different options.

Hopefully this document will help somewhat with finding options to evaluate.

bundle splitting

Webpack, to its credit, provided some useful ideas about how to handle splitting bundles up into multiple pages automatically. However, this idea was quickly taken up in browserify using the plugin API into factor-bundle.

If you want to, you can still split up bundles using -x and -r, but for creating multi-page bundles, it's pretty easy to automate that away with factor-bundle in a very similar way. In both cases for browserify and webpack you will end up with 2 <script> tags you can include on each page.

To split up common assets from entry points x.js and y.js into bundles bundle/x.js and bundle/y.js with a common bundle bundle/common.js, in webpack you can edit the webpack config with:

var webpack = require('webpack');
module.exports = {
    entry: {
        x: "./x.js",
        y: "./y.js",
    },
    output: {
        path: "bundle",
        filename: "[name].js"
    },
    plugins: [
        new webpack.optimize.CommonsChunkPlugin("common.js")
    ]
};

or in browserify with factor-bundle you can just do:

browserify x.js y.js -p [ factor-bundle -o bundle/x.js -o y.js ] \
  -o bundle/common.js

or from the API, that would be:

var browserify = require('browserify');
var fs = require('fs');

var files = [ 'x.js', 'y.js' ];
var b = browserify(files);
b.plugin('factor-bundle', { outputs: [ 'bundle/x.js', 'bundle/y.js' ] });
b.bundle().pipe(fs.createWriteStream('bundle/common.js'));

To load bundles asynchronously you can easily whip something up with xhr and eval() or for a more automatic option there's partition bundle.

stdout by default

This is a small thing, but writing to stdout by default can be very handy.

Instead of how in webpack you do:

webpack main.js bundle.js

In browserify you can also write to a file without using the shell:

browserify main.js -o bundle.js

but if you don't specify -o, output goes to stdout. This is a very handy feature! It means that for example you can pipe to a minifier like uglify in a very simple and straightforward way:

browserify main.js | uglifyjs -cm > bundle.js

This way, browserify doesn't need to include uglify by default, unlike webpack. Users are free to use different minifiers and will not be beholden to whatever version of uglilify that browserify ships with.

Plus you can make fancier pipelines without mucking about reading browserify core docs. Want to make both a minified and a minified+gzipped version?

browserify main.js | uglifyjs -cm | tee bundle.js | gzip > bundle.js.gz

These tricks work with watchify too:

watchify main.js -o 'uglifyjs -cm | tee bundle.js | gzip > bundle.js.gz' -dv

and factor-bundle:

browserify files/*.js \
    -p [ ../ -o 'uglifyjs -cm | tee bundle/`basename $FILE` | gzip > bundle/`basename $FILE`.gz' ] \
    | uglifyjs -cm | tee bundle/common.js | gzip > bundle/common.js.gz

Or you can use the API directly.

transforms instead of loaders

Webpack uses "loaders" to preprocess files, while browserify uses transforms.

In webpack you might do:

{
    module: {
        loaders: [
            { test: /\.coffee$/, loader: "coffee-loader" }
        ]
    }
}

but this configuration applies globally across an entire project. This is considered rather risky in browserify land and reserved for special occasions. In browserify, you would do:

browserify -t coffeeify main.coffee > bundle.js

but this transform would only apply to local files, not to modules installed with npm and placed in node_modules/. This is because those modules were written by other folks with different opinions about how to structure their projects. In browserify-land, decisions about how to transform source code are made very locally at the package level, whereas loaders in webpack are global.

Additionally in browserify-land, any package may have a package.json that specifies special transforms to use. These transforms are applied automatically and transparently with the reasoning that the people who wrote the code for those packages know best what transformations are necessary to make the code work. Webpack takes the opposite opinion: that application developers should be responsible for all the loader transformations across the entire dependency graph. I don't think this idea will pan out at scale the way that it already has in the browserify ecosystem.

Webpack loaders also care about file extensions to an unusual degree and are trying to make a central registry of file extensions to loaders. In browserify, file extensions are handled at the individual application and transform level because transformations might apply to the same file extensions in different contexts: one to handle inlining static assets, another to expand file globs.

Handily, transforms can also be configured from both package.json, the api, and the command-line.

inline source maps

In webpack you can do:

webpack --devtool inline-source-map main.js bundle.js

whereas in browserify you would do:

browserify --debug main.js > bundle.js

Inline source maps are the default in browserify because they require no additional configuration about how or where to serve up extra static assets.

external source maps

In webpack, separate source maps appear with:

webpack --devtool eval main.js bundle.js

which implicitly creates an external source map file that you will need to configure to be serve from the appropriate web root.

This can be done in browserify with an external tool, exorcist:

browserify --debug main.js | exorcist bundle.js.map > bundle.js

This creates a file bundle.js.map that bundle.js will point at as an external source map.

AMD

In webpack, AMD "just works" when the files that webpack needs are hosted in the proper directory.

In browserify, you can obtain AMD with deamdify or browserify-shim.

fast watching

In webpack lingo this is confusingly called "incremental compilation" and is built into webpack core with:

webpack --watch main.js bundle.js

In browserify, you can just do:

watchify main.js -o bundle.js -v

to watch and also print out a message when a build is complete. Unless something is very misconfigured, rebuilding with watchify after the first build should take a few hundred milliseconds at most, similar to webpack.

css

So many times people keep claiming that browserify "doesn't care about css". This is true, but very misleading. Browserify itself does not concern itself with css, but has plenty of extensibility points for different approaches toward modular css to bloom.

Webpack has one solution: put everything into the webpack asset graph. browserify has many approaches. I think that many different approaches map much better to the diversity of requirements that people have for building modular components.

css transforms

First of all, there are lots of css transforms. Some of them are generic like brfs that you can combine with other libraries like insert-css:

var fs = require('fs');
var insertcss = require('insert-css');
insertcss(fs.readFileSync(__dirname + '/style.css', 'utf8'));

and then just compile with brfs:

browserify -t brfs main.js > bundle.js

or you can use format-specific css preprocessors such as lessify:

require('./whatever.less');

and then:

browserify -t lessify main.js > bundle.js

parcelify

parcelify is a very overlooked but powerful answer to handling css in a modular way. parcelify uses browserify to build a dependency graph and automatically reads package.json files as necessary to concatenate all the necessary css.

To make a package with css assets, just put:

{
  "name" : "my-module",
  "version": "1.5.0",
  "style" : "*.css"
}

into the package.json for the module.

Now when you require('my-module') elsewhere starting from an entry point main.js and compile with browserify like usual:

browserify main.js > bundle.js

you'll also be able to run:

parcelify main.js -c bundle.css

to compile your css all in one big bundle!

There are options to apply css transforms and do file watching too.

npm-css

You can also use npm-css to have @import for css files similarly to require() in javascript code:

@import "typeahead";
@import "./foo.css";

.foo {
  color: red
}

which will resolve typeahead from node_modules and foo.css from the same directory as the css file just like how require() behaves.

npm-css doesn't actually have anything to do with browserify at all! It just so happens to pair nicely for handling css. Instead of a main field in package.json, use a style field to specify a css entry point.

atomify

atomify provides a slightly more opinionated solution, insofar as it's possible to do that in the browserify ecosystem, for handling both javascript and css. It can resolve @import statements with the node require algorithm and uses rework behind the scenes to provide prefixes, inline images, and variables.

points of divergence

browserify and webpack disagree one some fundamentals. browserify prefers:

  • more local decisions for better composition
  • node and npm compatability
  • userland experimentation and diversity instead of a large core

composability

Browserify takes great pains to compose well at scale across many packages from many different authors. To do this, it's very important to make decisions locally instead of globally. When people mouth off that something like webpack is "more powerful" than browserify, first of all that doesn't mean anything. Second of all, don't assume that power is always good. The bigger a project gets and the deeper its dependency graph, the more important structuring effects at scale becomes. When transforms can operate globally over the entire graph, it's hard to predict where those transformations could fail or interfere with existing code. It's still possible to achieve these kinds of global actions with plugins and global transforms because occasionally nothing else will do, but ordinary transforms with narrower effect are much more highly encouraged.

overloading require

webpack encorages overloading require() to mean something very different from what it means in node:

require("coffee!./cup.coffee");

This will make it very hard to reuse your code in anything that is not webpack, particularly node.

browserify will never encourage this, opting for simple node-style require() and node compatability where possible

node and npm compatability

browserify puts in a lot of tiny, subtle features so that modules from npm with mostly "just work". The module lookup algorithm is exactly the same, including tiny things like the order of operations and default extensions like .json. Node core modules that make sense in a browser context like path and util also just work without any special configuration.

According to some sophisticated static analysis done by airportyh, around half of the packages on npm are likely to just work with browserify. You can search this subset of packages on browserifysearch.org.

in closing

Be wary of bullet-point engineering.

Comparing technologies strictly on what checks all the boxes might seem like it will solve your problems faster, but lumping too many features together is a recipe for scope creep and inflexibility later on.

read more

@ericelliott

OK. I'm sold on Browserify (still). I was already aware of most of the arguments here, but I was unaware that Webpack didn't share a lot of these characteristics. I'm not willing to sacrifice Browserify's flexibility for Webpack.

What attracted me to Webpack was that it has less overhead, and I heard a rumor it has better performance (faster builds?).

Is there any truth to those perf concerns (particularly overhead, I'm always interested in smaller bundles)?

I was also attracted to easily breaking the bundles into smaller pieces, as I've discussed with you before in person, but it sounds like Browserify has a good answer to that I wasn't aware of!

Thank you very much for this writeup. I learned some things I didn't know about Browserify's ecosystem, and learned some of the tradeoffs I would have had to make to switch to Webpack -- tradeoffs I would not knowingly make.

@briandipalma

Often, if you want to structure effects at scale, you'll need to

Transforms are very carefully designed not powerful

Typos?

@oren
oren commented Jan 2, 2015

There is a small mistake in the factor-bundle example. it should be

browserify x.js y.js -p [ factor-bundle -o bundle/x.js -o bundle/y.js ] \
  -o bundle/common.js
@oncletom
oncletom commented Jan 2, 2015

Thanks for this write-up!

Webpack made my life more complicated when it was about reusability outside of the Webpack-world.

Browserify (factor-bundle and the CSS modules you mentioned) remains a perfect toolchain, especially for React, as both the user-interface and its layout can be packaged, distributed, reused and assembled wherever they need to.

Thanks! (and I'm happy to share a bit of my daytime-job-in-a-company-income on Gratipay ;-))

@fantasyni

for application level codes, always building up a bundle brings up a lot of pains and bad development experiences
browserify with asynchronous script loading​ will be a approach to resolve this pain

@gaearon
gaearon commented Jan 2, 2015

It's misleading to say "Webpack brings in all those features and Browserify leaves them in userland".

Even builtin Webpack features are implemented as plugins internally and you can substitute them with your own. Its core is thin.

@ColCh
ColCh commented Jan 2, 2015

Overloading require brings to us more flexibility - we can abstract on project's file structure utlizing aliases. We can abstract even on file types - everything becomes a module! It can be used on serverside too.

Yes, webpack uses loaders on modules from node_modules... it can be dangerous, but I don't faced any error using it...

My short opinion on webpack vs browserify: webpack is simpler to learn than browserify. . Approximately, they do same thing (building bundle, he he)

Thanks for this article; I think you should have a better look on the webpack, it deserves this.

@gaearon
gaearon commented Jan 2, 2015

Code splitting is fine but it's a bit useless in single-page apps without async chunk loading.
Is there a way to replicate Webpack's async chunk splitting in Browserify?

Edit: Yay, it seems possible! https://github.com/substack/browserify-handbook/blob/master/readme.markdown#partition-bundle

@kellyrmilligan

Maybe I'm wrong, but I don't believe browserify-shim or amdify provide async module loading at run time. If you want to load a module at run time I don't think this exists he way it does in web pack. There is the article linked to above so there may be a solution now.

@nmn
nmn commented Jan 2, 2015

The one important feature of loaders over transforms is that you can apply them to only a few files, while transforms apply to all local files.

@gaearon
gaearon commented Jan 2, 2015

this configuration applies globally across an entire project. [...] in Browserify, [...] this transform would only apply to local files, not to modules installed with npm and placed in node_modules/. This is because those modules were written by other folks with different opinions about how to structure their projects. In browserify-land, decisions about how to transform source code are made very locally at the package level, whereas loaders in webpack are global.

You can (and usually should) exclude /node_modules/ when transforming your own code:

{ test: /\.jsx?$/, loader: '6to5', exclude: /node_modules/ },

But you are also free not to. You can apply certain loaders globally, some only for your code, some for specific directories or files. Webpack gives you control over that but of course you can also shoot yourself in the foot. Loaders are not only used for language transforms; they can be used for shimming, wrapping, specifying asyc split points and even running Browserify transforms. Webpack thinks that consumer, not library author, should be able to control this.

Where Webpack is lacking is documentation about best practices. Your truth is that it's too flexible and doesn't warn you against stupid things enough. But it also gives you full control over what happens. You might want this control or not, but it's not some inherent fault of Webpack loaders.

As for your second point, I agree with Webpack's stance that one should not ship something that needs transformation to NPM anyway. Transform is on you as the library author, as a build step. I've made this mistake before and I don't want to complicate people's lives by shipping something that isn't runnable as is.

Webpack doesn't support per-package transforms because it inevitably leads to headaches for consumers. We're not in a perfect world where everybody uses one build tool, and Webpack chooses pragmatic approach over idealistic one. You can find similar decisions all over Webpack (e.g. read Motivation).

Inline source maps are the default in browserify because they require no additional configuration about how or where to serve up extra static assets.

Webpack optimizes for speed, I think that's why source maps are not on by default.
For development, you can use devtool: 'eval' as it gives much better performance.

In webpack, separate source maps appear with:
webpack --devtool eval main.js bundle.js
which implicitly creates an external source map file that you will need to configure to be serve from the appropriate web root.

What do you mean? I never needed to separately configure source maps, but maybe I'm missing your point.
Also devtool: 'eval' is not what you use for source maps. Did you mean devtool: 'source-map'?

In webpack, AMD "just works" when the files that webpack needs are hosted in the proper directory.

I'm not sure what files you are referring to at all. Webpack doesn't need any separately hosted files, especially to support a module format. Everything Webpack ever needs is in the bundle it compiles.

webpack encorages overloading require() to mean something very different from what it means in node:

require("coffee!./cup.coffee");

This will make it very hard to reuse your code in anything that is not webpack, particularly node.

Arguably it's obvious you shouldn't do that if you want to run on Node. Instead, specify loaders in configuration file. But this is non-issue from the start because 99% of browser apps don't run on node (and if they do, as I said, use config instead), and shared code (libraries) shouldn't expose "raw" precompiled source anyway (see my point about loaders above).

This way, browserify doesn't need to include uglify by default, unlike webpack. Users are free to use different minifiers and will not be beholden to whatever version of uglilify that browserify ships with.

But Webpack doesn't need to include Uglify plugin by default either. It does so for convenience (that's what you're against philosophically). Nevertheless Webpack's Uglify plugin is implemented as a plugin internally, just like most Webpack features.

You can copy-paste it into NPM package of its own if you want to and have full control over Uglify version you use. But for some reason nobody did that—maybe because nobody cares enough about managing Uglify versions.


As for “bullet checklists”, yes, they are important if we talk about innovation and not just incremental improvement. There are important features where Browserify ecosystem is lagging behind Webpack core. Painting it as “Browserify is modular, Webpack is not” is misleading.

I'd love to implement React hot reloading for Browserify. I even extracted Webpack-independent part into React Hot API. But I couldn't find a way to do it—there's just no mechanism like HMR in Browserify. It's a feature I can't live without and I don't appreciate you downplaying its significance as a gimmick worth giving up for a “more modular” tool.

We can argue ad nauseam about Bazaar vs Cathedral, Unix vs kitchen sink, etc, but this only makes sense when there is feature parity. Right now practical concerns are more important to me than philosophical ones.


Finally, “webpack is not modular” is just not true. Take a look at its source. Its core is small, and most features are plugins using the same plugin API that Webpack exposes for third-party developers. Yes, they are included for convenience, but they are not special in any way.

@xjamundx
xjamundx commented Jan 2, 2015

(With regard to the subthread about async loading)

I tried async loading by using browserify + require.js and it blew up in my face pretty badly (for fairly complex reasons, but you can imagine they both want to do different things with require()), eventually we got it to work using http://labjs.com/ which is similar to the loadjs solution above, but it required being very explicit about splitting and configuration and didn't feel perfectly awesome. We're still using r.js but hope to migrate away to an NPM friendly solution soonish. Really love browserify and webpack. <3 <3 <3

@kellyrmilligan @gaearon

@xjamundx
xjamundx commented Jan 2, 2015

For those that are pro bullet-points, putting something together (more comprehensive than this) at the top would probably be really helpful for everyone:

Webpack Feature Browserify Equiv.
bundle splitting -x and -r
coffee! coffeeify
AMD support deamdify
require('css!') insert-css
ES6 support es6ify
JSX support reactify
async require lab.js + bundle splitting
@petehunt
petehunt commented Jan 2, 2015

I like the idea of this gist in theory because sometimes webpack users get a little overzealous (like ignoring the fact that watchify exists). However I think you've countered some misinformation with more misinformation.

First of all, browserify transforms are just another way of overloading node's require() which will often break when running in straight Node (exact same tradeoffs as webpack's bundle-loader). With all this transpiling and the move to static assets in the dep graph it's inevitable that eventually we'll have something equivalent to a bundle step before running code in node (in fact it's already being done at several companies).

@gaearon covered a lot of it, but one other thing to note is that factor-bundle is not the same thing as webpack's code splitting. webpack includes an optimizer that understands how to split n modules into m bundles and (more importantly) is abstracted away from me as the end user, giving them the freedom to improve that optimization as they see fit. Unless something has changed since I looked at it, with the browserify suite that optimization responsibility is pushed to me, the end user. This is hugely important to me when I have a bunch of modules and impending deadlines.

Another thing. This "modularity shaming" thing is getting old and feels disingenuous. If you look at the webpack code (or another project that gets this a lot, React) the code is clearly organized in a reusable, pluggable, small-sharp-tools way. The only difference between this approach and the one you advocate is the commonly used modules are versioned together rather than separately. npm/semver is good, but it's still less friction to do it this way and there's very few downsides to it, and I don't think it's a good reason to discount entire projects.

As someone who's been on the receiving end of modularity shaming a few times now, it feels like there's an in-crowd that uses the modularity excuse to discount the work of others because the word "modular" is so nebulous you can use it to mean anything (kind of like how "culture fit" is used to exclude people for reasons outside of job interview performance). Normally I wouldn't care about this but there are tools that solve real problems very well that don't get widespread adoption because of this type of misinformation, so it's damaging to the open-source community.

With that said, browserify is a hugely important tool in the ecosystem and the fact that it started rallying everyone behind CommonJS/npm as the one true way can't be overemphasized. I just think it isn't the right tool for the job for larger applications (mainly due to its emphasis on async loading and abstracted away optimization of bundles), at least when I last evaluated it.

@Raynos
Raynos commented Jan 2, 2015

The only difference between this approach and the one you advocate is the commonly used modules are versioned together rather than separately.

Not at all. The difference with the approach we advocate is that our modules are designed in isolation, documented in isolation and can be used in isolation.

I can't speak for webpack, but the internal modules in React are unusable as standalone units because of interface coupling and general lack of documentation.

That being said there are exceptions. There are a lot of useful modules hidden in require('react/utils/x') ( https://github.com/facebook/react/tree/master/src/utils ). It still suffers from a lack of documentation though.

@petehunt
petehunt commented Jan 2, 2015

Then why don't you call it what it is, a documentation problem? Modularity implies a problem with the software architecture, which is why I called it disingenuous.

Also, this guy figured out how to use the internal modules in React...

@xjamundx
xjamundx commented Jan 2, 2015

Hey get a room!

@substack
Owner
substack commented Jan 2, 2015

@petehunt there are some huge problems with the batteries included approach that webpack takes as I've seen it play out elsewhere:

  • semver means less - if the interface to a plugin is upgraded in a backwards-incompatible way, the major version for the whole project should be updated. Perhaps your users weren't using that feature? Perhaps they need an older feature. Everything in the curated plugins is in lock-step with the main project.
  • unfair competition - it's much harder for userland alternatives to flourish when there's a defacto plugin sitting in the project already. This is bad because
  • bitrot risk - because plugins are not the primary focus of tool developers and because making breaking changes is very difficult due to the semver issues, they often become neglected over time. Just look at python or ruby core distributions for some great examples of batteries-included bitrot.

These are architectural problems in addition to documentation problems and social problems. Modularity doesn't live in isolation with those other characteristics. Convenience is not without huge pitfalls elsewhere.

@shama
shama commented Jan 2, 2015

I think it's important to note that using Browserify does not always equal composability and node compatibility, as advertised.

Many of the client side modules on npm embed browser APIs and a lot are intended to work with Browserify and/or a specific Browserify transform (a lot of times as a hidden peer dependency).

So, IMO, the advertised composability only works when everyone uses browserify. So my advice, being someone who continues to heavily use both is: Are all your modules node/browserify compliant? If yes use Browserify, if not use webpack.

The two really don't need to compete; they solve different problems. I shouldn't have to feel like a node heretic because I couldn't get Browserify to play nice with Ember. They're both simple and complex libraries.

@gaearon
gaearon commented Jan 2, 2015

@substack The kind of plugins included in Webpack is the kind unlikely to have breaking changes unrelated to breaking changes in Webpack itself. How do you break AMD support plugin or Uglify plugin? What new major incompatible features could they bring?

@jesseskinner

It's important for project developers to have options and to pick the right tool for the job.

For many projects, having a single tool for building and transpiling greatly simplifies things. There is less to learn, and no need to curate a suite of tools for every project (although webpack does allow you to use whatever loaders and plugins you want). Having a single JSON config also makes understanding at-a-glance and maintaing the build process easier.

I was considering using Browserify and Gulp instead of RequireJS and Grunt, but webpack seems to eliminate the need for all of these tools and more, at least on projects where the build process is completely composed of bundling and processing dependencies to be used on a production web site.

That said, I think there will always be room for Browserify, for developers that need to have more fine-grained control over their build process, or who need compatibility with other pipe-based command line tools.

Some people want a garage full of tools, other people can get by with a swiss army knife in their pocket. Whatever floats your boat.

@filipedeschamps

@substack keep up with the amazing work.

I think the most important thing and at the same time more difficult for people to understand or accept the importance is the competition between the core and the userland.

Userland need to be part of the solution. But for this to really get traction it needs to be explicit. This culture must be in the DNA of all involved parts and not only be technically possible to do.

As investments, the risk being higher, the return can be compulsorily higher. It's a completely understandable choice reduce the risks, but when we are in a scenario of accelerated creation and discovery, this can stop creating new results through fresh ideas.

@maxogden
maxogden commented Jan 7, 2015

After reading all the comments here I'm pretty convinced that none of the webpack people know what 'modular' means (they even admitted it up above). Pro-modularity people should probably put their effort into explaining what they mean by modular, e.g. expanding on things like what @raynos said above: modules are designed in isolation, documented in isolation and can be used in isolation

@bevacqua
bevacqua commented Jan 8, 2015

I think one of the most important points discussed here is the point being made by @raynos, and @maxogden, so I'll add my grain of sand. It's not just "lacking documentation", not that they're not "a single file", but that they're "sold together".

The problem is that they're delivered as a single package. That means a couple of different things if we are to consider every file in a package to be a "module".

  • versioning loses its meaning, because new versions don't necessarily translate into changes to the "module"'s API, because
  • these "modules" are typically considered implementation details unless something in the front-facing public API changes (e.g the API of the actual package), meaning that
  • these undocumented "modules" may break their behavior at any time and there wouldn't necessarily be so much as a changelog entry

When a "module" is a part of a [thing], it becomes an implementation detail of [thing] and stops being a "module". It's not documented, it's not versioned, it's not easily discoverable, it might even disappear entirely because [thing] doesn't have a need for it anymore, or maybe because the authors of [thing] decided it belongs somewhere else in [thing]'s directory structure.

And that's unreliable.

@mattdesl
mattdesl commented Jan 8, 2015

@petehunt

Also, this guy figured out how to use the internal modules in React...

Yikes, that code is a ticking time bomb. Anything that relies on a module's internals or lib/ would probably not fly in production.

@mjackson
mjackson commented Jan 8, 2015

@maxogden @raynos

none of the webpack people know what 'modular' means

modules are designed in isolation, documented in isolation and can be used in isolation

Browserify and watchify (and all the other "-ify" browserify plugins) ARE NEVER USED IN ISOLATION. They are always used with Browserify. There is 0 benefit from having these as separate npm packages. None, whatsoever. Splitting them into separate packages is not being "modular". It's being pedantic.

@substack
Owner
substack commented Jan 8, 2015

@mjackson there are a lot of benefits to keeping watchify and browserify separate which are rather general: Of course watchify is never used in isolation from browserify, it is a browserify plugin. This is true of all plugins, but that doesn't mean a tool should ship with all possible versions of its own plugins.

  • version updates to watchify to update things like versions of the file-watching machinery like chokidar don't need to update the version of browserify, cutting down spam on the browserify changelog for people who don't even use watchify (and even those who do)
  • separate issue tracking keeps bug reports more focused. Watchify's issue can focus on file-watching related features and browserify can focus on its own issues.
  • There are other ways to build a watchify-like tool, and it forces the browserify api to be general and featureful enough to support a rather difficult use-case without "cheating" by relying on internal undocumented interfaces.

There are also reasons for including features in a package, but these must be weighed against the downsides, and there are downsides!

@jbenet
jbenet commented Jan 8, 2015

Agree with @maxogden. Defining the properties of the "hyper-modular" approach is really important.

Here's three inter-related properties of small modules that have only been somewhat addressed above (afaict). This is not responding to anyone in particular, merely outlining concepts I care about:

  1. Module discovery is a search process. The amount of reuse depends highly on speed and friction of search. Trudging through orthogonal information defeats the point of modularity: to make building software a faster, and more robust process. (rule of thumb: "a great module" must be discovered in under 60 seconds.)
  2. High SNR - modules with small interfaces, standalone descriptions, and independent packaging, yield extremely high SNR for would-be users searching for precisely that tool. ("swiss army knife" vs "knife") It's not enough that a tool exists; it must be easily understood and tried. (rule of thumb: "a great module" must be sampled in under 5 minutes.)
  3. permanence - independently distributed and versioned modules (standalone) are much more likely to remain stable over long periods of time. Code embedded in larger projects tends to move (breaking links), and adapt to the changing use cases of the parent codebase. This nebulous fate breaks the trust required for module reuse. (rule of thumb: the link to "a great module" must never break.) (vendoring doesn't count: it dumps the complexity on the user)

At least for me, if these three properties aren't met by a module, I'm most likely never using it.

@balanceiskey

unfair competition - it's much harder for userland alternatives to flourish when there's a defacto plugin > sitting in the project already.

  1. I'm not sure the hyper-modularity argument supports this. Had Webpack opted to split its plugins out into separate modules, they'd still have certain favor over other, functionally equivalent modules. Grunt contrib plugins are a great example of this.
  2. The competitive landscape isn't ever affected by whether an author chooses to include or not include a given piece of plugin functionality. It's affected when the root architecture locks out or sabotages competitors, which is clearly not happening here.
@morenoh149

@jbenet SNR = Signal to Noise Ratio
For those like myself that didn't get the acronym.

@balanceiskey I disagree on the second point. If I were to see a feature already covered by the core of the project, there may be no technical reason I could not build another solution but a psychological one. For fear of stepping on someone else's shoes or causing offence. Similar reasons why a company may be hesitant to implement some software for fear that the Operating System (iOS/OSX/Android) may come along and make the product useless.

@balanceiskey

@morenoh149 - Sure, you may be less inclined to build a solution if one exists already, but equally true is that you'd be motivated to build a feature if the existing one didn't suit your needs. That's the nice thing about open source -- stepping on toes can lead to ideological diversity in software.

Note, whether or not a plugin is included with software or served external is irrelevant here. You're still subject to competition either way.

@spion
spion commented Jan 9, 2015

@balanceiskey unfair competition example:

  • es6 plugin 1 - not built in has support for destructuring which you find nice but not strictly necessary.
  • es6 plugin 2 - built in, no support for destructuring but supports everything else

Will you use plugin 1 or plugin 2?

Another example

  • lodash.map - not built in, much faster than regular map
  • Array.prototype.map - built-in, significantly slower than lodash.map

Which one are you more likely to use? Will you go out of the way to install and import lodash.map just because its faster? What if you had to install and import the built in map, would things change?

Install+import always has cost, the question is whether that cost is large enough to make a difference.

(There is another way to fight this though - by reducing install/import costs. npm already does a good job at this, but maybe it could do even more)

@balanceiskey

@spion - Fantastic examples, I'll try and respond based on what we do at the company I work for (Sprout Social, woohoo!), because we've dealt with both of these scenarios:

  1. This is tricky, but if you're saying that we might find ES6 destructuring unnecessary, we'd probably just go with Plugin 2, because there may be a speed-of-compliation benefit. That said, I can speak from using Webpack, that as soon as we decide destructuring is an imperative we'd just swap plugins. As an example, we're currently using @petehunt's jsx-loader to do JSX compilation. The jsx-loader comes with some ES6 features if you flip on the harmony flag, but it's a small set compared to something you might get with say 6to5. Until recently, I didn't know that 6to5 also does JSX compilation, so I might just swap it in to the mix one of these days to broaden our ES6 usage. I want to emphasize how little overhead we're talking about here, Webpack has done a good job of keeping its plugin architecture light, fast and mostly predictable.
  2. This one I can speak to more directly. We actually use lodash's map more readily than the natively-support map. We're not ever concerned with the library penalty of pulling in a handful of functions from lodash because in production builds we use dead code elimination via Uglify (see: http://swannodette.github.io/2015/01/06/the-false-promise-of-javascript-microlibs/). So typically we're not concerned with install and import costs, I don't think.
@spion
spion commented Jan 9, 2015

@balanceiskey what if you're building a tiny, 200-lines app? Will you still use lodash.map over the existing map? What if its a side-project? A new prototype? The relative cost becomes significant then.

I don't think that anyone is talking about the loading time or script size. It seems to me like its about the effort necessary to get a project going and the perceived number of moving parts you need to constantly juggle (e.g. webpack: 1, browserify: 5) in your head when you want to make things work.

Infact, the same thing that makes webpack more convenient (no need to install extra plugins, it "just works" out of the box) makes it slightly unfair to plugins that aren't built in (using a plugin that isn't built in is less convenient and therefore less likely to be done by most people). Its a trade-off. It may be the right trade-off, but it still is.

@balanceiskey

@spion - So I think you're right that convenience promotes the root project ahead of everything else. That can be cited as unfairness, but it's also a feature of the project. It's better for the end developer and as soon as that thing stops being useful, the developer is still free to go get what they need from elsewhere.

@RnbWd
RnbWd commented Jan 9, 2015

@mjackson

Browserify and watchify (and all the other "-ify" browserify plugins) ARE NEVER USED IN ISOLATION. They are always used with Browserify

I've used browserify transforms in the gulp ecosystem on many occasions. There's a plugin called vinyl-transform that maps these things with the vinyl-fs. I've also used internal modules within browserify for custom build systems. Just trying to make a point that these modules do have uses in isolation, and some of them will probably outlive browserify itself.

@Bockit
Bockit commented Jan 9, 2015

@RnbWd @mjackson One example from myself: I'm using https://www.npmjs.com/package/resolve to create a stylus function that lets me "require" stylus from packages in node_modules.

@joewood
joewood commented Jan 12, 2015

Interesting discussion, really around the pros and cons of the Single Responsibility Principle. Avoiding stating the obvious, but couldn't browserify use composition to offer equivalent functionality to webpack with a different, multiple package names with sacrificing modularity?

For example, a theoretical browser-pack package could compose browserify, watchify, excorcist, deamdify and other currently recommended plugins into a single deployable unit. There could be multiple packages, depending on the use-case. Each one could provide default integration with the plugins - this way offering the best of both worlds.

@nmn
nmn commented Jan 16, 2015

I wrote a blog post about this Gist, and my views on Webpack VS Browserify
http://blog.namangoel.com/browserify-vs-webpack-js-drama

@majgis
majgis commented Feb 14, 2015

I think this article, although it is about microservices, gets to the root of this argument.

Complexity is pushed onto the user. This complexity is welcome if you do want to compose your own build system, as you have the greatest flexibility. It is not welcome for those who don't have the time to invest and want to push the easy button.

Ultimately, what needs to happen is that the Browserify community needs more higher-level projects like Cartero. If webpack had been built on top of Browserify, it would have hit the ball out of the park, allowing it's composable parts to be reused in other browserify based projects. There is no reason that solution can't still be created, contributing to the browserify ecosystem with higher level targeted solutions.

@RnbWd
RnbWd commented Mar 12, 2015

I am still so confused.. way moreso than when I originally read this.

@bichotll

I tried webpack once. The configuration was a pain on the ass.
I'm very happy with bower+npm+browserify+gulp/grunt.
Anyway, I don't discard to give Webpack a second chance...

@chevex
chevex commented May 15, 2015

👍

I just gave webpack a go in a side project. It worked fine but I still prefer my usual approach with browserify and all it's glorious plugins. This reminds me of when I switched from .NET to node and going from the monolithic Visual Studio IDE to learning Sublime and Vim and installing a bunch of plugins. I can no longer go to big IDEs where everything is included. I hate that I don't have alternatives in many cases and I hate that everything is configured through the IDE's enormous pile of settings. My .vimrc is already complex enough, but at least I wrote most of it and it's not half as large as the number of settings I had to fiddle with in Webstorm or Visual Studio.

Now with Vim I get to update individual plugins more often and don't have to wait for the extremely slow Vim updates to come out. My IDE is always changing and getting updated in little chunks because my plugins get updated on their own. I suppose it all just boils down to where you'd rather spend your time. Personally I prefer to spend my time in community registries seeing what options I have, as opposed to spending my time combing through page after page of settings (sometimes only to find there isn't a setting for what I want, or what's there doesn't quite work how I expected/wanted it to). For some people, combing through settings is more comfortable than navigating through userland. I'd rather spend my time on npm and plugin READMEs with their own issue trackers, than spend my time in my webpack.config.js fiddling with loaders and completely new syntaxes that only mean something to webpack.

I also feel like issues get resolved much more quickly when I'm relying on the plugin author rather than relying on the monolithic lib author(s) to get around to addressing what seems like a small issue from their perspective. Even a "well-maintained" monolith still keeps every part of it in lock-step with the overall version, meaning I have to wait much longer most of the time to see my fix actually rolled out. I no longer use Webstorm, but I STILL have two outstanding issues from more than a year ago for two of their own home-grown plugins. I understand why; they have a lot to maintain and my issues are minor to them, but that doesn't change the fact that they aren't minor to me.

If they were well-maintained userland plugins the chances of them getting resolved would be much greater. It's also easier most of the time for me to fork a small plugin and submit a PR, fixing the issue for them. I've had much greater success getting PRs merged into userland plugins than I've _ever_ had getting fixes merged into the kitchen sink libs. It doesn't matter to me if internally everything is modular; when it's all bundled together it becomes a giant pain in my backside. Not to mention that if they don't roll my fix into their plugin, forking a plugin for my own purposes in a specific project is far more trivial than forking a giant library just for one small change. In that case I'm only merging minor changes to the plugin down into my fork for that project rather than the entire kitchen sink lib.

I was confused that my experience didn't seem to jive with all the hype so it's good to know that my concerns are backed up by the opinions expressed here by @substack.

@avesus
avesus commented Aug 12, 2015

@joewood very nice idea to bundle webpack analog. Of course, moving parts will slide one from other and the project will be complicated to maintain.

I think, that Unix philosophy is to learn all of the moving parts. Yes, it is time consuming and makes me angry. Think about it like learning SQL, Javascript, your favorite API. If you know all that internals, Browserify should be the most adequate choice for you, in pair with Gulp and npm run.

As a try, you could use Webpack to build your project's sources and give Browserify excellent pipeline to make a third-party modules bundle (with Gulp, indeed)!

@ljharb
ljharb commented Aug 28, 2015

Note: the bullet-point engineering article was deleted. Preserved here for posterity: https://web.archive.org/web/20140911092156/http://en.wikipedia.org/wiki/Bullet-point_engineering

@RnbWd
RnbWd commented Sep 24, 2015

Is there anyway we could combine some aspects of npm / browserify / webpack / babel / and gulp into a single library? There's so much overlap between these dev tools, it's redundant to use them together and each has it's own strengths / weaknesses.

Similarities: Each library creates / parses (and transforms) a tree (AST) of modules

Differences: Implementation of AST, transform/loader syntax, ecosystem

One example - we have a babelify transform for browserify AND a babel loader for webpack, each one essentially does the same function, calling babel which (potentially) uses it's own AST tree parallel with the build system. Furthermore, (mostly) everything relies on the npm_modules' folder structure for requiring packages, so npm is very relevant because it resolves dependencies. If npm installs a wrong library, then the build system or app will break.

Idea: Instead of reading with the filesystem, creating an AST, transforming files, saving files, then repeat... what if everything started with a shared and singular AST tree implementation? We could reduce redundancy, increase cohesion between different implementations, and guarantee consistency between different versions and builds and filesystems.

I personally prefer the streaming API of gulp / browserify, but I also the love power and simplicity of webpack. Although webpack's config file is concise, the syntax is non-standard, and the internal libraries (independent of a config) are not really compatible with the UNIX philosophy of composability that browserify is built upon. For example, I can use the internal modules of browserify individually, and compose them together however I want using the CLI, gulp, node streams, or a combination of all the above. If I wanted to use a config file, I could easily build one on top of that, package it all together and have a webpack-esc syntax running browserify under the hood.

If webpack and browserify internals shared a common streaming API for connectivity, then everything could be composed / mixed together under the hood, Put npm, babel, uglify, etc. into the mix, then we could have a common syntax for plugin management under the hood. Also, if we downloaded precompiled AST (at least for non-native modules) then it'd be easier to share 'power-combos' of front-end libraries / tools more consistently. It could be a file included with a library npm installable, like a concatenated file for browsers, but meant for 'universal build system'.

Just an idea...

@mfields106

I have been reading a lot about the difference in architectural approaches but I am dealing with a more practical problem that I have yet to find an answer to. I am currently migrating from RequireJS to Browserify and cannot seem to find away to debug properly using an IDE such as Intellj. It seems webpack has a solution for this and I cannot seem to find a way to do this with Browserify in development mode even before it is uglified. So could one of you take the time to return to a more practical yet very important topic and answer a simple question? How can you debug using Intellj source generated by Browserify allowing to create breakpoints in the original files?

@jmm
jmm commented Oct 8, 2015

@substack

There are other ways to build a watchify-like tool, and it forces the browserify api to be general and featureful enough to support a rather difficult use-case without "cheating" by relying on internal undocumented interfaces.

Hmm, not sure that it forces it to. I think it's cheating currently, isn't it? :P

It relies on internal undocumented properties _options and _expose:
https://github.com/substack/watchify/blob/d130f01954239036f1ee57360fdb9efa16a27d79/index.js#L14-L15
https://github.com/substack/watchify/blob/d130f01954239036f1ee57360fdb9efa16a27d79/index.js#L41

It also uses row.source and row.deps. There are mentions of those, without commentary, in places like the handbook and module-deps and browser-pack READMEs, but they're not properly documented.

https://github.com/substack/watchify/blob/d130f01954239036f1ee57360fdb9efa16a27d79/index.js#L43-L44

@faergeek
faergeek commented Nov 4, 2015

@substack it's sad but you should probably "enforce" some convention to require styles, it's crappy and too diverse for now.

It's crappy because there are a tons of plugins for that and most of them do too much (preprocessing, postprocessing, injecting, writing to disk, each of them should be a separate transform, the unix-way).

I think you should build an ecosystem around browserify and point some conventions to follow.

Also I would like to have a feature similar to webpack's test and exclude for loaders, it's implemented in any plugin out there right now, but I think it should be built into browserify somehow.

You should try to use webpack a bit for that probably, learn something.
Or if you don't want to I can help you with it.

webpack has stylus-loader, sass-loader, css-loader, postcss-loader, ExtractTextPlugin and they're easily composable in that sense.

What do you think?

P.S. Right now, I'm trying to migrate from webpack to browserify, it's all great and very powerful, but requiring styles doesn't work the way it works "out of the box" in webpack (css-loader provided right convention for that thing).

@faergeek
faergeek commented Nov 7, 2015

I started implementing somewhat like css-loader alternative for browserify in cssify transform
davidguttman/cssify#25

Disk writing probably requires a separate plugin like extract-text-webpack-plugin for webpack.

The general idea is that cssify will have support for css modules and be composable with other plugins.
But to support pre/post-processing css in a composable way we need a convention for plugins.
They should not wrap processed output to a module and output just raw css for other transforms to consume.

Here are issues for sassify and stylusify.
See davidguttman/sassify#20 and asbjornenge/stylusify#2

If anyone here is interested in these, please contribute or open issues for tools like lessify or something for postcss probably.

Here's a hypothetical example:

browserify \
  -t babelify \
  -t [ stylusify --inline-images ] \
  -t [ postcssify --test '/\.(css|styl)$/' ] \
  -t [ cssify --test '/\.(css|styl)$/' --modules ] \
  entry.js

Also would be cool to have a support for --test and --exclude options builtin (like in webpack) or something to pipe certain matched files through a pipeline of transforms without matching in every transform.

Here's more complex hypothetical example with separate pipeline:

browserify \
  -t babelify \
  -p [ \
    extractify \
      --test '/\.(css|styl)$/' \
      -t [ stylusify --inline-images ] \
      -t [ postcssify ] \
      -t [ cssify --modules ] \
      -o bundle.css \
  ] \
  -o bundle.js \
  entry.js

Would be good to hear some feedback.

@acacha
acacha commented Nov 7, 2015

@fantasyni I really appreciate your link: http://bearcatjs.org/2015/01/02/browserify-with-asynchronous-script-loading/ It is what I was looking for a while! I come from PHP/Laravel world and your explanation helps me a lot! Thanks!

@anko
anko commented Feb 5, 2016

Looks like the "bullet-point engineering" Wikipedia page has been removed—here it is on Wayback.

@niketaB
niketaB commented Mar 2, 2016

Hi,
Currently in ionic2, Webpack creates the js file but its still heavy and app takes time to load as webpack combines all module's js file in to one.
Does any body has tried browserify or Webpack with ionic2 ?
If yes, then please let me know which one is good and what to do configure it the way to load app faster?

@calidion
calidion commented Apr 1, 2016

I don't find too much difference between webpack and grunt/gulp.
why should i config with webpack not with grunt/gulp with more free options?

@liujingbreak

Can't agree more, love Browserify's world

@liujingbreak

I can't find a something in Webpack like Browserify's pipeline concept, where I can inspect and interfere the bundle making process, get dependency information and so on.

@TheLarkInn

@substack have you considered we combine efforts? I know this doc is a bit old but would be open to talking combining projects or man power. We owe alot to you and your work on browserify

@maxharris9

@substack I love browserify. It does exactly what I need and want it to do! Thank you for making an amazing tool.

@Restuta
Restuta commented Feb 23, 2017 edited

@TheLarkInn he may not not be able to see your comment since gists don't send notifications even when you mention (so you won't see my comment either :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment