Skip to content

Instantly share code, notes, and snippets.

@halfbyte
Last active April 3, 2016 17:05
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save halfbyte/1260e923c6f14e87c57f to your computer and use it in GitHub Desktop.
A draft of an Article that wanted to be written after the left-pad drama and the parallel discussion on the super-small-npm-modules approach

Preface

I like to call myself a web developer. Some people would call me "full stack" developer, but I'm absolutely terrible at writing file system, so, no.

As a web developer, since 1996, I have written code, markup and styles in HTML, CSS, Perl, PHP, JavaScript, Java, Ruby, Tcl, Python, C (yes, at one point I wrote CGI-Scripts in C), Haxe, Flash, Coffeescript and probably also a handful of other more obscure languages I can't even remember. I'm not writing this to brag (although that's a nice side effect, of course) but to make clear to you, the reader, that the following is not a rant by a disgruntled Rubby engineer who hates JavaScript.

In terms of modern JavaScript, I've spent some time, on and off, during the last, say, 6 years, in dabbling with node.js, but let me be honest with you: It was never my happy place. I wrote a lot of frontend JavaScript, though, often by using CoffeeScript as a way to more simply express my idea of how code should be organised.

I don't consider myself a great or even good JavaScript developer. I think most of the JavaScript I wrote is pretty terrible. But then again, I don't consider myself a good software developer at all. That doesn't mean that I think that I suck at my job, quite the contrary. But writing software, while something I do enjoy most of the time, is not my biggest strength.

As serendipity goes, I am currently working on my first npm module, so I'm currently spending a lot of time trying to get more familiar with the npm eco system, coding conventions and such. And so the left-pad disaster, which I watched, slightly intoxicated by too much craft beer, unfold with what I can only describe as disbelief. It wasn't really about the tech side of things at that time, though, more about the clusterf**k of failed human interaction that made this the catastrophe it was. I'm not going into that discussion, though. I've started another blog post on that and stopped after I took a more thorough look at the situation and decided that it was not worth my time.

What I really found interesting, though, is the following discussion on the micro-module approach that seems to be so common in npm. Of course it is easy to laugh about the fact that a high profile project like babel has a dependency on 11 lines of okayish code with okayish test coverage and a totally-not-okay license (WTFPL, since end of february 2016, btw.).

But while I think that making fun of it is totally legit, especially from the perspective of other languages like Ruby or Python, I think it is more worthwile to look at why this actually happened and if it is really a bad thing.

And here things get messy. JavaScript was not designed as a language for building complex applications and systems programming. In fact, it wasn't, for the most part, if you care about the history of JavaScript, actually designed, but more or less implemented on the spot. Later on, it was standardized and by now, we seem to have somewhat of a stable design process with ES2015 making massive improvements to the language.

But still, it was conceived as a language that would run in a very limited environment, in the browser, or more specifically, in the context of the DOM in the browser. Which meant that JavaScript didn't need a lot of features that are taken for granted in other languages. File system access and I/O in general, but also a stable extension or module system come to mind.

node (and later npm) changed that for server side code. The node core API, which still has a very limited surface and the module system and the combination with npm made it very easy to write modular code that runs on many operating systems.

But, and this is crucial, JavaScript still misses large parts of what other languages would call their standard library. There simply is no method on the String prototype that allows you to left pad a string or a number. And this is the simplest example. I can come up with literally 100 examples of functionality that is available to me from Ruby's standard library (and language core even) which are completely absent from the JavaScript we write today.

At this point, it's important to note that this is slowly, but steadily improving. And guess what, there's even a proposal for adding a padStart function to the String prototype.

But back to the topic. How do people who have been doing node since years see this? Here's the take of Sindre Sorhus on this. Please read this now, if you haven't already.

I do get it. I like the LEGO metaphor. And yeah, from the viewpoint of a node developer, this seems to make sense. And I'm definitely not one of those people who would rather write a left-pad function from scratch. I'm not in the business of reinventing the wheel. I'm in the business of delivering working software.

But again, a lot of times, these npm lego blocks Sindre is raving about are things that are just duct tape to make JavaScript and the node runtime look less of a mess than it is. Take this excerpt from Sindres post:

Yet another example. I have this module user-home which get's the user's home directory. You might think it would be simpler to just do process.platform === 'win32' ? process.env.USERPROFILE : process.env.HOME. And most do this. But first, why require everyone to know how to get the home directory? Why not use a "lego block"? What you also might not realize is that this check is incomplete. On Windows you should also check process.env.HOMEDRIVE + process.env.HOMEPATH and you might also want to do additional checks. Lego blocks.

This touches on something I learned the hard way in 2014 when setting up some node code I wrote on a Windows machine: The abstractions the node core provides are, especially when talking about cross platform support, leaky as an old barn roof. In ruby, I call Dir.home and be done with it. If it's the developer's part to think that hard about differences in operating systems, you have a problem with your development environment and then you can praise your LEGO blocks all you want, it still is a problem that needs to be solved at a completely different level.

But, for the sake of the argument, let's go back to the left-pad example. What strikes me as odd is not that someone in the babel team thought that it is okay to pull in an 11 line dependency. Like I said, better that than coming up with your own, badly tested alternative. I really irritated that the node/npm community obviously thinks that this is better than having a proper, official, well documented, versatile string manipulation library. And to me, Sindre's post doesn't even touch on that. As if it never crossed his mind that this is actually a possibilty.

To understand why this is happening, I think it makes sense to again, look at the way JavaScript, but also node in particular are currently developed, because it is completely different from what, for example, happened with Python or Ruby.

JavaScript is both a set of specifications and a set of very, very different runtimes, running either in browsers or on servers. This is one of the prime reasons why the API surface of JavaScript is relatively limited: It depends on the spec committee to spec out changes and the browser vendors to actually implement the changes as well. Also, again, JavaScript used to have a very narrow usecase, and additionally one with very strong security implications, so it made sense to keep the API surface small.

Node.js on the other hand, is a runtime to run JavaScript programs from the command line, based on originally one JavaScript engine of one of the browser vendors (V8 from Chrome, of course). To make server side developmemt viable, like I said, node.js adds a number of modules to the API surface. If you look at the current API Documentation you can see that these additions are mostly low level building blocks for things you would do to implement a server of sorts.

What the node standard library does not add are features like, you know, complex string manipulations and stuff. And I can see why. The purpose of the core modules is to make node work as a means of building server side things. If you would add a lot of non-specific stuff to the API surface, you automatically nail down the code written for node to the node runtime. It's not as easy to move code from server to client and back. Or between different runtimes. So, that approach makes sense from the unique JavaScript perspective.

Now, add to that that prior to the node era, reusable modules were hardly a thing. People had asset pipelines and tools to minify scripts and stylesheets for a while, but still, dependency management largely consisted of copying files (including the mandatory jquery.min.js) into the project tree somewhere. This also, in my opinion, had the effect of amplifying the NIH (not invented here) syndrome, something that feels much stronger in the JavaScript community than in the Ruby community, at least in my experience. Says Sindre:

Some years ago. Before Node.js and npm. I had a large database of code snippets I used to copy-paste into projects when I needed it. They were small utilities that sometimes came in handy. npm is now my snippet database. Why copy-paste when you can require it and with the benefit of having a clear intent. Fixing a bug in a snippet means updating one module instead of manually fixing all the instances where the snippet is used.

As a last factor, for many people, coming to node from other popular choices for web development at the time such as Rails, the appeal was exactly the minimal approach. Only include what you need. Not the bloaty "batteries included" approach from Rails. I also get that. I think it is, at least partly, misguided, but I get it. This of course, calls for small, very focused modules. Including a module that would solve all your string formatting problems would probably already count as bloat, especially when all you need is to left pad some numbers.

So, combining all of these factors, an intentionally, but also historically minimal API surface, mostly developers who grew up in a very NIH syndrome heavy environment and a sense of "include only what you need" gives me at least a pretty good picture of why the node landscape looks like it does today.

Now, on to the more complicated question: Is this a bad thing or not. My own perspective on this leads to two main points I want to bring up: Accessibility and Maintainability.

Like I said, I spent some time in the node eco system, but I am not a very good node developer and most of the time I start to get back into node, so much has changed that I need to learn everything from scratch. I'm not complaing, node is a very young piece of technology and it would be weird if it didn't change like crazy in the last years, even though it kind of looks to me as if things have somewhat stabilized throughout the last two years or so. But this means that every time I want to use node, I come in with pretty much a beginner's perspective. And the node eco system doesn't really shine here. And npm doesn't really help. The small modules approach doesn't help. The NIH doesn't help. Searching for functionality on npm can be a very tedious job. Now, you could claim the same for rubygems or PyPI, and you would be right, but as I explained earlier, given that JavaScript has such a minimal API surface, for 75% or more of the things I would search an npm module for, I would simply use the stdlib function in Ruby.

What node (and npm) at this point is missing, in my eyes, is curation. If you think about it, a good standard library is exactly that: A (more or less) carefully selected collection of functionality the designer thought will be used by enough users by the language to warrant inclusion.

I don't really care if this curation would be in the form of a bigger standard library or simply a collection of officially recommended npm modules. (Incidentally, Sindre has a list of modules he thinks are awesome, which is awesome, but I wonder how much consensus the node community generally has on this).

I believe, that this poses a huge accessibility problem for node and npm.

The other thing with small modules is that you suddenly have so many of them, especially when your applications get bigger (as almost all applications get). This is a problem in it self and if you don't agree with me on that, you are either inexperienced or you have never maintained a large application. Dependencies are always a liability. The more you have of them, the more you have to keep up to date. Even worse: In node, with the help of npm, every dependency can declare their own dependencies in isolation, so it's absolutely possible that you end up with 3-4 different versions of an 11 line module to left pad a string in your project. This feature is awesome for a number of reasons (technically impossible in Ruby due to the way the module loader works, unfortunately), but also a huge problem for someone responsible for managing an app's dependencies. This is such a huge problem that the awesome people at hood.ie came up with a technical solution for maintaining your dependencies, greenkeeper.io.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment