Instantly share code, notes, and snippets.

Embed
What would you like to do?
Better local require() paths for Node.js

Better local require() paths for Node.js

Problem

When the directory structure of your Node.js application (not library!) has some depth, you end up with a lot of annoying relative paths in your require calls like:

const Article = require('../../../../app/models/article');

Those suck for maintenance and they're ugly.

Possible solutions

Ideally, I'd like to have the same basepath from which I require() all my modules. Like any other language environment out there. I'd like the require() calls to be first-and-foremost relative to my application entry point file, in my case app.js.

There are only solutions here that work cross-platform, because 42% of Node.js users use Windows as their desktop environment (source).

0. The Alias

  1. Install the module-alias package:

    npm i --save module-alias
    
  2. Add paths to your package.json like this:

    {
        "_moduleAliases": {
            "@lib": "app/lib",
            "@models": "app/models"
        }
    }
  3. In your entry-point file, before any require() calls:

    require('module-alias/register')
  4. You can now require files like this:

    const Article = require('@models/article');

1. The Container

  1. Learn all about Dependency Injection and Inversion of Control containers. Example implementation using Electrolyte here: github/branneman/nodejs-app-boilerplate

  2. Create an entry-point file like this:

    const IoC = require('electrolyte');
    IoC.use(IoC.dir('app'));
    IoC.use(IoC.node_modules());
    IoC.create('server').then(app => app());
  3. You can now define your modules like this:

    module.exports = factory;
    module.exports['@require'] = [
        'lib/read',
        'lib/render-view'
    ];
    function factory(read, render) { /* ... */ }

More detailed example module: app/areas/homepage/index.js

2. The Symlink

Stolen from: focusaurus / express_code_structure # the-app-symlink-trick

  1. Create a symlink under node_modules to your app directory:
    Linux: ln -nsf node_modules app
    Windows: mklink /D app node_modules

  2. Now you can require local modules like this from anywhere:

    const Article = require('models/article');

Note: you can not have a symlink like this inside a Git repo, since Git does not handle symlinks cross-platform. If you can live with a post-clone git-hook and/or the instruction for the next developer to create a symlink, then sure.

Alternatively, you can create the symlink on the npm postinstall hook, as described by scharf in this awesome comment. Put this inside your package.json:

"scripts": {
    "postinstall" : "node -e \"var s='../src',d='node_modules/src',fs=require('fs');fs.exists(d,function(e){e||fs.symlinkSync(s,d,'dir')});\""
  }

3. The Global

  1. In your entry-point file, before any require() calls:

    global.__base = __dirname + '/';
  2. In your very/far/away/module.js:

    const Article = require(`${__base}app/models/article`);

4. The Module

  1. Install some module:

    npm install app-module-path --save
  2. In your entry-point file, before any require() calls:

    require('app-module-path').addPath(`${__dirname}/app`);
  3. In your very/far/away/module.js:

    const Article = require('models/article');

Naturally, there are a ton of unmaintained 1-star modules available on npm: 0, 1, 2, 3, 4, 5

5. The Environment

Set the NODE_PATH environment variable to the absolute path of your application, ending with the directory you want your modules relative to (in my case .).

There are 2 ways of achieving the following require() statement from anywhere in your application:

const Article = require('app/models/article');

5.1. Up-front

Before running your node app, first run:

Linux: export NODE_PATH=.
Windows: set NODE_PATH=.

Setting a variable like this with export or set will remain in your environment as long as your current shell is open. To have it globally available in any shell, set it in your userprofile and reload your environment.

5.2. Only while executing node

This solution will not affect your environment other than what node preceives. It does change your application start command.

Start your application like this from now on:
Linux: NODE_PATH=. node app
Windows: cmd.exe /C "set NODE_PATH=.&& node app"

(On Windows this command will not work if you put a space in between the path and the &&. Crazy shit.)

6. The Start-up Script

Effectively, this solution also uses the environment (as in 5.2), it just abstracts it away.

With one of these solutions (6.1 & 6.2) you can start your application like this from now on:
Linux: ./app (also for Windows PowerShell)
Windows: app

An advantage of this solution is that if you want to force your node app to always be started with v8 parameters like --harmony or --use_strict, you can easily add them in the start-up script as well.

6.1. Node.js

Example implementation: https://gist.github.com/branneman/8775568

6.2. OS-specific start-up scripts

Linux, create app.sh in your project root:

#!/bin/sh
NODE_PATH=. node app.js

Windows, create app.bat in your project root:

@echo off
cmd.exe /C "set NODE_PATH=.&& node app.js"

7. The Hack

Courtesy of @joelabair. Effectively also the same as 5.2, but without the need to specify the NODE_PATH outside your application, making it more fool proof. However, since this relies on a private Node.js core method, this is also a hack that might stop working on the previous or next version of node.

In your app.js, before any require() calls:

process.env.NODE_PATH = __dirname;
require('module').Module._initPaths();

8. The Wrapper

Courtesy of @a-ignatov-parc. Another simple solution which increases obviousness, simply wrap the require() function with one relative to the path of the application's entry point file.

Place this code in your app.js, again before any require() calls:

global.rootRequire = name => require(`${__dirname}/${name}`);

You can then require modules like this:

const Article = rootRequire('app/models/article');

Another option is to always use the initial require() function, basically the same trick without a wrapper. Node.js creates a new scoped require() function for every new module, but there's always a reference to the initial global one. Unlike most other solutions this is actually a documented feature. It can be used like this:

const Article = require.main.require('app/models/article');

Since Node.js v10.12.0 there's a module.createRequireFromPath() function available in the stdard library:

const { createRequireFromPath } = require('module')
const requireUtil = createRequireFromPath('../src/utils')

requireUtil('./some-tool')

Conclusion

0. The Alias
Great solution, and a well maintained and popular package on npm. The @-syntax also looks like something special is going on, which will tip off the next developer whats going on. You might need extra steps for this solution to work with linting and unit testing though.

1. The Container
If you're building a slightly bigger application, using a IoC Container is a great way to apply DI. I would only advise this for the apps relying heavily on Object-oriented design principals and design patterns.

2. The Symlink
If you're using CVS or SVN (but not Git!), this solution is a great one which works, otherwise I don't recommend this to anyone. You're going to have OS differences one way or another.

3. The Global
You're effectively swapping ../../../ for __base + which is only slightly better if you ask me. However it's very obvious for the next developer what's exactly happening. That's a big plus compared to the other magical solutions around here.

4. The Module
Great and simple solution. Does not touch other require calls to node_modules.

5. The Environment
Setting application-specific settings as environment variables globally or in your current shell is an anti-pattern if you ask me. E.g. it's not very handy for development machines which need to run multiple applications.

If you're adding it only for the currently executing program, you're going to have to specify it each time you run your app. Your start-app command is not easy anymore, which also sucks.

6. The Start-up Script
You're simplifying the command to start your app (always simply node app), and it gives you a nice spot to put your mandatory v8 parameters! A small disadvantage might be that you need to create a separate start-up script for your unit tests as well.

7. The Hack
Most simple solution of all. Use at your own risk.

8. The Wrapper
Great and non-hacky solution. Very obvious what it does, especially if you pick the require.main.require() one.

@isaacs

This comment has been minimized.

isaacs commented Dec 20, 2013

Just set up your stuff as modules, and put them in node_modules folder, and then they're top-level things. Problem solved.

@tj

This comment has been minimized.

tj commented Dec 20, 2013

solution we often use:

  • a single path (usually ./lib) exposed via NODE_PATH
  • shallow nesting (if ever)

let's you drop in node modules if you need to "fork" them and don't yet have a private registry. Lots of nesting in an app ends up sucking more often than not, and I'd argue that ../ in any module is usually an anti-pattern, maybe other than var pkg = require('../package') for bin .version etc

@branneman

This comment has been minimized.

Owner

branneman commented Dec 20, 2013

@isaacs; yes I know that's an option, but the node_modules folder currently is a nice clean place for only the external modules we use. All the application-specific modules are not generic enough to be put inside node_modules. Like all kinds of Controllers, Models and stuff. I don't think the node_modules folder is intended for that, is it?

@mikeal

This comment has been minimized.

mikeal commented Dec 20, 2013

yeah, whenever i see '../../../dir/name' i immediately think that someone has either 1) prematurely broken out their app in to a million directories and file or 2) hasn't modularized these components in to modules yet, and they should.

@mikeal

This comment has been minimized.

mikeal commented Dec 20, 2013

@branneman we do things in 3 phases.

  1. something is a single file library in our app
  2. we break it in to a proper node module and check it in to node_modules
  3. we publish it and give it its own repository.

If it has application logic, it's not in node_modules. If a lot of things call it or depend on it, it shouldn't have application logic in it, it should be a node_module.

This helps us keep things clean and lets us write things for ourselves, make sure they work, then publish them and hopefully see others getting use from them and contributing.

@tj

This comment has been minimized.

tj commented Dec 20, 2013

I should note that NODE_PATH can be confusing too if you're not familiar with the app, it's not always clear where a module is coming from unless it's named in an obvious way, we prefix ours with s- so it's obvious but they now live in a private registry

@branneman

This comment has been minimized.

Owner

branneman commented Dec 20, 2013

Thanks for all the feedback!

I hear mostly: if you have this problem: you have a bad architecture or bad application design. I also hear: maybe it's time for a private npm repository?

As an example, most modules in one of my applications depend on a config file, still I can not remove application logic from that, and I'm already using a proper (external) module to handle common config logic. But the data itself needs to be either loaded a lot or passed around a lot.

Would it then be a best practice to save that config object once per request to the req variable in express.js? I doubt that, because I'm touching objects I don't own. What is the way to do that kind of thing?

One of the other things I tried with a old version is require.paths, but that's removed now. That was actually the most elegant solution in my opinion. At least everything would stay inside the app, it's the developers responsibility to use it wisely.

@creationix

This comment has been minimized.

creationix commented Dec 21, 2013

I used to use the symlink method, but it's too much trouble on windows so I don't use it anymore.

In most my projects nowadays I don't have this problem. I use relative requires for intra-package modules.

I used to mix local deps with npm deps in node_modules, but that made my .gitignore too much trouble to only ignore certain deps.

My current behavior is:

1 - Write a single file
2 - when it gets too big, start moving parts to other files in the same folder with relative requires
3 - When there are too many modules, package some into reusable modules independent of my app or library.

I use symlinks (or nested directories on windows) to link my different packages to each-other, but each has it's own git repo and if it's generally usable, it's own npm name.

@defunctzombie

This comment has been minimized.

defunctzombie commented Jan 30, 2014

A while back I proposed the file:/// dependency for private installs.

Essentially the following in your package.json

"dependencies": {
    "whatever": "file///relative/path/to/folder"
}

It would only work for private packages but is an easy way to have the package management/install system take care of setting up the symlink for you at install time. This avoids all of the above described hacks and also has the benefit of letting you reference package.json when you want to learn about a dependency (which you do already).

@dskrepps

This comment has been minimized.

dskrepps commented Feb 6, 2014

The start up script is a good option, though all the solutions have some drawback. At the very least others looking at your code might not know where the require is looking for modules. You also want to eliminate the possibility of new dependencies colliding with modules of the same name.

I haven't noticed anyone mention using the relationship between your dependencies and your project root. So I went and built it myself: requireFrom. This method is intuitive to anyone looking at it, and requires no extra steps outside of adding a dependency. Third-party modules can use it relative to themselves, as well.

var requireFrom = require('requirefrom');
var models = requireFrom('lib/components/models');

var Article = models('article');

Thanks for writing up this overview.

@alexgorbatchev

This comment has been minimized.

alexgorbatchev commented Feb 20, 2014

I've been using symlinks with the following structure:

/node_modules
/package.json
/src
  /node_modules
    /client -> ../client
    /server -> ../server
    /shared -> ../shared
  /client
    /apps
      /main
        /test
          main.spec.js
        index.js
    /modules
      /foo
        /test
          foo.spec.js
        index.js
  /server
    /apps
    /modules
  /shared

it also solves the problem of not know where the modules come from because all app modules have client/server/shared prefixes in require paths

@indirectlylit

This comment has been minimized.

indirectlylit commented Feb 22, 2014

I ran into the same architectural problem: wanting a way of giving my application more organization and internal namespaces, without:

  • mixing application modules with external dependencies or bothering with private npm repos for application-specific code
  • using relative requires, which make refactoring and comprehension harder
  • using symlinks or environment variables which don't play nicely with source control

The start-up script is a good idea, but I didn't like the extra moving parts.

In the end, I decided to organize my code using file naming conventions rather than directories. A structure would look something like:

  • node_modules
    • ...
  • package.json
  • npm-shrinkwrap.json
  • src
    • app.js
    • app.config.js
    • app.models.bar.js
    • app.models.foo.js
    • app.web.js
    • app.web.routes.js
    • ...

Then in code:

var app_config = require('./app.config');
var app_models_foo = require('./app.models.foo');

or just:

var config = require('./app.config');
var foo = require('./app.models.foo');

and external dependencies are available from node_modules as usual:

var express = require('express');

In this way, all application code is hierarchically organized into modules and available to all other code relative to the application root.

The main disadvantage is of course that in a file browser, you can't expand/collapse the tree as though it was actually organized into directories. But I like that it's very explicit about where all code is coming from, and it doesn't use any 'magic'.

@flodev

This comment has been minimized.

flodev commented Mar 5, 2014

Hi,

the start up script doesn't work very well with nodemon (or node forever).
If something changes nodemon tries to restart the start-up script and in my case the childprocess (express js) is still bound to my IP and I got a EADDRINUSE error.
I also tried to kill the child process but this will be executed too late.

var app = spawn(process.execPath, args, opt);

process.on('exit', function() {
    console.log("kill child process");
    app.kill('SIGINT');
});

edit:
I've switched to the approach used by alexgorbatchev using a server and shared folder and making symlinks to node_modules folder.
Thank you it works great.

@gumaflux

This comment has been minimized.

gumaflux commented Mar 14, 2014

@visionmedia: quite like the idea of the no/low nesting, but how does that work with larger a source base - I have seen a few of your github reps which manifest what you say - I'm thinking that maybe an application has a more sprawling areas of functionality? ( I'm a newbie on node so I might be speculating? )

@tuliomonteazul

This comment has been minimized.

tuliomonteazul commented Mar 24, 2014

I also found a good way to use the start-up script solution with Grunt and nodemon.

In my Gruntfile.js, I just have set:

grunt.initConfig({
        concurrent: {
            dev: {
                tasks: ['nodemon', 'node-inspector', 'watch', 'mochaTest'],
                options: {
                    logConcurrentOutput: true
                }
            }
            ...
        },
        nodemon: {
            dev: {
                script: 'index.js',
                options: {
                    nodeArgs: ['--debug'],
                    env: {
                        NODE_PATH: './app'
                    }
                }
            }
        },
        ...

So just setting the options.env inside nodemon configuration and my application is still starting by just calling $ grunt

@patrick-steele-idem

This comment has been minimized.

patrick-steele-idem commented Apr 4, 2014

Here's another option to consider:
https://github.com/patrick-steele-idem/app-module-path-node

The app-module-path modifies the internal Module._nodeModulePaths method to change how the search path is calculated for modules at the application-level. Modules under "node_modules" will not be impacted because modules installed under node_modules will not get a modified search path.

It of course bothers me that a semi-private method needed to be modified, but it works pretty well. Use at your own risk.

The startup script solution will impact module loading for all installed modules which is not ideal. Plus, that solution requires that you start your application in a different way which introduces more friction.

@a-ignatov-parc

This comment has been minimized.

a-ignatov-parc commented Apr 28, 2014

You can create helper function in global scope to be able require modules relative to root path.

In app.js:

global.app_require = function(name) {
    return require(__dirname + '/' + name);
}

var fs = require('fs'),
    config = app_require('config'),
    common = app_require('utils/common');

It also will work in other files.

@esco

This comment has been minimized.

esco commented May 17, 2014

@gumaflux I believe @visionmedia is only talking about modules which usually wouldn't require "sprawling areas of functionality" because a single module isn't meant to do as much as an application. I think the nesting issue is more of a problem in applications, especially MVC apps.

@slorber

This comment has been minimized.

slorber commented May 19, 2014

I'm using browserify for a browser app.

The problem using paths, or putting code into node_modules is that in your app you may have sources to transform, for exemple CoffeeScript or JSX files.

When using require("some_private_node_module"), browserify doesn't seem to transform the files and builds a bundle with unprocessed sources.

@substack

This comment has been minimized.

substack commented May 30, 2014

@slorber Put the transforms in each module's package.json https://github.com/substack/browserify-handbook#browserifytransform-field

Now your code will work and is less vulnerable to system-wide configuration changes and upgrades because each component can have its own local transforms and dependencies.

See also: avoiding ../../../../../../.. which pretty much echos what @isaacs has said already: just use node_modules/.

If you're worried about how node_modules might clutter up your app, create a node_modules/app and put all your modules under that package namespace. You can always require('app/whatever') for some package node_modules/app/whatever.

Not sure how node_modules/ works? It's really nifty!

@joelabair

This comment has been minimized.

joelabair commented Jun 20, 2014

So....

This is a small hack. It relies only on node.js continuing to support the NODE_PATH environment variable. The NODE_PATH env setting is a fine method for defining an application specific local modules search path. However, I don't like relying on it being properly set external to javascript, in all cases (i.e. export, bash profile, or startup cmd). Node's module.js absorbs process.env's NODE_PATH into a private variable for inclusion into a list of global search paths used by require. The problem is, node only looks at process.env['NODE_PATH'] once, on main process init, before evaluating any of the app's code. Including the following 2 lines allows the re-definition of NODE_PATH, post process-init, and should be included prior to any local module specific requires. In a top level file include:

process.env['NODE_PATH'] = __dirname + '/lib';
require('module').Module._initPaths();

Then simply require any modules in ./lib

var myLocalLibModule = require('myLocalLibModule'); 
...

This does not change the behavior of module.js as documented; node_modules, package.json, and global modules all behave as expected.

@kgryte

This comment has been minimized.

kgryte commented Jun 24, 2014

Another option for complex application logic (config files, loggers, database connections, etc) is to use inversion of control (IoC) containers with dependency injection. See @jaredhanson's Electrolyte for one implementation.

@branneman

This comment has been minimized.

Owner

branneman commented Jun 30, 2014

I just updated the article again and added more solutions. Thanks for all the feedback, keep it coming!

@joelabair: Great suggestion, added it as solution 6.

@a-ignatov-parc: Love the simplicity, added it as solution 7. Great and non-hacky.

@dskrepps: I don't like the fact that I would need to call require('requirefrom') in every file, unless you make it global like @a-ignatov-parc's solution as well. And then it's not that different from solution 7. (Altough I now see that you commented that one first!)

/cc @isaacs, @visionmedia, @mikeal, @creationix, @defunctzombie, @dskrepps, @alexgorbatchev, @indirectlylit, @flodev, @gumaflux, @tuliomonteazul, @patrick-steele-idem, @a-ignatov-parc, @esco, @slorber, @substack, @joelabair, @kgryte

@awei01

This comment has been minimized.

awei01 commented Jul 26, 2014

FWIW, in case anyone is using Jest for testing, I tried solution 1 referenced above and it broke everything. But after hacking around, I figured out a way to make symlinks work: facebook/jest#98

@valtido

This comment has been minimized.

valtido commented Aug 12, 2014

This might be the worst IDEA ever, but what do you guys think about this ?

# CoffeeScript Example
$require = require
require = (file)->
    if /^\/\/.*$/.test file
        file = file.slice 1, file.length
        $require.resolve process.cwd() + file
    else 
        $require file


//JavaScript Example
var $require, require;
$require = require;
require = function(file) {
  if (/^\/\/.*$/.test(file)) {
    file = file.slice(1, file.length);
    return $require.resolve(process.cwd() + file);
  } else {
    return $require(file);
  }
};

You can add that on the first line to override the require function with a reference to itself...

now, you can use require("express") as normal, and require("//lib/myLibFile") the difference is the leading //, inspired by the use in http requests //ajax.googleapis.com/ajax/libs/jqueryui/1.11.0/jquery-ui.min.js.

@MarkKahn

This comment has been minimized.

MarkKahn commented Sep 14, 2014

My current solution is to have my script spawn a child-process to itself if NODE_PATH isn't set. This allows me to just run node file.js and not worry about anything else:

if( !process.env.NODE_PATH ){
    // set NODE_PATH to `pwd`
    process.env.NODE_PATH = __dirname + '/';

    require( 'child_process' ).spawn( 'gulp', [].slice.call( process.argv, 2 ), {
        stdio: 'inherit'
    } );

    // "throw away" logging from this process.  The child will still be fine since it has access to stdout and its own console.log
    console.log = function(){};
}else{
    // start app
}
@UnquietCode

This comment has been minimized.

UnquietCode commented Sep 17, 2014

Thank you for this write-up! I went with #7 and have global method Require which complements require.

@cronvel

This comment has been minimized.

cronvel commented Oct 2, 2014

And what about:
var myModule = require.main.require( './path/to/module' ) ;
... seems to work pretty well as long as your main js file is at the root of your project.

@azu

This comment has been minimized.

azu commented Oct 4, 2014

npm 2.0 support Local Paths.

@viruschidai

This comment has been minimized.

viruschidai commented Oct 10, 2014

I did a lib when I tried to restructure some source code in a large project. https://github.com/viruschidai/node-mv move a source file and update all require paths to the moved file.

@stringparser

This comment has been minimized.

stringparser commented Oct 26, 2014

@azu nice! Still...

This feature is helpful for local offline development and creating tests that require npm installing where you don't want to hit an external server, but should not be used when publishing packages to the public registry.

What I've been doing is to exploit the require.cache. If I have a package, say utils on node_modules I'll do a lib/utils and on there I'll merge the cache of utils to have whatever I want. That is:

var util = require('utils');
util.which = require('which');
util.minimist = require('minimist');
module.exports = util;

So I only have to require that package once and then utils.<some package> will give the necesary pack.

@gagle

This comment has been minimized.

gagle commented Nov 9, 2014

This is my contribution to this topic: https://github.com/gagle/node-getmod

It just shortens the relative paths by introducing marks, points from which paths can be relative.

@renatoargh

This comment has been minimized.

renatoargh commented Nov 17, 2014

My solution is;

var path = require('path');

global._require = function(path) { //I call it 'reversal require'
    return require(path.join(__dirname, path));
}

//PS.: This code should in the root level folder of your project!

You are now basically requiring your .js files from base instead of cwd

@booleangate

This comment has been minimized.

booleangate commented Dec 2, 2014

A word of caution for people using the symlink approach with Browserify: you are likely to break transforms. This has been my experience with brfs and trying to include a module through a symlinked path. The transformer seems to ignore symlinked paths (or probably packages that are in the node_modules directory).

However, it turns out that there's an additional option for strategy #4 if you're using a build tool like gulp (and still works with browserify transforms). I've simply added process.env.NODE_PATH = "./my/include/path:" + (process.env.NODE_PATH || ""); to my gulpfile.js and everything works great now.

@enricostara

This comment has been minimized.

enricostara commented Dec 13, 2014

I released requirish, a solution that mixes strategy #3(rekuire) and #7 (require.main.require)
The tool is also a browserify-transform that convert back all the require() statements for browser, adding again the long relative paths only for the browserify processor

@davidshimjs

This comment has been minimized.

davidshimjs commented Jan 1, 2015

@azu Local path in npm isn't be synchronized with original source code when I edit it in original folder. It doesn't make a symbolic link.

@gavinengel

This comment has been minimized.

gavinengel commented Jan 3, 2015

I just made this module (my first) so I'd love to hear feedback (on my github page, not on this thread): https://www.npmjs.com/package/magic-globals

// require this module without assigning export
require('magic-globals');

// you may now use additional global objects in any module,
// in addition to built-ins: __filename and __dirname
console.log('__line: ' + __line); // ex: 6
console.log('__file: ' + __file); // ex: server
console.log('__ext: ' + __ext); // ex: js
console.log('__base: ' + __base); // ex: /home/node/apps/5pt-app-model-example/api-example
console.log('__filename: ' + __filename); // ex: /home/node/apps/5pt-app-model-example/api-example/server/server.js
console.log('__function: ' + __function); // ex: (anonymous) 
console.log('__dirname: ' + __dirname); // ex: /home/node/apps/5pt-app-model-example/api-example/server
@andineck

This comment has been minimized.

andineck commented Jan 16, 2015

For me the hack presented by @joelabair works really well. I tested it with node v0.8, v0.10, v0.11 and it works well. In order to reuse this solution, i made a little module where you can just add the folders that should behave like the node_modules folder.
https://www.npmjs.com/package/local-modules

require('local-modules')('lib', 'components');

like @creationix, I didn't want to mess with private dependencies in node_modules folder.

@ivan-kleshnin

This comment has been minimized.

ivan-kleshnin commented Jan 21, 2015

If you put parts of your app into node_modules you can't exclude node_modules from search scope anymore. So you lose the ability to quick search through project files. This kinda sucks.

@ivan-kleshnin

This comment has been minimized.

ivan-kleshnin commented Jan 21, 2015

As for local-modules solution and likes...

When you start to import app modules like require("something") and those modules are not really reside in node_modules it feels like an evil magic to me. Import semantics was changed under the cover.

I actually think it should be resolved by adding special PROJECT ROOT symbol and patching native require. Syntax may be like require("~/dfdfdf").
But ~ will be confused with unix home dir so it's better to choose something else like require("@/dfdfdf").

Explicit is better than implicit, as noone may miss "@" symbol in import statements.
We basically add different syntax for different semantics which is good imo.

I believe having a special shims.js file for every non-standard installation like this in project folder is sane and safe enough.

https://gist.github.com/ivan-kleshnin/edfa4abefe8ce216b9fa

What do you guys think?

@gagle

This comment has been minimized.

gagle commented Jan 25, 2015

This is my second approach. It just implements the __root solution which, in my opiniom, it's the best solution to this problem and nodejs/iojs should implement it.

https://github.com/gagle/node-groot

I also like the require("@/dfdfdf") approach.

@gustavohenke

This comment has been minimized.

gustavohenke commented Feb 1, 2015

I wrote in my blog about a few solutions presented here versus ES6 problems:
http://injoin.io/2015/01/31/nodejs-require-problem-es6.html

@gagle

This comment has been minimized.

gagle commented Feb 1, 2015

@gustavohenke nice one, very hackish but cleaner and cross-functional among OS's. But the problem with it is the same as with putting the modules inside node_modules. Having a require call require('my/package') it's very confusing for me because I associate require paths without a leading ./ with core or external modules. You could have an external module named my, collisions may happen.

@gustavohenke

This comment has been minimized.

gustavohenke commented Feb 1, 2015

Yeah @gagle, I understand these problems, but my case is special, I won't be dropping ES6 modules. Fortunately, I have taken care of namespacing my libs so there's only a single collision point. Also, my app is well documented for developers.

@aforty

This comment has been minimized.

aforty commented Feb 4, 2015

This gist is so incredibly helpful. Kind of embarrassing that Node has an issue with this many hackish solutions.

@ColCh

This comment has been minimized.

ColCh commented Feb 5, 2015

It seemed that NODE_PATH is most clean solution

@doron2402

This comment has been minimized.

doron2402 commented Feb 6, 2015

seems like
if you can turn this into a node module do it
else just define it in your index.js or app.js
if (!global.__base) { global.__base = __dirname + '/'; }

@ericelliott

This comment has been minimized.

ericelliott commented Feb 9, 2015

Holy crap. Lots of hacky solutions here.

Try this instead: rootrequire

The readme:

rootrequire

Require files relative to your project root.

Install

npm install --save rootrequire

Use

var
  root = require('rootrequire'),
  myLib = require(root + '/path/to/lib.js');

Why?

  • You can move files around more easily than you can with relative paths like ../../lib/my-lib.js
  • Every file documents your app's directory structure for you. You'll know exactly where to look for things.
  • Dazzle your coworkers.

Learn JavaScript with Eric Elliott

This was written for the "Learn JavaScript with Eric Elliott" courses. Don't just learn JavaScript. Learn how to change the world.

@koresar

This comment has been minimized.

koresar commented Feb 11, 2015

To make node.js search for modules in an additional directory you could use require.main.path array.

// require('node-dm'); <-- Exception
require.main.paths.push('/home/username/code/projectname/node_modules/'); // <- any path here
console.log(require('node-dm'));  // All good
@Talento90

This comment has been minimized.

Talento90 commented Feb 15, 2015

I'm using the wrapper solution. No magic just elegance.

Thanks for this post!

@ivan-kleshnin

This comment has been minimized.

ivan-kleshnin commented Feb 24, 2015

@ericelliott, with your solution IDE navigation is lost in the same way as with others...
There is no escape from this problem at app code level. Every "trick" breaks IDE move-to functionality.
From all those "solutions", only symlinks keep IDE working as it should.

@sylvainv

This comment has been minimized.

sylvainv commented Feb 25, 2015

Thanks for the post, very useful and detailed. I found the wrapper solution to be the most elegant, works on any latest node instance and does not require any pre-setup / hacks for it to work.

Besides it let me set the path to the library and avoid any potential name conflict issues.

@etcinit

This comment has been minimized.

etcinit commented Mar 2, 2015

I'll add my library to the list: https://github.com/etcinit/enclosure (It's very Java-like though)

@jondlm

This comment has been minimized.

jondlm commented Mar 3, 2015

Turns out that npm now flattens your dependency tree which breaks the "rootrequire" method by @ericelliott.

I found a work around though: http://www.jondelamotte.com/solving-node-project-requires/

@rahularyan

This comment has been minimized.

rahularyan commented Mar 9, 2015

Thanks for the awesome tutorial

@scharf

This comment has been minimized.

scharf commented Mar 13, 2015

Create symlink using node in npm postinstall

Since symlink is the only solution that does not confuse IDEs (as @ivan-kleshnin noted), here is my solution: add a postinstall script to the package.json that creates a symlink from the app directory the to node_modules (note the srcpath link is specified relative to the node_modules):

  "scripts": {
    "postinstall" : "node -e \"var srcpath='../app'; var dstpath='node_modules/app';var fs=require('fs'); fs.exists(dstpath,function(exists){if(!exists){fs.symlinkSync(srcpath, dstpath,'dir');}});\""
  },

The script could also be put into a separate file, but I prefer to specify it directly inside the package.json...

For readability, here is the one-liner well formatted:

// the src path relative to node_module
var srcpath = '../app';
var dstpath = 'node_modules/app';
var fs = require('fs');
fs.exists(dstpath, function (exists) {
    // create the link only if the dest does not exist!
    if (!exists) {
        fs.symlinkSync(srcpath, dstpath, 'dir');
    }
});

I think it should work on windows as well, but I have not tested it.

@tomatau

This comment has been minimized.

tomatau commented Apr 1, 2015

Would like to see an updated article for JS module syntax, as it requires you to be static with your imports - many of these solutions won't work

@sh-a-v

This comment has been minimized.

sh-a-v commented Apr 2, 2015

@scharf, on windows it works. You only should run cmd as admin
But fs.exists returns always false, so I replaced it with fs.readlink:

fs.readlink(dstpath,function(err, existLink){if(!existLink){fs.symlinkSync(srcpath, dstpath,'dir');}})
@jaubourg

This comment has been minimized.

jaubourg commented Apr 3, 2015

I developed wires because we had configuration and routing nightmares at my company. We've been using it for 2 years now and I just released version 0.3.0 which is world-ready, so have fun using it and don't hesitate with feedback, questions or death-threats :P

Using wires, you would create a wires.json file at the root of your app:

{
    ":models/": "./lib/models/"
}

And then just require models like this:

require( ":models/article" );
require( ":models/client" );

And call your main script using the wires binary:

wires startServer

There's a lot more to wires but I felt like sharing on this specific topic.

Hope this helps! :)

@sinejoe

This comment has been minimized.

sinejoe commented Apr 6, 2015

We (sineLABS) created and published the very minimal rqr node package for this as well.

@ArnaudRinquin

This comment has been minimized.

ArnaudRinquin commented Apr 13, 2015

If this issue was solved, I think local modules would be the ultimate key to the problem.

All we need is npm outdated and npm update to not ignore private (local + not published) modules and handle them properly based on local package.json version.

Here is a proof of concept project, showing how clean and easy it would be.

@timoxley

This comment has been minimized.

timoxley commented Apr 22, 2015

yet another solution to this… building atop npm's local modules: https://github.com/timoxley/linklocal

@nicksellen

This comment has been minimized.

nicksellen commented Apr 26, 2015

I was surprised how unsolved this situation is, I summarized some of the available techniques here http://nicksellen.co.uk/2015/04/17/how-to-manage-private-npm-modules.html (with a focus on private rather than local modules) - I'd appreciate any thoughts/corrections/feedback!

@ArnaudRinquin

This comment has been minimized.

ArnaudRinquin commented May 6, 2015

My PR solving the npm local module handling has been merged and is now shipped with npm 2.9.0 / iojs 2.0.0.

It's very simple to refer to a local module, simply update your package.json:

{
  "name": "my-app",
  "dependencies": {
    "my-app-models":"file:path/to/my/app-modles"
  }
}

You can now use local module and enjoy a very simple, clean, non-hacky way. It comes with additional perks, like having proper modules.

Explained here, proof of concept updated.

@heyimalex

This comment has been minimized.

heyimalex commented May 13, 2015

The package.json script for #1 has an error: dstpath needs to just be d. Also you could make it even shorter by doing f=require('fs'). If you're into that kind of thing.

@mmahalwy

This comment has been minimized.

mmahalwy commented May 15, 2015

@branneman

This comment has been minimized.

Owner

branneman commented May 25, 2015

Updated. Thanks for the contributions all!

@jrouleau

This comment has been minimized.

jrouleau commented May 31, 2015

The package.json postinstall code doesn't work as the destination d variable is being shadowed by the fs.exists() callback d. Additionally, fs.exists() will be depreciated: https://nodejs.org/api/fs.html#fs_fs_exists_path_callback

See fixed code below:

"scripts": {
  "postinstall" : "node -e \"try{require('fs').symlinkSync('../app','node_modules/app','dir')}catch(e){}\""
}

Tested with npm version 2.7.6 and node version v0.12.2

@shachr

This comment has been minimized.

shachr commented Jun 2, 2015

Joynet and then Jetbrains need to support "/" in the require path parameter, this will reference the process.cwd(),
if a module is being loaded from the node_module directory meaning it is a dependency "
/" should be the module's root and not the process.cwd()

every other workaround is a temporary solution,
i tried to do that using a different aproach by overriding the require prototype function:

module.constructor.prototype.require = function (path) {
    try {
        var dirname = pathModule.dirname(this.filename);
        if( path.indexOf("./") > -1 && path.indexOf("./") < 2 ){ // if starts with ./ or ../
            path = pathModule.resolve(dirname, path);
        }

     var mdl = globalContainer.resolve(path);
        if( !mdl ){
            mdl = this.constructor._load(path, this); //todo: suport DI here also?
            if(mdl)
                mdl = globalContainer.instantiate(mdl);
        }

        return mdl;

    } catch (err) {
        handleException( err, path );
    }
};

this allowed me to easily create mocks and modify the path just before i pass it to the real require.
this can potentially allow me to support "~/" but then i'm loosing intellij "go to decleration" feature.

@tinwatchman

This comment has been minimized.

tinwatchman commented Jun 16, 2015

Hey -

I've also been working on this problem recently. Here's what I've come up with: use-module and projectjs. The latter is a work-in-progress.

@rapilabs

This comment has been minimized.

rapilabs commented Jun 19, 2015

For anyone looking to do this with webpack there's an alias setting: http://webpack.github.io/docs/configuration.html#resolve-alias

@nuc

This comment has been minimized.

nuc commented Jun 20, 2015

@rapilabs Thank you! 🍻

@Mingling94

This comment has been minimized.

Mingling94 commented Jul 9, 2015

This was VERY well-written. Kudos!

@poxrud

This comment has been minimized.

poxrud commented Jul 21, 2015

@mmahalwy used your method, works great!

@adrianguenter

This comment has been minimized.

adrianguenter commented Aug 2, 2015

A simple "require-proxy" solution that depends on npm-clone (note: doesn't work out-of-the-box for multiple include path cases):

modules_dir_path = '/path/to/node_modules/';
_require = require(modules_dir_path + 'clone')(require);
require = function (name) { return _require(modules_dir_path + name); };

. . .

// Undo:
require = _require; delete modules_dir_path; delete _require;
@victorherraiz

This comment has been minimized.

victorherraiz commented Aug 5, 2015

https://www.npmjs.com/package/xreq
It creates alias for base and local paths. I think, you could find it useful.

@fdaciuk

This comment has been minimized.

fdaciuk commented Sep 4, 2015

https://www.npmjs.com/package/getmodule
Just add getmodule once and enjoy ;)

@akiva

This comment has been minimized.

akiva commented Sep 5, 2015

Symlinking works okay, but I prefer to have each module in my projects contain their own dependencies explicitly stated in their respective package.json files. I find this cleaner and easier to keep up-to-date than having each module's dependencies stored in the global package.json file. With this in mind, symlinks don't cut it.

I've done a few projects where the approach was placing modules directly in node_modules with some prefix, like node_modules/app or node_modules/@app. However, I also feel like I shouldn't have a moment's panic when issuing rm -rf node_modules, so I am not that keen to implement it that way.

The local file: approach is great, but a nuisance while still developing the modules, as they require re-installation with each modification.

Another issue to keep in mind is if one of your local modules requires another of your local modules. The npm install step here (with each module maintaining it's own requirements) can lead to all kinds of fun (!).

@akiva

This comment has been minimized.

akiva commented Sep 5, 2015

After reading npm/npm#7426, I see there hasn't been much in solving this. I understand everyone is looking at it with different requirements. I guess I am just left feeling that as a local module, defined by using a relative path as the target in a package.json file, doesn't contain a version number, it should be approached in such a way that it always remains up-to-date. Whether that means checking the local modules version in package.json or not doesn't really bother me. So, if I have a dependency like "@app/router": "./lib/router" and I modify router, then update the router package.json file to a new version, I would think it would make some difference, but it doesn't. In project root: cat node_modules/@app/router/package.json | grep version will still reflect the old one originally installed at the project's initial npm install time.

After looking into some other recent comments, I saw someone approach it with an example: https://github.com/ArnaudRinquin/local_modules_poc However, the steps suggested at the bottom still do not work for me as described (npm update in a local module after a version bump in it's package.json file). I am using npm 2.14.2, so I am up-to-date there.

I know there are solutions out there, such as @timoxley 's linklocal, but I really feel this should come free with npm modules.

Until this is all resolved, I am left having to always issue rm -rf node_modules && npm install && npm start or the like. For each module. Every time. Evar.

@qoomon

This comment has been minimized.

qoomon commented Sep 28, 2015

i just use following line in my app.js

global.requireFrom = require.main.require;
@fatfisz

This comment has been minimized.

fatfisz commented Oct 20, 2015

I'd like to suggest a new way that I've thought of after unsuccessfully trying to use the solutions listed above (I use Git and I work on Windows). I consider it a hack, because it abuses the require algorithm. But it is almost painless, so maybe someone else will find it useful too.

The solution

Move your project to node_modules/app/ (or anything else instead of app).

Let's say you have a project in a dir project/.
That means every file project/path/to/file.js becomes project/node_modules/app/path/to/file.js.
Any file project/node_modules/something.js becomes project/node_modules/app/node_modules/something.js.

Now you can require files using a simple require('app/path/to/file');.

The cons

  • Longer paths - This could possibly be a problem on Windows, but with an excellent work from the npm team the module structure has now become quite flat. So 14 + appName.length additional chars shouldn't be the thing that could tip the scale
  • How it looks like - Horrible, I know. Although more advanced editors allow you to specify project directories, so replacing the current one with the nested one shouldn't be a problem
  • Possible problems with tools that ignore node_modules - in particular, I had a problem with making Nodemon work, as the default config caused everything to be ignored, and ignore: [] wasn't a solution. This was because of Nodemon's merging algorithm that merged options with the defaults. ignore: ['.git'] solved this

The pros

  • No additional environment variables - Nobody likes them
  • No messing with scripts - This often makes the project incompatible with some tools, e.g. Browserify
  • It's cross-platform - For free

Edit:
This isn't exactly a "new" way, as seen here: http://stackoverflow.com/a/14349344/2211199, but I only stumbled upon this answer on SO after thinking up the solution myself.

@jshanson7

This comment has been minimized.

jshanson7 commented Nov 12, 2015

If you're using Babel, you can hook into the resolveModuleSource option.

In app/index.js:

require('babel-core/register')({
  presets: ['es2015'],
  resolveModuleSource: require('babel-resolver')(__dirname)
});

require('./app');

In app/app.js:

import User from 'models/User';
// => resolves: "app/models/User.js"
@Temaruk

This comment has been minimized.

Temaruk commented Dec 1, 2015

Just a sidenote: I assume that most IDEs have no idea what to do with transformed/aliased require/import paths. Take this into consideration, when looking for better require paths.

@akobler

This comment has been minimized.

akobler commented Dec 18, 2015

Thanks for sharing!

There might a little error in the section 1 The Symlink. The outer string variable d is hidden by the inner argument d that is not a string.

"postinstall" : "node -e \"var s='../app',d='node_modules/app',fs=require('fs');fs.exists(d,function(d){d||fs.symlinkSync(s,d,'dir')});\""

-->

"postinstall" : "node -e \"var s='../app',d='node_modules/app',fs=require('fs');fs.exists(d,function(obj){obj||fs.symlinkSync(s,d,'dir')});\""
@srackham

This comment has been minimized.

srackham commented Jan 4, 2016

This works for me:

module.paths.unshift(__dirname);

Which prepends the current directory to the list of module search directories.
The module object paths field is not documented though, so it's still a hack
(see https://nodejs.org/api/modules.html#modules_the_module_object).

@orchidlegba

This comment has been minimized.

orchidlegba commented Jan 4, 2016

I like using require.main.require() however you will lose IntelliSense using that approach in Visual Studio Code. Are there any IDE or Code editors that can still give IntelliSense in such a situation?

@pavel06081991

This comment has been minimized.

pavel06081991 commented Jan 13, 2016

Here is a one more solution https://www.npmjs.com/package/sp-load

@schwiet

This comment has been minimized.

schwiet commented Jan 27, 2016

If you use the Symlink solution (since node_modules is always in my .gitignore, I have tended to use the method), you may run into the following bug if you later try to npm install anything locally

npm ERR! Cannot read property 'localeCompare' of undefined

npm/npm#9766 may occur if you do not include a package.json file in your target directory. Just an FYI, may be worth a note.

If you're using Webpack, @rapilabs' suggestion of using resolve.alias seems like the right way to go.

@gamtiq

This comment has been minimized.

gamtiq commented Feb 20, 2016

Thank you for the great article!

There is a small typo in the "Conclusion" section:

... that you need to create a seperate start-up script...

@ilearnio

This comment has been minimized.

ilearnio commented Feb 20, 2016

I created a package module-alias (which is highly inspired by app-module-path). The main difference is that this package also allows creating aliases of directories for further usage with require/import.

Example:

const moduleAlias = require('module-alias')

moduleAlias.addAliases({
  '@root'  : __dirname,
  '@server': __dirname + '/src/server'
})

const someModule = require('@server/some-module')


// Register custom module directories
moduleAlias.addPath(__dirname + '/src')
// now use it anywhere like your src folder is in node_modules
require('server') // ./src/server
@franciscolourenco

This comment has been minimized.

franciscolourenco commented Feb 22, 2016

+1

@ArtificerEntertainment

This comment has been minimized.

ArtificerEntertainment commented Feb 27, 2016

Another alternative: https://www.npmjs.com/package/lokal

"localDependencies": {
  "dependency1": "test/dependency1/",
  "dependency-2": "/test/dependency2/dependency2.js",
  "dependency3": "test/dependency3/nested/index.js"
}
const dependency1 = require('lokal')('dependency1');
const dependency2 = require('lokal')('dependency-2');
const rewire = require('rewire');
const rewiredDependency3 = require('lokal')('dependency3', rewire);
@jjqq2013

This comment has been minimized.

jjqq2013 commented Feb 27, 2016

+1

@mderazon

This comment has been minimized.

mderazon commented Feb 29, 2016

@ArnaudRinquin npm local module would have been perfect, but it has problems.

If you change the local module often, you have to run npm install each time in order for npm to copy the local module into node_modules. To make things even worse, you have to change the module's package version in order for npm to detect any change in the module.

If you're in development and changing stuff rapidly, this workflow is too cumbersome.
(see npm/npm#7426)

Best approach IMO is adding NODE_PATH to package.json

{
  "scripts": {
    "start": "NODE_PATH=./lib node app"
  }
}

(not cross platform)

@danawoodman

This comment has been minimized.

danawoodman commented Mar 6, 2016

Worth mentioning; there is a library called babel-root-import that rewires the import statement via a Babel plugin so you can do:

import something from '~/lib/something'

The plugin supports changing the prefix (eg instead of ~ you could use @ or something else).

You can also change the root that is used so if your code is in src/ you could set that and then just require ~/lib/foo which is located at src/lib/foo instead of ~/src/lib/foo.

This seems to be a nice solution as it is explicit that you're requiring from the root of a project and it doesn't require a build environment variable like the NODE_PATH hack does (eg it works everywhere in your code as long as you're using Babel).

I don't think it (yet) works on Windows tho, so be aware of that if you need Windows support.

@trinitronx

This comment has been minimized.

trinitronx commented Mar 11, 2016

Kinda surprises me that NODE_PATH is considered a "Hack" in the sense that it's not encouraged or not to be depended on Node supporting this in the future? 😮

This is the simplest, most elegant, and most similar across-language solution, as used by many other OO & scripting languages (e.g.: $LOAD_PATH in Ruby, $CLASSPATH in Java, $PYTHONPATH in Python, $PERLLIB, $PERL5LIB, or @INC in Perl, $PATH in Bash, etc... etc...)

@rbosneag

This comment has been minimized.

rbosneag commented Mar 11, 2016

Correct me if I'm wrong but most of the above solutions may affect different needs or setups, like:

  • running the app with pm2 or script
  • testing individual files: '../../../models/article' with mocha
  • reuse of components
  • use of webpack, browserify, etc
  • impact on IDE
  • source control: git or svn
  • deployment server OSs
  • access level to deployment server, etc

I suggest we restructure the list (or at least mention the affected use cases) based on the impact each solution might have for different needs.

Awesome post! Kudos to @branneman and all the contributors!

@tuananhtd

This comment has been minimized.

tuananhtd commented Mar 13, 2016

There is a typo in the postinsall hook. The fs.exists's callback will receive a boolean value so it should be like this:

"postinstall" : "node -e \"var s='../src',d='node_modules/src',fs=require('fs');fs.exists(d,function(exists){exists||fs.symlinkSync(s,d,'dir')});\""
@jmm

This comment has been minimized.

jmm commented Apr 8, 2016

I've looked through a bunch of these and I'm experimenting with an approach that I haven't seen in any I've looked at, that has these features:

  • Doesn't require any changes to normal code except using the paths you want (e.g. in each module you don't have to require() a dependency to make it work or use some replacement function instead of normal require() calls). It only requires initialization upon startup, e.g: node -r ./init.js normal-entry.js.
  • Relies only on Node public API.
  • More flexible than NODE_PATH.

It's prototyped here: relatively. There's a link to a small demo project in the README.

I think this is most similar to the app-module-path approach, but while that relies on Node internals, this relies only on public API (although it needs better documentation): require.extensions.

@mlippens

This comment has been minimized.

mlippens commented May 19, 2016

I really don't know what the best solution is here. It seems it's still not solved without an extra dependency/hack being used :(. I'm inclined to use the node_modules symlink only if it were working in windows. It seems the most sane solution to me. Context: I am using babel, webpack to build a react-redux application.

Looking into the babel plugin which is nice, but then IntelliJ will not provide most of its functionality anymore when relatively requiring using ~...

@tuxsudo

This comment has been minimized.

tuxsudo commented May 23, 2016

Why not?

project/
    src/
        node_modules/    <-- project specific
            local-lib1.js
        routes/
            index.js
        app.js
    node_modules/       <-- global lib...
        bluebird/
    readme.md
@ainthek

This comment has been minimized.

ainthek commented May 24, 2016

tuxsudo: now you need also test folder, normally test is sibling of src and test may also require project specific local libs

@Prozi

This comment has been minimized.

Prozi commented May 29, 2016

This is great!
https://github.com/DSKrepps/requireFrom

I had in:
directory/index.ts

let myLib = require(__dirname + '/../../...');

in webpack and it did work on compile time
but couldnt find on runtime

then I switched to dskrepps script and it works thx!

@velopert

This comment has been minimized.

velopert commented Jun 24, 2016

Awesome!!

@devildeveloper

This comment has been minimized.

devildeveloper commented Jun 27, 2016

what about run npm post install script and copy your modules to node_modules and use that as npm module?

@protometa

This comment has been minimized.

protometa commented Jun 28, 2016

You can put them in node_modules/app, but if you'd like to be even more clear about the namespace (there is a module called app out there) you can use something like node_modules/_local and require('_local/whatever'). Then you don't have to worry about the namespace because npm module names cannot start with an underscore. It sorts nicely as well.

@jordanh

This comment has been minimized.

jordanh commented Jul 4, 2016

Very similarly to what @jshanson7 suggested, if you are using babel-register you can use a method similar to this without adding the dependency on babel-resolver:

const path = require('path');
const resolve = require('resolve');
require('babel-register')({
  resolveModuleSource(source, filename) {
    return resolve.sync(source, {
      basedir: path.resolve(filename, '..'),
      extensions: ['.js'],
      moduleDirectory: [
        path.join(__dirname, '..', '..'),  // application root
        path.join(__dirname, '..', '..', '..', 'node_modules'),
      ]
    });
  }
});

All credit to @mattkrick.

@eggmatters

This comment has been minimized.

eggmatters commented Jul 5, 2016

As a variant of the Module approach, I create a module which acts similar to a bootstrap module / environment loader. Like in a Rails app, it:

  • defines environment and dependencies
  • loads environment specific configurations
  • determines a 'docroot' path for the application.
  • sets up logging

This module as well as application specific shared libraries (api connection classes, custom validators and parsers) all are created as node modules and stored locally in a "libs" directory:

  my-node-app
  src/
     some_script.js
      somedir/
         some_other_script.js
  configs/
     dev-config.js
  libs/
     environment.js
   package.json

A simple, sample environment script would be:

var environment = "dev";
 env = {
   appname: "my-nodeapp",
   environment  :  environment,
   script_path : process.argv[1],
   config : environment + "_config.js",
   docroot : traverse script path to appname.
  //initialize loggers, load configs
 };
 module.exports = env;

add this and other shared code in the libs directory as standalone node modules (each in a directory with a package.json file"

In the application's package.json, add your local libs:

 dependencies : {
   . . .
   "env" : "file: ./libs/environment.js",
 }

running npm install at the app level will load your local libs for any script running underneath into a node_modules directory in the application.

@Sarruby

This comment has been minimized.

Sarruby commented Jul 12, 2016

Because require.main was not index.html in my node-webkit app when running mocha tests, it threw errors left and right about not being able to resolve modules. Hacky fix in my test-helper.js (required first thing in all tests) fixed it:

var path = require('path')
require.main.require = function (name) {
  // navigate to main directory
  var newPath = path.join(__dirname, '../', name)
  return require(newPath)
}

This feels wrong, though it worked. Is there a better way to fix this? It's like combining some of the above solutions with #7 to get mocha testing working, but modifying main's require just to make everything work when testing feels really wrong.

@dchekanov

This comment has been minimized.

dchekanov commented Jul 21, 2016

#3 "modifies the internal Module._nodeModulePaths method to change how the search path is calculated for modules at the application-level" (author's comment). If #6 is considered hacky, #3 should be marked as such too.

@kuba-orlik

This comment has been minimized.

kuba-orlik commented Jul 31, 2016

My solution of choice was require.main.require, but it turned out that it didn't work once my package was required by another package.

To fix that, I decided to create my own solution: locreq. It searches for the package.json up the directory structure, assumes that to be the root of the project and resolves the relative path with that root in mind. It works fine! :)

var locreq = require("locreq")(__dirname);
var moduleA = locreq("lib/my-modules/moduleA.js");
  • it works even if your package is required by a different package (which is not the case for the require.main.require trick);
  • it doesn't mess with the global scope;
  • it doesn't need changes in environment variables;
  • it doesn't need any additional start-up scripts;
  • it doesn't overwrite the default require behavior.

All feedback is welcome! :)

https://github.com/sealcode/locreq

@arkadiusz-wieczorek

This comment has been minimized.

@benatkin

This comment has been minimized.

benatkin commented Aug 10, 2016

locreq looks good but it adds an extra line, so it's a matter of choosing between:

var _root = '../../../'; // this file differs based on the level of the file, has to be changed when things are moved up and down a level
var Pet = require(_root + 'models/pet');

and

var locreq = require('locreq');
var Pet = locreq('models/pet');
@cecilemuller

This comment has been minimized.

cecilemuller commented Sep 3, 2016

Instead of all of those verbose solutions, you can also try putting your code in /src/node_modules and let Node's own paths resolution handle it, e.g. /src/node_modules/components/hello/index.js can call /src/node_modules/components/world/index.js using require('components/world') without any hack or config or symlink. And if it's not found in /src/node_modules, it will look for it in the root node_modules.

And if you don't want to call the module entry index.js, you can add a package.json with a "main" property that points to the file.

The word "node_modules" does not imply "thirdparty NPM modules", it's just how Node resolves paths (hence why NPM uses it).

@mscdex

This comment has been minimized.

mscdex commented Sep 13, 2016

Another solution would be to walk up the module.parent chain until !module.parent, then you know you've hit the "main" module and can get the path from its filename. For example, here's implementing it as a getter that you can require():

var dirname = require('path').dirname;
Object.defineProperty(module, 'exports', {
  get: function() {
    var m = module;
    while (m.parent)
      m = m.parent;
    return dirname(m.filename);
  }
});
@kuba-orlik

This comment has been minimized.

kuba-orlik commented Sep 15, 2016

@mscdex this is only going to work if you're creating a top-level app that is not required by other modules

@wmhilton

This comment has been minimized.

wmhilton commented Sep 23, 2016

+1 what @cecilemuller said. Took me a year of flailing before I figured it out, but the idea is simple once you grok it:

  1. Put your own modules in a directory called "node_modules" somewhere.
  2. Put third party modules you require in a "node_modules" directory in a parent directory of your code.
  3. That's all. It just works for the most part. You can shadow 3rd party modules, have private modules, whatever you want and require them just like you require 3rd party modules.
./my-project
./my-project/package.json
./my-project/node_modules/(third party modules go here)
./my-project/app/index.js
./my-project/app/node_modules/(your modules go here, add their dependencies to the main package.json)

Lengthier explanation at https://github.com/wmhilton/my_modules

Edit:
I'd like to point out that I think this is what @isaacs and @substack were trying to hint at, back in 2014. Not put your git revisioned modules in the same 'node_modules' folder that gets clobbered by "npm install", but put them in A DIFFERENT folder that is also called 'node_modules'.

Edit 2:
This is exactly the strategy that @tuxsudo proposed, except instead of "app" he used "src".

@basickarl

This comment has been minimized.

basickarl commented Sep 23, 2016

Add a 4.3 section:

4.3. Only when executing through package.json scripts

Like 4.2, this solution will not affect your environment other than what node preceives. Must be started via npm.

In your package.json file:

{
    ...
    "scripts": {
        "start1": "NODE_PATH=\"$(dirname $(readlink -f ./package.json))\" node index",
        "start2": "NODE_PATH=\"$(pwd)\" node index"
    },
    ...
}
@cecilemuller

This comment has been minimized.

cecilemuller commented Nov 4, 2016

Setting an env variable directly before a command is not a cross-platform syntax, it won't work on Windows.

@centigrade-thomas-becker

This comment has been minimized.

centigrade-thomas-becker commented Nov 14, 2016

@cecilemuller: It won't work if you use platform-specific syntax, but cross-env might be a way around this.

@F1LT3R

This comment has been minimized.

F1LT3R commented Nov 21, 2016

@substack, @isaacs, if you put your libs in node_modules and you're using XO/ESlint, you get the warning import/no-extraneous-dependencies when using require('app/dep'), because "app should be listed in the project's dependencies".

Should we then be adding the package to the dependancies or turning that rule off for our whole project?

(I'm assuming adding a symlinked node_modules/app => ./lib dir to local dependancies in thepackage.json is going to do bad things.)

@andurilan

This comment has been minimized.

andurilan commented Dec 4, 2016

@joelabair is the only solution worth considering for its shortness and implementation right before anything has been loaded.Even on Windows, where it kills 2 birds with 1 stone (Path limits), it worked wonders with cross-env, Great job.

@koshuang

This comment has been minimized.

koshuang commented Dec 4, 2016

If you are using babel, I would recommend to use babel-plugin-resolver.

You can make multiple folders to resolve the path. So my test imports can be like:

import UserModel from 'models/User'; // => resolves: "src/app/models/User.js"
import userHelper from 'helpers/userHelper'; // => resolves: "src/app/helpers/userHelper.js"
import testHelper from 'helpers/testHelper'; // => resolves: "test/helpers/testHelper.js"

with .babelrc file

{
  "plugins": [["resolver", {"resolveDirs": ["src/app", "src", "test", "."]}]]
}
@dschinkel

This comment has been minimized.

dschinkel commented Dec 4, 2016

@mikeal you said:

yeah, whenever i see '../../../dir/name' i immediately think that someone has either 1) prematurely broken out their app in to a million directories and file or 2) hasn't modularized these components in to modules yet, and they should.

What do you mean? For example lets say I have the following structure:

root

            `-- src
                |-- client
                |   `-- components
                        `-- companies
                |           `-- CompanyList.js
                `-- utilities.js

CompanyList.js has in it:

import utilities from '../../../utilities'

so why would something like that be a red flag to say one hasn't broken stuff into modules or has prematurely broken stuff out? Note: I have other files in the companies directory, not just one?

@liuchong

This comment has been minimized.

liuchong commented Dec 20, 2016

Linux: ln -nsf node_modules app

Isn't it should be "ln -nsf ../app node_modules"? 😹

@jsykes

This comment has been minimized.

jsykes commented Dec 22, 2016

I found it handy to put it in the package.json scripts area. Where the /dist/src folder is where all my Typescript compiled javascript goes.

Now I can do things like import { DatabaseService} from 'system/core'

Where the file is dist/src/system/core/database.service.js
Of course there is an index.ts file exporting things in the dist/src/system/core folder.

"scripts": {
"start_osx": "export NODE_PATH=./dist/src/&& ../engine/osx/bin/node ./dist/server.js",
"start_win": "set NODE_ENV=&& ../engine/win/bin/node ./dist/server.js"
}

Then you can launch custom scripts with
npm run start_osx

@necrifede

This comment has been minimized.

necrifede commented Jan 15, 2017

I've been using the first option on windows. however admin privilege is necessary.
To avoid to use admin privileges I did little updates on @scharf's code.

// the src path relative to node_module
var srcpath = '../app';
var dstpath = __dirname + '/node_modules/app';
var fs = require('fs');
fs.exists(dstpath, function (exists) {
    // create the link only if the dest does not exist!
    if (!exists) {
        fs.symlinkSync(srcpath, dstpath, 'junction');
    }
});

The __dirname on second line and use 'junction' instead 'dir' parameter for fs.symlinkSync function.

Hope it'll help someone.

@alexsorokoletov

This comment has been minimized.

alexsorokoletov commented Apr 14, 2017

Thank you for this package, @branneman!
As @orchidlegba said, VS Code doesn't understand this and looses all the intellisense features (code autocompletion). Wondering how do we handle that?

@jpickwell

This comment has been minimized.

jpickwell commented Apr 24, 2017

There's a small error with #5. ./app and ./app.sh are two separate files on *nix.

On Windows, there's a PATHEXT environment variable which indicates which extensions are executable. The value of PATHEXT on my Windows 10 box is .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC. If you have two files, app.com and app.exe, calling app will execute app.com. (https://en.wikipedia.org/wiki/Environment_variable#Default_values) (https://www.nextofwindows.com/what-is-pathext-environment-variable-in-windows)

*nix doesn't have this system built-in; I'm sure there's something out there that can add this feature to a *nix box.

@Gaafar

This comment has been minimized.

Gaafar commented May 2, 2017

I wrote this small package for this reason
https://github.com/Gaafar/pkg-require

It works like this

// create an instance that will find the nearest parent dir containing package.json from your __dirname
const pkgRequire = require('pkg-require')(__dirname);

// require a file relative to the your package.json directory 
const foo = pkgRequire('foo/foo')

// get the absolute path for a file
const absolutePathToFoo = pkgRequire.resolve('foo/foo')

// get the absolute path to your root directory
const packageRootPath = pkgRequire.root()

The main point is that it does not mutate any globals or introduce any side effects.

@oknoorap

This comment has been minimized.

oknoorap commented Jun 2, 2017

@abc-z

This comment has been minimized.

abc-z commented Jun 21, 2017

Another solution is use path.resolve.

var path = require('path');
var article = require(path.resolve(process.cwd(), 'include/article')); // or path.resolve('include/article')
@sultan99

This comment has been minimized.

sultan99 commented Aug 5, 2017

I can offer my solution: sexy-require
npm i sexy-require -S

@vincent-ly

This comment has been minimized.

vincent-ly commented Aug 23, 2017

Will using method #4 have any noticeable performance bottlenecks, since it has to look into node_modules before attempting to resolve with NODE_PATH?

@hyphaene

This comment has been minimized.

hyphaene commented Aug 24, 2017

A simple solution is to create: a .env file and add NODE_PATH=. inside of it !

@joelbarbosa

This comment has been minimized.

joelbarbosa commented Sep 21, 2017

Now you can resolve with:
"start": "NODE_PATH=src/ nodemon scripts/start.js --exec babel-node",
"build": "NODE_PATH=src/ node scripts/build.js",
"test": "NODE_PATH=src/ node scripts/test.js",

@jez9999

This comment has been minimized.

jez9999 commented Oct 1, 2017

I created a npm module to solve this problem, requirebase. It is located here:
https://www.npmjs.com/package/requirebase

Usage looks like:

const requirebase = require("requirebase")("base-module-name");
const utilities = requirebase("lib/utilities");
const parser = requirebase("lib/parser");

It works perfectly by locating package.json, and requiring everything relative to that directory. Because requirebase itself is an npm module, it can be safely required from the node_modules directory.

Unfortunately, I've now moved on to TypeScript and none of this will work with that because it needs to determine the base package.json path at runtime. :-(

@jez9999

This comment has been minimized.

jez9999 commented Oct 2, 2017

I think I've come up with a solution, and it even works with TypeScript too because TypeScript uses the same mechanism for locating modules as Node.js: have a second node_modules directory for your modules lower down than the main npm node_modules directory, and Node.js/TypeScript will check that one first as they check up the file tree for node_modules directories when you do a non-relative import.

The source code will be in this structure:

/node_modules
  /...
/package.json
/src
  /node_modules
    /mymodules
      /client
        /foo.js
      /server
        /bar.js
      /shared
        /baz.js
  /scripts
    /init.js

Just put all your module code under your node_modules directory in src! Then you can reference modules from any code file underneath the src directory (eg. init.js) using:

import myClientModule from "mymodules/client/foo";

@alexgorbatchev proposed something similar above, but I'm not sure why he created symlinks under node_modules, which complicates things, instead of just directly putting the code there. Since it's a separate node_modules directory from the npm one, you can check it in to source control and just put your local modules in there.

@aichholzer

This comment has been minimized.

aichholzer commented Oct 12, 2017

Here's my take on this issue: https://github.com/aichholzer/attract
👍

@fdorantesm

This comment has been minimized.

fdorantesm commented Oct 29, 2017

I did asapp to avoid require sentence, check it in http://github.com/fdorantesm/asapp

@jpillora

This comment has been minimized.

jpillora commented Oct 30, 2017

//activate the require hook with the provided base dir
const activate = base => {
  const Module = Object.getPrototypeOf(module);
  if (Module.require.hooked) {
    return; //already hooked
  }
  const fs = require("fs");
  const path = require("path");
  //if not provided, default to dir with .git
  if (!base) {
    base = __dirname;
    while (base !== "/" && !fs.existsSync(path.join(base, ".git"))) {
      base = path.join(base, "..");
    }
  }
  let original = Module.require;
  Module.require = function(mod) {
    //convert @ requires
    if (mod.indexOf("@") === 0) {
      mod = path.join(base, mod.slice(1));
    }
    return original.call(this, mod);
  };
  Module.require.hooked = true;
  console.log("[require] @requires are now relative to", base);
};
activate("/a/dir/of/my/choice");
//or
activate(); //first dir up from __dirname with a .git subdir
//then
require("works-as-normal");
require("./also-works-as-normal");
require("@special/case"); //becomes <base>/special/case
@IUnknown68

This comment has been minimized.

IUnknown68 commented Nov 2, 2017

Here is another one: https://www.npmjs.com/package/require-rewrite .

Features:

  • aliases
  • include paths before and after node's paths
  • regular expression match / rewrite
  • custom rewrite functions
  • package aware (separation between packages)
  • global rewrites
@caub

This comment has been minimized.

caub commented Nov 3, 2017

@defunctzombie it works now, npm i ../../foo/bar (--save/-S is no longer needed)

@psaeuerl

This comment has been minimized.

psaeuerl commented Nov 21, 2017

actually, https://www.npmjs.com/package/app-module-path solved those issues for us

@stevethedev

This comment has been minimized.

stevethedev commented Jan 6, 2018

I solved this problem in my projects with Kraeve. Kraeve moves from the top-level application folder to search for a package.json folder and then uses the name property as an alias for the application's path. This is specifically built to be non-intrusive, so it can be included as a dependency in other projects without breaking things. The goal is that the system behaves as if there were a symlink in the node_modules folder.

Since this doesn't require any specific configuration or knowledge of how the system behaves, and it uses the same signature that a module would be if it were installed as a dependency of another project, Kraeve doesn't interfere with parent projects (except for the side-effect of that parent project also being able to require() itself).

// This is just what a require() looks like, no matter what.
require('my-app/foo/bar/baz');
@Manikanta-Gandham

This comment has been minimized.

Manikanta-Gandham commented Mar 9, 2018

I have a question,
I have a npm module which i would like to integrate with the the JS code of current project. Finally I want that my maven build of the current project to have JS code from node modules.

Is web pack a solution?

@ORESoftware

This comment has been minimized.

ORESoftware commented Apr 26, 2018

if package.json allowed for something like webpack, with something like "map" or "paths":

{
  "name": "foobar",
  "dependencies": {
     "lodash":"1.2.3"
  },
  "paths": {        
     "fizzbizz": "lib/fizz/bizz.js"       
   }
}

now we can just require fizzbizz directly

const fizzbizz = require('fizzbizz');

alternatively call "paths" "map" instead, or whatever.

maybe NPM already allows this, for all I know.

@7elven

This comment has been minimized.

7elven commented Jul 4, 2018

try abrequire No config, and very easy
https://www.npmjs.com/package/abrequire

@LiamKarlMitchell

This comment has been minimized.

LiamKarlMitchell commented Sep 26, 2018

Still a hack as it relies on module.paths being editable and existing? (Although module.paths is documented?)
In main entrypoint script.

module.paths.unshift(path.resolve("./modules"));

In some other js file.

const Thing = require.main.require(
  "YourProject/Thing"
);

Where ./modules/YourProject/Thing.js is a module you want to load in.
VSCode intelisense seems to be working for me with this approach, that could be because the files are in my project workspace however.

@uniibu

This comment has been minimized.

uniibu commented Oct 21, 2018

Just to update this gist conversation, the new api createRequireFromPath has just landed on NodeJs v10.12.0 which basically can be used for this specific task.

@heisian

This comment has been minimized.

heisian commented Dec 1, 2018

symlinks work with git.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment