Skip to content

Instantly share code, notes, and snippets.

@lancejpollard
Created November 28, 2011 01:50
Show Gist options
  • Save lancejpollard/1398757 to your computer and use it in GitHub Desktop.
Save lancejpollard/1398757 to your computer and use it in GitHub Desktop.
What is your folder-structure preference for a large-scale Node.js project?

What is your folder-structure preference for a large-scale Node.js project?

0: Starting from Rails

This is the reference point. All the other options are based off this.

|-- app
|   |-- controllers
|   |   |-- admin
|   |   |   |-- postsController.js
|   |   |   `-- usersController.js
|   |   |-- postsController.js
|   |   |-- sessionsController.js
|   |   `-- usersController.js
|   |-- models
|   |   |-- post.js
|   |   `-- user.js
|   |-- views
|   |   |-- admin
|   |   |   `-- posts
|   |   |       |-- edit.jade
|   |   |       |-- index.jade
|   |   |       |-- new.jade
|   |   |-- layouts
|   |   |   `-- application.jade
|   |   `-- posts
|   |       |-- index.jade
|   |       `-- show.jade
|   `-- helpers
|       |-- admin
|       |   |-- postsHelper.js
|       |   `-- tagsHelper.js
|       `-- postsHelper.js
`-- config
|    |-- application.js
|    |-- locale
|        `-- en.js
|    |-- routes.js
`-- lib
`-- spec
|    |-- helper.js
|    |-- models
|    |   |-- postSpec.js
|    |   |-- userSpec.js
|    `-- acceptance
|        |-- loginSpec.js
|        |-- signupSpec.js
|        `-- postsSpec.js
`-- vendor
|    |-- javascripts
|    |   |-- jquery.js
|    |   |-- underscore.js
|    `-- stylesheets
|        `-- prettyPhoto.css

1: Rails-like, with nested /app folders for client, mobile, etc.

|-- app
|   |-- controllers
|   |-- models
|   |-- views
|   `-- browser
|       |-- controllers
|       |-- models
|       |-- views
|   `-- mobile
|       |-- controllers
|       |-- models
|       |-- views
`-- config
`-- lib
`-- spec
`-- vendor

Pros:

  • lends itself great to progressive enhancement so-to-speak. You start with just /app, and if your app starts growing, you add sub directories.
  • doesn't pollute the top-level directories, which have a pretty uniform convention across apps. However, people do add /themes to the top level, which is just about the same as adding /client to the top-level.

Cons:

  • Now you have /app/models and /app/browser, etc., which isn't a totally clear naming convention -- /app/models is for a subset of code for the server, while /app/browser is a totally different app. It's different than a namespace like /app/models/admin though, which makes sense.

My vote: no

2: A /app/client folder, similar to Rails' /app/assets

|-- app
|   |-- controllers
|   |-- models
|   |-- views
|   `-- client
|       `-- browser
|           |-- controllers
|           |-- models
|           |-- views
|       `-- mobile
|           |-- controllers
|           |-- models
|           |-- views
`-- config
`-- lib
`-- spec
`-- vendor

Pros:

  • The main reason you need these extra folders is for the different clients for the app. So putting them in /app/client conceptually makes a lot of sense. It's easy to reason about.
  • Similar to Rails, which has /app/assets/javascripts instead of /app/client. You don't want to start naming the folder /app/assets because, conceptually, everything is JavaScript, and calling one chunk of JavaScript "assets" and the rest "app" is conceptually jarring.

Cons:

  • You have deeply nested folders for core code, which can be annoying. /app/client/browser/controllers/postsController.js is 4 folders down. But with TextMate and CMD+T, it shouldn't be an issue.

You could also have this structure if you only had 1 client (or just a default client):

|-- app
|   |-- controllers
|   |-- models
|   |-- views
|   `-- client
|       |-- controllers
|       |-- models
|       `-- views

That's pretty clear, and it lends itself to agile development really well.

My vote: ✔

3: More top-level folders

|-- app
|   |-- controllers
|   |-- models
|   |-- views
|-- browser
|   |-- controllers
|   |-- models
|   |-- views
|-- mobile
|   |-- controllers
|   |-- models
|   |-- views
`-- config
`-- lib
`-- spec
`-- vendor

Pros:

  • Minimum folder nesting
  • model/view/controller folders are all at the same level

Cons:

  • Having multiple top-level folders, all of which are mvc code, is not conceptually clear. They should be part of one directory (taking us back to #2). Having every folder at the top level be a completely conceptually distinct part of the app (database vs. app vs. config vs. tests) is a clarifying convention.

My vote: second choice, but no

4: Namespaces inside /app

|-- app
|   `-- client
|       |-- controllers
|       |-- models
|       |-- views
|   `-- mobile
|       |-- controllers
|       |-- models
|       |-- views
|   `-- server
|       |-- controllers
|       |-- models
|       |-- views
`-- config
`-- lib
`-- spec
`-- vendor

Pros:

  • Clear, normalized separation of concerns. Everything fits into the same folder structure, instead of having the "default" stuff in /app, and then also nesting components in there.

Cons:

  • For the simple case, you have to create a nested folder.
    • Counterargument: But, you're setup to easily add other clients for the app
  • By not having the default code go in the top level /app, it's not as clear that server code can be used on the client (e.g. using /app/server/models/user.js on the client. it makes sense if it's in /app/models/user.js however.).

My vote: no

5: Top-level /client and /server directories

This breaks convention, but it's an option. Rename /app to /server

|-- client
|   |-- controllers
|   |-- models
|   |-- views
|-- server
|   |-- controllers
|   |-- models
|   |-- views
`-- config
`-- lib
`-- spec
`-- vendor

Pros:

  • Clear: there is a client and server app.

Cons:

  • If you had multiple client apps (browser, mobile, ipad, etc.), you'd end up doing things like #2 or #3.

My vote: no

Which do you prefer?

I prefer folder structure #2.

@maedi
Copy link

maedi commented Aug 24, 2012

Oh and I do really like the concept of behaviors, this is something I use currently in a Rails MVC. It doesn't have to be baked in by default but would be nice if it was... complex MVC apps can be messy without them.

Widgets are a great Rails concept too, just thinking it might be an 'add-on' and not in Tower core. I'm not a core Dev though so this is just thinking aloud.

@autoferrit
Copy link

Yes regarding multisites, I agree it would be much more advanced. And I can easily see not wanting to include it because of that. In my last comment I was mainly discussing what others are doing and how the similar * could * be done in tower if it was decided to do so. Realistically, a multisite system would probably be more along the lines of something a cms or other system would do to build on top of this (this is also something I am trying to build myself. it is part of a requirement for an app I am building where multiple sites will share much of the same server side code and each site will have a different subset of functionality).

Plugins, yes these would ideally be npm modules that are made to be compatible with whatever system is built. Since the platform I am making will have a simple core and have its own plugins to enhance it, which will be either included with the core and enabled by a setting, or installable from npm. I think the key there is to keep the node/npm standards and make everything follow what people are familiar with.

uploads, database, etc could all be configured by a config/datasources/coffee file. In there you would define and source to save or pull data from. You can define that you shall either use uploads, json files, amazon s3, rackspace files (which on a rackspace server you can tie to a local directory i believe on a VPS), or whatever you want. those other options could each be an npm module defining how to connect and uses the api, and in the datasources.coffee file you would tell your app about that datasource module, and its credentials to pull/save data from/to it. Also this would obsolete the databases.coffee file since they would be defined as a data source, which is kinda what it is.

When you have a model, by default they use your default data source (usually mongo), but you could also have a datasources variable that would define what sources it uses. For example, User model could have something like

var datasources = ["mongodb", "facebook", "twitter"]

and a page model:

var datasources = ["mongodb", "s3"]

an file/upload model:

var datasources = ["uploads"]   // the datasource config would define where the uploads dir actually is.

I think behaviours/components would be great too. there tends to be code commonly shared between some models/controllers.

Yea I agree widgets could be optional too. But to me, a widget really would just be a special way to use a template. and anything beyond that would be up to the user to define, unless there is something that would be beneficial to have in core.

@autoferrit
Copy link

Another option is similar to how django does it. it would still be mvc but the methodology on the stricture is a lot different. mostly related to the actual models, views, and controllers. So imagine the other directories here are the same. tmp, config, node_modules, plugins, vendor, etc. rather than:

|-- controllers
|   |-- fooController.coffee
|   |-- barController.coffee
|-- models
|   |-- foo.coffee
|   |-- bar.coffee
|-- views
|   |-- layout
|   |   |-- content.coffee
|   |   |-- header.coffee
|   |-- foo
|   |   |-- index.html
|   |   |-- _form.html
|   |-- bar
|   |   |-- index.html
|   |   |-- _form.html

we would have this

|-- apps
|   |-- foo
|   |   |-- fooController.coffee
|   |   |-- foo.coffee
|   |   |-- views
|   |   |   |-- index.html
|   |   |   |-- _form.html
|   |   |-- templates        // templates specific to the foo 'app'
|   |   |   |-- widget.coffee
|   |-- bar
|   |   |-- barController.coffee
|   |   |-- bar.coffee
|   |   |-- views
|   |   |   |-- index.html
|   |   |   |-- _form.html
|   |   |-- templates        // templates specific to the bar 'app'
|   |   |   |-- widget.coffee
|   |-- themes          // the normal app wide templates ( i still like the themes idea though)
|   |   |-- myTheme
|   |   |   |-- content.html

so rather than a folder for each of controllers, models, and views, we essentially have a mini 'app' for each part of functionality. so in the normal 'app/' dir, would be an independant app, or a few that maybe use/depend on each other. Doing this helps separate different parts of the app code from others. for example, I could create a 'comments' app that I can use for other apps. and it would be easier to just synlink that dir or copy code of one directory, than individual files. it forces you to make each app independent and testable on its own.

I really like this idea from a testing standpoint as well. it also makes the app easier to separate different jobs or tasks that it does. when you have something like a behaviour( for models), or components (for controllers) they can either be in their own app subdirectory, or in a special app/components, or app/behaviours directory.

@j
Copy link

j commented Aug 9, 2013

So, this may be a dumb question, but do you consider "client" code to be what the device sees? Aka, are the models something like Backbone models, and views Handlebars, etc? Or are those just ways to organize "multi-site" applications.

Lets say I have a website, mobile website, and a REST API. I want to share all the same domain models between all of these. Each one will have a different subset of "controllers|routes", "views" and more domain models, etc. Maybe even utility files / middlewares for each. So at first glance, the client namespace would be a decent place to organize multisite style applications.

Meaning... each "client" in "app" would literally be a separate instance of express() or restify() (on the API), and the root app.js or server.js would configure each accordingly.

OR, should I create separate repositories/projects for this, then create a main one too bootstrap them and bring them together.

i.e) I have a company with domain widgets.com which has an api, mobile version, and main site. I could have the following "repositories" / project names and be configured/packaged privately in NPM:

1) widgets-common:  this will be a library containing common models / utilities / middlewares / etc that all projects may share.
2) widgets-browser: typical "express" app
3) widgets-mobile:  typical "express" app
4) widgets-api:     typical "express" or "restify" app
5) widgets.com:     main repository which installs above dependencies and bootstraps them / starts servers / vhosts / etc, etc

Anyway, what do you think?

@mb-dev
Copy link

mb-dev commented Dec 13, 2013

I would recommend to watch this video:
http://confreaks.com/videos/759-rubymidwest2011-keynote-architecture-the-lost-years

I am considering grouping app by functionality:
so instead of the usual controllers/models/assets

There will be:

|-- app
|   |-- common
|   |   |-- common.coffee
|   |   |-- common.less
|   |-- users
|   |   |-- users.coffee
|   |   |-- users.less
|   |   |-- login.html
|   |   |-- logout.html
|   |   |-- register.html
|   |-- posts
|   |   |-- posts.coffee
|   |   |-- posts.less
|   |   |-- show.html
|   |   |-- form.html
|   |-- blog
|   |   |-- blog.coffee
|   |   |-- blog.less
|   |   |-- index.html

Pros: Like the video lists: your app functionality can now be learnt by looking at the top level structure. All files that activate a section on the site are right next to each other.

What are the cons?

Note: I included client side files only in the example, still trying to decide if I should tie in server side files too.

@vjpr
Copy link

vjpr commented Jan 18, 2014

@mb-dev I am settling on a similar approach. Grouping by functionality is much easier to work with.

@flemay
Copy link

flemay commented Jan 21, 2014

@mb-dev and @vjpr: where do you put your server files in that structure?

@koistya
Copy link

koistya commented Feb 4, 2014

@vanng822
Copy link

don't have big project but here is the structure I use and works really well for me, more or less option 3
https://github.com/vanng822/appskeleton

@fidanov
Copy link

fidanov commented Oct 10, 2014

I prefer something very similar to your first example (the rails inspired). Here is a tutorial about it and below is an example of an app applying it.

|--project
|  |--controllers/
|  |  |--comments.js
|  |  |--index.js
|  |  |--users.js
|  |  helpers/
|  |  |--dates.js
|  |--middlewares/
|  |  |--auth.js
|  |  |--users.js
|  |--models/
|  |  |--comment.js
|  |  |--user.js
|  |--public/
|  |  |--libs/
|  |  |--css/
|  |  |--img/
|  |--views/
|  |  |--comments/
|  |  |--|--comment.jade
|  |  |--users/
|  |  |--index.jade
|  |--tests/
|  |  |--controllers/
|  |  |--models/
|  |  |  |--comment.js
|  |  |--middlewares/
|  |  |--integration/
|  |  |--ui/
|  |--.gitignore
|  |--app.js
|  |--package.json

I've also prepared a small repository on GitHub, which uses the same structure, for any new projects. I will try keep its dependencies up to date.
https://github.com/terlici/base-express

@ssmereka
Copy link

I prefer to group my files by feature, rather than function. So for example group all user related files (controllers, models, tests, and etc.) in a folder. Then that folder can be easily reused in other projects or when making changes it is easier to find the files affected. So for example:

I created a module to allow this type of file structure by dynamically requiring files.
https://github.com/ssmereka/crave

@dschinkel
Copy link

Definitely everyone (even seasoned devs) should first watch Uncle Bob's vid as suggested by mb-dev. Talks about how your top level should not indicate what kind of app it is (Rails, CoffeeScript, etc.). The language shouldn't dominate your project structure at a top level. http://confreaks.tv/videos/rubymidwest2011-keynote-architecture-the-lost-years

@emadb
Copy link

emadb commented Aug 3, 2015

I don't like categorizing files based on their framework role (ie Controllers, Views, ....) but I like divide my files based on their functionality.
So, for example, in an ecommerce app, I end up having a folder all the files of the products, another for the basket management, another for the warehouse and so on.
In this way every modules of the application is separated from the others and it's easier maintain them or even rewrite if necessary.

|--app
|  |--orders/
|  |--basket/
|  |--warehouse/
|  |--common/
|  |--tests
|  |  |--basket/
|  |  |--warehouse/
|  |  |--common/
|  |  |--users.js
| --gruntOrGulp.js
| --README.md

@jpierson
Copy link

As an experienced developer who is new to NodeJS I've been hunting for a reasonable organization structure myself. From my experience working a bit with AngularJS I do prefer grouping by functionality over roll in the framework as others such as @emadb have mentioned. I also have had particular difficulty in parsing which code belongs to the build process, the server side, and the client side parts of the application.

@rkt2spc
Copy link

rkt2spc commented Apr 5, 2017

Grouping files by feature become complicated when many reusing happen. What if 2 features required the same model, in which component should I put the model definition?

@mareksuscak
Copy link

@rocketspacer at that moment you lift it up and make it a shared/common concern

@Randy808
Copy link

Randy808 commented Sep 2, 2017

@autoferrit I see where you put "I vote to have an en[v]ironment variable of some sort for each directory which is an array of paths." but still don't see how it would actually be used. You only separate sites by views, so where would you be able to swap out a models directory? Is it in the server file that a request first hits (the file that includes all the controllers, server config, etc)? Also, do you determine who the client is in that server file? I also don't understand how all the sites would share the same public folder unless each site has their own controller within the app subdirectory of the site. Is that what's happening? I'm fairly new to API architectures and would appreciate any clarification, thanks!!

@demisx
Copy link

demisx commented Sep 18, 2017

+1 for @emadb's organization. Feature based structure is the way to go.

@inancgumus
Copy link

@rocketspacer That means, there's a new concept waiting to go into its own package. And then, separate the dependencies via abstractions.

@rkt2spc
Copy link

rkt2spc commented Jan 30, 2018

@mareksuscak @inancgumus I can't really make a solid project structure out of a vague emerging concept simply because I don't know how to properly name it. How common is common, what should I call the abstraction of login and transaction?
We decided to adopt Microservices and solve this problem at a different level now.

@alamenai
Copy link

@lancejpollard ,what we put in 'spec' folder ?

@nbaua
Copy link

nbaua commented Mar 13, 2019

@lancejpollard ,what we put in 'spec' folder ?

Test cases

@bomzj
Copy link

bomzj commented May 8, 2019

Feature Folders is the way to go for any size of project!

@danmandel
Copy link

I don't think there's any optimal answer between grouping by function vs grouping by feature, it seems to just come down to personal preference. It would be nice if there was an IDE setting or global tool that devs could install to make swapping between folder structures as easy as a 1-line change in a config file.

@mytharcher
Copy link

mytharcher commented Sep 9, 2019

My principal:

  • Any folder should match the concept of dimension, so /<feature> and /test should not be placed under same folder.
  • Node.js project should not combine both client and server codes, unless you need SSR, so I will not need to concern about the mixed project folders.

And my further consideration is that sometime (I can say more than half) my services code can't always match to the model (database tables) / controller (routes). Because controller sometimes needs to handle webhooks which not match to model, and services may also for third-party integration logics. So I prefer the function rather than feature.

While the architecture will evolve into micro services in future, function layer first will not become more complex.

Something I am confused is when I divide the function, how my main app logic folder should be named? Like src / lib / main / app? Another confusion is should I just place the main logic layer folders under project root? Such as controllers / services / model / test ... etc.

I have read about the guideline of golang project layout, and agreed with their idea about the /src:

Some Go projects do have a src folder, but it usually happens when the devs came from the Java world where it's a common pattern. If you can help yourself try not to adopt this Java pattern. You really don't want your Go code or Go projects to look like Java :-)

So my practice now has been summarized into a repository here: express-bootstrap. Any discussion or idea will be welcome.

@lancejpollard
Copy link
Author

Here is my current system.

@vish9812
Copy link

vish9812 commented Feb 28, 2020

Feature based structure seems the way to go!

Here is my app

|--src 
|  |--core  
|  |  |--orders
|  |  |--basket
|  |  |--common
|  |--infrastructure
|  |  |--auth/
|  |  |--logger/
|  |--utils
|  |  |--config
|  |  |--crypto
| --package.json
| --index.js
| --README.md

@nashid
Copy link

nashid commented Jun 24, 2020

@Vishch - you don't give a clear separation of tests. All the tests go to the tests folder?

@vish9812
Copy link

vish9812 commented Jun 25, 2020

@nashid yes the tests should go in their respective feature folders

@vikrantsingh47
Copy link

@nashid yes the tests should go in their respective feature folders

should the routes also go in their respective feature folders?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment