Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

TL;DR

Meteor is great at sharing code between different builds for different platforms. You can use the same source for your browser builds, server builds, your builds for iOS, Android, ... But how to organize your project to be able to orchestrate your builds for different apps and services from the same source? This post elaborates on the reasons why you need these different builds and how you could accomplish this with Meteor easily.

Use cases: Why would you build different apps?

1. Different apps for different roles

Say you have an app with completely different end user experiences depending on their role. It is common practice to have the user logged in, check her authorization (role) and then setup different routes or load different templates to serve that type of user’s needs. While doing so, all types of users load the same build and the app decides what portions of the build to use and what not to use.

This results in builds that can be way too large for a user with a role that can actually do very few things with your app. A common example that illustrates this perfectly is a landing page. An unauthenticated user only needs to be shown the landing page, she must be able to login and register a new account, only. Worse, this approach may lead to security vulnerabilities. Because all of the logic is deployed and catered to all users (whatever their role), you must be damn sure access rights are checked all the way through, in every single aspect of your app.

Wouldn’t it then make more sense to never load any code at all for users who’ll never need it, or shouldn’t use it? Sure it would, and deploying different apps at different subdomains (yourapp.io, app.yourapp.io, admin.yourapp.io, etc.) while making sure users are redirected to using the app which fits their current role and access rights, may well be the way to go. But then you’ll soon run into maintenance headaches and will struggle with code organization.

2. Different builds for different microservices

Say you have a very computation heavy process somewhere in your backend, which you know will have scaling issues as soon as the user base is growing. You then want to be able to isolate this process from the rest of your app, by deploying it as a independent (micro)service and by scaling that service seperately, along with the user base.

Sure it’s possible to setup this process as a different project, and have your Meteor app communicate with the service as it would do with any other third-party service. (Or, better still, over Meteor’s built-in DDP since both the mother app and the microservice speak Meteor). But say you would now want to go this way and deploy a dozen different components within your backend — the burden of building and maintaining different services would grow into a very tedious process, again. Development itself would be getting harder too: you’d be forced to setup a complicated local development environment, running all of the microservices involved on different ports, and to configurate communication between them.

Wouldn’t it then make more sense to develop all those services within a single app during development, while at build time separate all the many different microservices again into stand-alone production apps, each scalable in their own right?

Organize your code in app-packages

So we want to organize our code such that we may build (deploy and scale) different apps/services from a single development source, very similar as to how one would go about when developing an app for several native platforms, only this time around targetting the Web as the platform, to which you’ll be releasing an app-network of services talking with each other.

Key to the solution is to architect your project as a collection of app-specific packages. You’ll then be able to lift-out all of the app-level Meteor code and organize it in packages. Not as in those regular packages you’re used to distribute on Atmosphere, but as in local, app-specific packages. Indeed, per package you would add a directory in your packages folder, add a package.js file to it, include the package in your app-dependencies (.meteor/packages) and you’re good to go.

example project directory structure:

/project
  /.meteor
    packages
  /packages
    /myproject:core
    /myproject:app-ui
    /myproject:app-profile
    /myproject:app-projects

example package.js - /packages/myproject:app-ui/package.js

Package.describe({
  name: 'myproject:app-ui',
  version: '0.0.1',
  summary: 'My Projects User Interface',
});

Package.onUse(function(api) {
  api.versionsFrom('1.1.0.3');
  
  api.use([
    'myproject:core',
    'myproject:app-profile',
    'templating',
    'kadira:flow-router',
    'kadira:blaze-layout'
  ], 'client');

  api.addFiles([
    'head.html',
    'layout.html',
    'components/logo.html',
    'routing.js'
  ], 'client');
});

With all of your code neatly organized in packages, you now have much more control over build order and your dependencies are more clearly stated, too. Also, you’ll benefit from naturally writing more modular code, since you start thinking about encapsulating business logic in separate packages, with a clear dependency chain in mind.

Setup umbrella packages for builds

You should also setup an umbrella package for every app you want to build in your project. An umbrella package is a package only implying other dependant packages, but not adding any logic (and thus not adding any file to the build, or using any package).

example umbrella package.js - /packages/myproject:app/package.js

Package.describe({
  name: 'myproject:app',
  version: '0.0.1',
  summary: 'My Projects User facing app umbrella package',
});

Package.onUse(function(api) {
  api.versionsFrom('1.1.0.3');
  
  api.imply([
    'myproject:core',
    'myproject:app-profile',
    'myproject:app-projects',
    'myproject:app-ui'
  ], ['client', 'server']);

});

Basically you're replacing your .meteor/packages file by a package. You're moving dependency management to an app umbrella package. Doing so, there is no logic or dependency control left at app-level, but more importantly you can start creating apps that inherit from other apps by implying other app umbrella packages. This way you can setup your apps as a cascading model of functionality: admin-app > moderator-app > user-app > guest-app > unathenticated-app.

example packages file - /.meteor/packages

meteor-platform
myproject:app

Organize your project in different meteor projects for each build

Ok, we now have organized all of our app code for reuse, but we didn't reuse it for now, because we haven't setup other apps yet.

Instead of creating a meteor project at the root of your project repository, create a meteor project in a sub-folder for every build you need (landingpage, app, microServiceA, ...).

Setup a packages directory at root level of your project, and symlink to this directory from every single app-directory.

Doing so, every app has its app-specific .meteor directory, containing all needed files to orchestrate the meteor build process (.id, packages, platforms, ...). They can run independently from other apps in your project, but still share all of your business logic, picking what is needed for each build, using your umbrella packages.

This results in the following directory structure:

/project
  /app
    /.meteor
    /packages => ../packages
  /landing
    /.meteor
    /packages => ../packages
  /services
    /.meteor
    /packages => ../packages
  /packages
    /myproject:core
    /myproject:app
    /myproject:app-profile
    /myproject:app-projects
    /myproject:app-ui
    /myprojects:services
    /myprojects:landing

Configuring your apps for local development and production builds

When you have all your apps and packages in place, you should optimize on local development. You don't need to mimick the deployment for production on your local development machine. Microservices are needed to be able to scale your app at the next bottleneck, but this is only the case for production builds. During development you just want to work with one node process (for each of your end user apps) including all of your microservices. I'm not saying you shouldn't test the microservices build, but it's overhead to do this while developing. This is only needed during testing (using continuous integration techniques).

settings.json files for every app

You can control this by providing each app with its own set of settings files for local development. In these settings files you can control what background processes must run in your services. During local development, you mostly enable all of your background processes for all services, while you disable them for production builds in your user-facing app. You'd rather configure separate service-apps enabling only one background process per build, so you can scale them separately.

Make sure to add a folder to your apps containing your settings files:

/project
  /package.json
  /app
    /.meteor
    /.deploy
      local.json
    /packages => ../packages
  /landing
    /.meteor
    /.deploy
      local.json
    /packages => ../packages
  /services
    /.meteor
    /.deploy
      local.json
    /packages => ../packages
  /packages
    ...

You now have your settings files for local development and can start your apps in development mode:

$ cd app
$ meteor --settings .deploy/local.json --port 3000
$ cd landing
$ meteor --settings .deploy/local.json --port 3001

build & deploy for production

I presume you are familiar with MUP. MUP is a node script for building and deploying your builds on your infrastructure of choice. MDG is working on Galaxy and recently released a preview in private beta, but I believe (and others with me) MUP will still be valuable for lots of developers even when Galaxy is released to the public.

setup mup for all your apps/services

Organize your mup settings in the .deploy directories at app-level for every environment you want to deploy to:

$ mkdir -p app/.deploy/production
$ cd app/.deploy/production
$ mup init

You can then easily build and deploy all of your apps and services by configuring mup.json and defining other settings for production deployments.

/project
  /package.json
  /app
    /.deploy
      local.json
      /production
        mup.json
        settings.json
      /staging
        mup.json
        settings.json
    /packages => ../packages
  /landing
    /.deploy
      local.json
      /production
        mup.json
        settings.json
  /services
    /.deploy
      local.json
      /production
        mup.json
        settings.json

setting up npm scripts

To make this even more developer friendly, you can set up npm scripts, to setup your own CLI to run, build and deploy your apps. Generate a package.json file at the root level of your project and setup your scripts like the following example.

package.json

{
  "name": "myproject",
  "scripts": {
    "start": "cd app && MONGO_URL=mongodb://localhost:27017/myproject meteor --settings .deploy/local.json --port 3000",
    "landing": "cd landing && MONGO_URL=mongodb://localhost:27017/myproject meteor --settings .deploy/local.json --port 3001",
    "services": "cd services && MONGO_URL=mongodb://localhost:27017/myproject meteor --settings .deploy/local.json --port 3002",
    "deploy": "npm run deploy-app & npm run deploy-landing & npm run deploy-services",
    "deploy-app": "cd app/.deploy/production && mup deploy",
    "deploy-landing": "cd landing/.deploy/production && mup deploy",
    "deploy-services": "cd services/.deploy/production && mup deploy"
  }
}

You can now run your servers and execute build with these simple commands:

$ cd project
$ npm start
$ npm run landing
$ npm run services
$ npm run deploy

Conclusion

This Meteor pattern for organizing builds for apps and microservices is working very well for us. You’ll gain more control over your builds, get better oversight over the dependencies within your project, write more maintainable and modular code, and you’ll be able to scale your processes (microservices) separately, each according to their need. We think it’s a pattern every large Meteor project should adapt, decidedly as soon as it grows beyond the prototyping phase.

@SierraGolf

This comment has been minimized.

Copy link

commented Aug 28, 2015

Sadly meteorpatterns.com does not have a comment feature, so I do it here.

I think this is an interesting approach. Do you imply that all code is located in one repository? I could imagine this becoming quite difficult to manage considering that there might be multiple teams working on different services in parallel.

How do you handle branching, continuous integration and delivery with this approach?

@Alino

This comment has been minimized.

Copy link

commented Nov 19, 2015

hi, I've just set up our project following this article, and I am also really interested in how do you approach continuous integration with this. Do you use CircleCI, what is your setup? I would love to see second part for this article about CI. Thank you very much!

@stxxart

This comment has been minimized.

Copy link

commented Dec 12, 2015

For me, every Meteor app needs to be in its own repository. This applies to creating a network of Meteor apps, using a micro-services architecture. We deploy on Kubernetes, and use Wercker to manage our build and test process. We build each Meteor app inside a Docker container and then deploy the container to Kubernetes.

I like the concept behind packaging up code into logical components and services. I'm just trying to establish whether your approach would work with our pipeline. From what I can see, it would be difficult to set up multiple repositories with your structure. For us, it makes sense to only deploy the code we need to deploy. How do you manage this?

@SatadruBhattacharjee

This comment has been minimized.

Copy link

commented Dec 30, 2015

Any Sample Git for showcasing this ?

@markshust

This comment has been minimized.

Copy link

commented Jan 2, 2016

I'm using a similar setup in regards to many local packages and an umbrella package. It works great and locks down the code a great deal (great for security and management, load order, etc).

However, I'm having a problem where docker images take a long time to build, and it seems because I'm requiring many of the same libs in multiple local packages. Has anyone else been experiencing this issue?

It seems that the new npm capabilities being introduced in 1.3 will allow us to rid meteor of local packages and use private npm packages instead. I'm assuming this will help load time and some other things. Any thoughts appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.