We are a group of developers working on several Meteor apps. We've been working on them since almost one year now from the Meteor 1.2.1 version up to 1.4.2.3, which is the one we currently use.
As for our product requirements, we have needed to pose a complex structure in favor of logic reusability across all our applications. Recently, we started to wrap up our first realease and we noted the significant slow performance that our apps get.
We have invested time to study the causes and possible solutions, but at this point any decision we take on is going to consume time and we'd want to ensure we are on a safe ground and understand how a Meteor app has to be approached given the complexity we need. That is why we'd appreciate to get some feedback from you in this matter and help us to focus on the right direction.
First of all, we give you details about how the infrastructure of our apps and packages is tackled.
There are two main applications named app and admin. These include both NPM dependencies and Meteor packages.
We've needed to implement our Meteor packages for bringing up reusability of our logic shared between the two apps and others that will come in the future. In that sense, we created the following packages:
api- define the common methods and publications.collections- define the collections, models and factories.schema- define the schema for the models and methods, publications and components signature.ui- define the visual and data components shared on the apps.utilities- define the helpers, constants, and so on.- others for styles, build plugins,...
In effect, our apps and packages are setup in the next manner:
app or admin
|_ NPM & Meteor dependencies
|_ api
|_ collections
|_ schema
|_ ui
|_ utilities
api
|_ NPM & Meteor dependencies
|_ collections
|_ schema
|_ utilities
collections
|_ NPM & Meteor dependencies
|_ schema
|_ utilities
schema
|_ NPM & Meteor dependencies
|_ utilities
ui
|_ NPM & Meteor dependencies
|_ api
|_ collections
|_ utilities
Having explained our infrastructure, lets talk about the performance issues that we've experienced.
Once the Meteor building process prepares the scripts for every package and application, and we hit the browser to run the apps, we get the next profiles (via Chrome tools) on the apps.
For admin app,
For app app,
As you can see the times spent on both are quite similar but so high. The next graphic shows how much time our packages gets to be downloaded. That only give us an idea of how much size they have.
And finally the next picture shows the timeline for the admin case (app is similar but it gets less time to download and load app.js since it implements less content).
Going through the analysis we have noted the large amount of time spent in both downloading and loading the packages and the app itself. The app execution could be also something to look into, it envolves subscription and other network tasks related, however the time spent before feels more considerable and we want to understand why, since our app doesn't have large of features implemented there.
After getting some time analyzing everything we have noted some possible causes. They are related on how we have designed our Meteor packages, but the solutions already take up some issues, for that we'd want to know how Meteor usually is approached on large projects.
The way meteor packages handle NPM dependencies, is that all of them, whether they are included directly at package.json or using Npm.depends, are bundled on the package itself, which produces increasing our apps in both size and loading. As you can check in the next graphic there libs are being included one time per package that uses,
So just imaging the times that we could be including and loading react, or other common libs like lodash and so on.
As Meteor doesn't have a way to handle peer dependencies between the apps and packages as npm does, we've been researched that the way to tackle that is to move all NPM dependencies from the packages to the app level. However that produces to move all our package tests to a separate application and running them there, which make our delivering process slower and more complex. And it also requires a way to handle the requirements on the NPM versions that each package needs, check-npm-version has not resulted to be worth for us, we've faced so many issues like #1, and others run failures due to dependencies needed on server and between packages that depend on other packages. That apporach feels like quite buggy and the project seems not to provide the solutions.
- Do you happen to know other ideas or ways to handle peer NPM dependencies on your projects?
Other solution could be to create our packages as plain NPM modules and have a custom way to build and test (through webpack or other env setup tool), and then we could benefit from NPM advantages to handle peer dependencies and so on. The problem here is that some of our modules need to include meteor package dependencies. So,
- How can we import
meteoror other meteors athmospgere package functionality in our npm modules?
We've heard that this last thing is what Meteor wants to transit to, So,
- How is this transition going to work?
All our packages dependencies are importing all their contents automatically though the api.mainModule definition. So, that is causing that the apps depending on each package are importing the contents fully on each app, no matter is they ever use it or not.
api.mainModule('api.client.js', 'client');
api.mainModule('api.server.js', 'server');Then api.client.js or api.server.js exports all the logic for each end. I am not sure if that could have the worst impact on our current setup, but inseting more than the needed logic. We probably want to have every collection definition on the correspond package but we don't want it to be loaded in all our apps since it won't be used is some of them.
Researching over the Internet, a possible solution could be to stop using an api.mainModule to expose our logic modules. So a possible alternative could be to export our package modules lazily using api.addFiles(..., ..., { isAsset: true }). So as example,
file structure
collections
|_ some
|_ index.js
|_ collection.js
on the package.js:
api.addFiles(['./some/index.js', './some/collection.js'], ['client', 'server'], { isAsset: true });at app/other package context:
import { someCollection } from 'meteor/collections/some';That is likely to not have any impact on the package size, but it will probably have on some apps that doesn't need such extra logic. I guess another thing is to rely on some logic to be defined on app level, but as long as they are needed in other apps they will be moved to the package. So having a way to import the functionality on demand could be good.
Questions here
- Would that way to expose and import by demand work?
- Is that a common pattern used in Meteor packages for improving the performance? Or what is?
We are making a lot of work trying to figure out other causes, like some possible heavy dependencies that could be imported different or alternatives, check that every server-only logic is not delivered to the client, possible circular dependencies handled badly on javascript, etc.
But as stated at the beginning, we need an extra opinion about this. There may exist other causes that you have faced with other meteor users that could drive us to check them and could be causing major impact. We really need the feedback you could provide us about meteor designing and best practices on apps and packages.