Monorepo workspaces

proposal.md

Yarn workspaces (as I understand them) use the root of the Lerna-style repository to install node_modules. Lerna-style repos tend to contain libraries rather than apps, though.

In a monorepo with multiple apps, we want each app to have its own node_modules directory. This is so that each app's node_modules directory contains only that app's dependencies, which can lead to better deduping for that app. Also if an app has postinstall scripts that modify node_modules, those scripts should modify only that app's node_modules.

How can we combine parts of these two worlds? If an app depends on other packages inside the same monorepo, we (usually) want to use those local packages. We also want each app to have its own node_modules.

One idea is for each app's package.json to declare a "workspace" key with deps/devDeps to install.

{
  "workspace": {
    "dependencies": {
      "typeaheadpro": "<rootDir>/libraries/typeaheadpro"
    },
    "devDependencies": {
      "jest-preset": "<rootDir>/testing/jest-preset"
    }
  }
}

When installing this app's dependencies with a special workspace command (yarn install --workspace?), Yarn would install the app's dependencies (listed under the regular deps/devDeps keys) except for the ones listed under workspace.dependencies and workspace.devDependencies. For the workspace dependencies, Yarn would go into those dependencies' directories, run yarn pack (to simulate publishing) and then in the app's directory run yarn add <the packed tar.gz> --pure-lockfile (to install the library in the app). This way we're now using local copies of the libraries as if they had been published to npm.

This proposal could also incorporate a --link flag. If you run yarn install --workspace --link in an app's directory, instead of running yarn pack for each of the workspace dependencies, Yarn would run yarn knit (or whatever that command ends up being called), which also simulates publishing (this is key! gonna want that parity between development and production) but uses symlinks.

Also if an app has postinstall scripts that modify node_modules, those scripts should modify only that app's node_modules.

What do you mean exactly? Why would a postinstall script modify a node_modules directory?

If I understand your need correctly (you want your repositories to be synced as if they were published, including any post-processing triggered by a postinstall, correct?), wouldn't your use case be solved by something like this?:

$> yarn workspace --all run build

What is the difference between this proposal and using link:/ specifier for dependencies?

link:// would not work outside of the monorepo. Ex: if you use shipit to sync just the app with an external GitHub repo, then people need to get the app's dependencies from npm.

I also want link:// to simulate publishing and then create symlinks for each file or directory within the package rather than symlinking the entire package. I don't trust using the existing npm link / yarn link behavior for development since it differs too much from prod.

What do you mean exactly? Why would a postinstall script modify a node_modules directory?

Sometimes node_modules contains code that messes with Babel or Flow. The postinstall script lets you make adjustments and get unblocked.

you want your repositories to be synced as if they were published, including any post-processing triggered by a postinstall, correct?

I also want my node_modules hierarchy to be as if I ran yarn install in the app directory and not place node_modules in the repo root. This affects npm's deduping and file-watching roots, for example. As I understand it, Yarn workspaces assume there's only one root but in this proposal, each app directory would be its own root (but can link to libraries outside of that root).

@ide you have some valid use case here.

First gut feeling is to reject term "workspaces" because it is associated in my mind with the structure babel and jest repos have.
I would not want to overload it.

Do you want something like you knit proposal but in an automated way?

Please cc us when answering, I did not find a way to subscribe