Nimble RFC for project local dependencies

README.md

Following is the RFC for providing project local dependencies in Nimble. This is being tracked in issue 131 and discussed in the Nim forum.

Definitions:

• local deps mode - this RFC to enable project local dependencies where $prj/nimbledeps is used by Nimble on a per project basis
• user deps mode - current Nimble behavior of using ~/.nimble since dependencies are usable across projects for a specific user
• global deps mode - tracked in issue 80 where a system-wide folder can be used to store dependencies across users and projects. It is not covered in this RFC.

Motivations:

• The main value of local deps mode is to provide dependency isolation from other projects. Isolation simply makes development easier and avoids any conflicts caused by mixing dependencies across projects.
• local deps mode is simply a mode of operation during development and is not expected to be checked into source control. The intention is not to push vendoring or an alternative for lock files. It is not aimed at solving any distribution related challenges or enabling reproducible builds.
• If a user wants local deps mode for a particular project, it implies they want complete isolation. As a result, Nimble will no longer consider user or global dependencies and commands will all act as if $prj/nimbledeps is the only directory Nimble uses.
• Once a working configuration is reached regardless of deps mode, the user could then generate a lock file that improves on distribution. That design will be discussed separately and not distract this RFC.
• This RFC mainly describes Nimble behavior at this time. The changes required for Nim to understand local deps mode will be addressed later.

All behaviors below that drive actual code changes are highlighted with *.

Behavior:

• All nimble commands will highlight when in local deps mode so that it is amply clear to the user *
• An explicit --nimbleDir:xxx overrides local deps mode.
• If $prj/nimbledeps directory exists, set local deps mode by forcing nimbleDir = $prj/nimbledeps *
• Else, continue in default user deps mode

Command-line behavior in local deps mode:

• nimble install
  • Install all the project's dependencies into $prj/nimbledeps
  • Build binaries if applicable
  • Do not copy the project itself into $prj/nimbledeps - inform user *
• nimble install -d | --depsOnly - install all the project's dependencies into $prj/nimbledeps
• nimble install url|pkg - install dependency into $prj/nimbledeps
• nimble uninstall pkg - remove pkg from $prj/nimbledeps
• nimble uninstall pkg -i - remove pkg and packages that depend on it from $prj/nimbledeps
• nimble build - install deps into $prj/nimbledeps in the processDeps step
• nimble develop
  • Install all deps into $prj/nimbledeps in the processDeps step
  • Do not create any links to itself it in $prj/nimbledeps - inform user *
• nimble develop prj2
  • Clone prj2 into sub-directory within prj1 and add a link into $prj/nimbledeps
• nimble develop prj2 --nimbleDir:$prj1/nimbledeps
  • Clone prj2 into directory outside prj1 and link into $prj1/nimbledeps
• To use an existing project $prj2 as a dependency in local deps mode project $prj1:
  • cd $prj2
  • nimble develop --nimbleDir:$prj1/nimbledeps

local deps mode is a mode of operation during development and not for influencing distribution. While it is not recommended, users can save local deps mode in source control by:

• Checking in $prj/nimbledeps/empty.txt
• Checking in $prj/nimbledeps/* - vendoring or poor man's lock files
• Need to verify that nimble install of such a project does not break - Nimble should not get confused what the true nimbleDir is when processing such deps

No changes:

• All commands will continue to run like they do today when in user deps mode
• Dependencies will need to be added to the .nimble file in the requires section per usual to get installed from scratch regardless of deps mode

Caveats:

• Checking in $prj/nimbledeps/* into source control will work as expected for the project when checked back out but if the project is itself installed by a parent project, these checked-in deps should not be inherited. The dependencies will get pulled from ~/.nimble or $prj/nimbledeps of that parent project depending on its configuration. This is similar to how lock files work for libraries.
• If a project with local deps has a dep which builds binaries, it will install into $prj/nimbledeps which means the binaries will not be in ~/.nimble/bin, typically in $PATH. User is expected to add $prj/nimbledeps/bin to the $PATH.

If I add a dependency to my .nimble file, I already need to do nimble bump (which really should be a thing...) or manually git tag it. Not immediately afterwards but soon enough.

Huh? You only need to do this when releasing a new version, which shouldn't happen often at all.

Likewise adding/removing dependencies doesn't happen all that often in practice.

Updated this RFC - removed --user since local deps mode is all about isolation. Adding support to break out is unnecessary complexity. This greatly simplifies this RFC.