The following RFC aims to implement lock file support for Nimble. This is being tracked in issue 127 but this proposal is a simpler variation.
Intro:
lock mode
will be enabled by a flag innimble.ini
- Since it is a new feature, this will reduce impact on the existing user base
- It can be turned on by default in the future when stable enough
- Lock information will be stored in
$prj/nim.cfg
- Nimble will append autogenerated section if existing
- Nimble will update when any dep changes occur
- Nimble will consume when having to install deps
- Nim will consume per usual
Lock file format:
- Separate section in
$prj/nim.cfg
maintained by Nimble - Nimble related metadata will be stored as nim comments
- E.g.
# Nimble autogenerated start
--noNimblePath
# url = https://...
# checksum = xyzxyz
--path:$nimbleDir/pkgs/package-1.0.0
# Nimble autogenerated end
- Format currently includes package name, version, url and checksum
- Nimble does not store source control hashes so that's not being considered for now
Nimble changes:
- Add code to write and read back from
$prj/nim.cfg
- After any package install
- Calculate and store checksum in either nimblemeta.json or a new file in pkg root
processDeps()
- Get packages from
.nimble
file - If
$prj/nim.cfg
exists- Get package versions from cfg file
- For each package in
.nimble
requires- If package locked version exists
- Install specific version of package if not already in
~/.nimble
- Install specific version of package if not already in
- Else
- Install package using standard workflow today
- Remember package version installed
- Set create flag
- If package locked version exists
- If create flag set, create/update
$prj/nim.cfg
- Get packages from
nimble install
- use updatedprocessDeps()
workflownimble install url|pkg
- run same workflow as todaynimble uninstall pkg
- remove package per usualnimble build | docs | c | run
- Use updated
processDeps()
workflow - Run Nim without
--path:
flags
- Use updated
nimble develop
- use updatedprocessDeps()
workflow- Impacted by
develop --all
RFC - TBD
- Impacted by
Nim required changes:
- Replace instances of $nimbleDir in
--path:$nimbleDir/xyz
with path to OS specific Nimble directory - Once lock files are always on and the only way to use Nimble, as part of issue 654
- Deprecate and eventually remove
--NimblePath
and--cleanNimblePath
- Remove code that searches packages in
NimblePath
- Deprecate and eventually remove
No changes:
- Dependencies will need to be added to the
.nimble
file in therequires
section per usual to get installed and included in$prj/nim.cfg
- Once nimble can update the
.nimble
filerequires
section automatically in the future during install/uninstall, the lock file can get auto-updated as well
- Once nimble can update the
Other issues addressed:
- Project isolation - issue 131
- Since lock files point to specific versions, changes due to other projects will not matter
- If someone does remove a dependency, it can be reinstalled with one of the nimble commands that runs
processDeps()
- Reducing Nim's knowledge of Nimble - issue 654
Other issues expecting fix via lock files - not solved yet:
- Nimble develop of http package - issue 543
- Nimble does not currently save sufficient info about packages to know for sure
- Skip dep checks - issue 589
- Nimble will still run
processDeps()
- unclear if lock file will be faster
- Nimble will still run
Notes:
- Does not include
strictMode
in original issue proposal - URL is simply base URL to project and no commit hash - Nimble does not store this info today
It is really about simplifying interop with Nim. This method informs Nim exactly where to find packages and the package manager is fully responsible for that experience so we are not avoiding the package manager. Neither are we forcing it to be the mandatory all the time. Nim also doesn't need any special code related to finding packages cause that can be facilitated by the package manager.
I'll summarize with the following points:
The
nim.cfg
proposal is simple - it has minimal impact on Nim, and even opens the possibility of reducing existing code around finding packages. Nimble already knows how to generate--path
lists, all we need to do is put it in a file and we get a basic lock file for free. More rigor can be added over time.We don't have the luxury of time or resources to get this done - the community needs a solution and a simpler solution is practical and achievable. Otherwise, we leave the community to thrash around the issues and limitations.
Our user base wants both workflows and we cannot force them into one way. There are strong opinions on both sides and concerns are legitimate but both sides have to compromise and proceed and help iron out corner cases and issues.
Nim is already 1.0 and we have to build on what we have and carefully deprecate legacy. We cannot simply toss out stuff nor make ambitious changes. There has to be a roadmap and we have to keep evolving and improving incrementally.