_Distfile.lock.json

"Automatically Generated, Not for Editing"
"Acts as a Lock file"
"Uses a List, but is actually a set"
"All packages without a group are in the default group"
{
    "_meta": {
        "sources": [
            {"url": "https://simple.crate.io/"},
            {"url": "https://pypi.python.org/simple/", "verify_ssl": false},
        ]
     },
    "default": [
        {"name": "requests", "version": "0.11.2", "hash": "...."},
        {"name": "Django", "version": "1.4", "hash": "..."},
        {"name": "pinax", "git": "git://....", "branch": "1.4"},
        {"name": "crate", "path": "~/blech", "editable": true}
    ],
    "development": [
        {"name": "test", "version": "0.1", "hash": "..."},
        {"name": "test2", "version": "2.0", "hash": "..."},
        {"name": "another thing", "version": "3.5", "hash": "..."}
    ],
    "testing": [
        {"name": "test", "version": "0.1", "hash": "..."}
    ]
}

Distfile.py

# This is designed to be edited by hand by python developers
# --index-url and friends look like command options, non inutive. No extra metadata available
source("https://simple.crate.io/")
source("https://pypi.python.org/simple/", verify_ssl=False)


# Design:
#  - Use real datastructures, make things clearer
#  - Seperate package name from version, using real strings.
#      Django==1.4 is a valid PyPI package, uninstallable from current files
#  - using kwargs creates a great way to provide optional options on a line by line basis
#     that python programmers are already familar with
#  - People should only have to worry about the things they depend on, the installer
#      should do the right thing
#  - People have different dependency based on environment
#  - Allow advanced usage for "wierd use cases" or "patterns not anticipated"

# Concerns:
#  - Using Python file might cause the same problems as setup.py
#    - This File not designed to be directly executed
#    - setup.py is typically sent to other people, requirements are typically for internal use
#    - Final result is still deterministic
#  - Somewhat more verbose
#    - Uses a syntax familar with all python programmers


dist("requests")  # Install the latest version of requests, and all dependency's
dist("Django", "==1.4")  # Install a version of Django matching the "==1.4" version spec and all dependencies
dist("pinax", git="git://github.com/pinax/pinax.git", branch="1.4") # Install pinax, using git and the 1.4 branch
dist("crate", path="~/blech", editable=True) # Install crate from the supplied path, and install it as an editable
dist("test", group="development")  # install test, but only if the development group is passed
dist("test2", group=["development", "testing"])  # install test2, but only if the development or testing group is passed

with group("development"):  # start a context that is equivilant to passing everything a certain group
    dist("another thing")   # install ``another thing`` if the development group is passed

with source("https://inhouse.example.com/"):
    # Things in here MUST be installed from the above source, the idea being that if you have forked, e.g. django-model-utils
    # you can uplaod to an inhouse server and force django-model-utils to install from this source, even in the case
    # there is another version higher then yours, or even the same version.
    # Additionally if packages installed inside of a with source(): have dependencies their deps are searched for
    # on source, and installed from there if possible. However if they are not available dependencies will fall back
    # to the global source()es.
    dist("django-model-utils")  # Not shown in the json lockfile.

# Details
#  - This file does NOT cause anything to directly get installed, the program uses it to build up an internal list of requirements
#  - All dist()'s are considered for the requirements, even if they are not going to get installed (e.g. they are in a not currently active group)
#    - This will allow things to work smoothly where the requirement in a group might modify the final installed version (e.g. a group might pin to Django<1.4)
#  - This file will be "compiled" down to json.
#    - When the compiled json exists, there is no need to do any sort of version resolving. We know exactly what it is we want to install. (e.g. --no-deps etc)
#    - We still need to find out where to download the packages from? (Unless we encode that in the json. Might be risky)
#  - If there's a corner case not thought of, file is still Python and allows people to easily extend
#  - You don't need to pin to exact versions, you just need to pin to what you want to support. e.g. if a pacakge follows semver, you could do package>1.2.1,<1.3 (you first started using the 1.2.1 version, and you expect 1.2.X to always be good for you.
#    - Exact versions are pinned automatically in the json file

(Especially given the comment I just left on the other Gist) personally I like this direction, but slightly fewer globals would be nice? E.g.:

dependencies = Dependencies(
    sources=Source(...),
)

dependencies.add("requests")
...

git branch should be ref

Possibly we want to be able to exclude a package by default (e.g. no gunicorn on Windows). We should expose environment markers somehow.