nixcon

nixcon

- agenda/ideas for the nixcon talk

 - brief introduction
  - what is pypi
  - how "awesome" python packaging is 
  - what problems we have when packing with nix

 - how it works (differently then other 2nix tools)
  - 3 stages
   - stage1: using nix-shell
    - run pip becuase we can only trust that  ``pip install <package>`` as an "api"
    - there is no package.json that we could parse, there is only setup.py
   - stage2: extra metadata from "wheel" installed python packages
   - stage3: fetch extra metadata (url/sha256) from pypi and generate nix expression
  - how our nix expression looks
   - generated + overrides
   - example with lxml

 - what does not work (yet)
  - we dont support tests
  - multiple pypi instances (eg. private pypi)
    TODO: patch pip to know where the package came from

 - nixpkgs-python
  - TODO: should this be an overlay :) if yes we need to own "python-next" namespace
     % nix-shell -p python-next.pelican.interpreter
  - fully automated with travis, it updates packages on a daily basis.
    very little intervention needed.
  - multiple python "sets", explain what a python set is
     a set of packages that work together (aka requirements.txt)
  - give example for creating blog with pelican (nixos-weekly is using it)
    https://github.com/NixOS/nixos-weekly/blob/master/default.nix
  - in haskell you have stackage there is nothing like this in python,
    nixpkgs-python could be a "stagage for python"
  - "global" overrides file, which any pypi2nix can use
    TODO: we need to add this as default behaviour to pypi2nix
  - (will - in progress) generate a static page with all python sets/packages (a static pypi)

 - future: enable tests
  - require rework of python infrastructure how it work
  - example:
      propagatedBuildInputs <- A B
          PYTHONPATH <- A B C D

        A:
         buildInputs = pytest
         propagatedBuildInputs = C

         buildPhase
          # flattenUnique create a flat list of packages
          pip install ${flattenUnique ${buildInputs ++ propagatedBuildInputs}} > $env

         checkPhase
           # load pythonpath from $dev
           pytest ...

         installPhase
           pip install ${flattenUnique ${propagatedBuildInputs} > $out

    for each package you need to create python set

 - future: pypi2nix -> nixpkgs-python -> nixpkgs
  - it possible with a bit of manual work to generate something to be used by
    nixpkgs, but we officially dont support it, but we would like to get there.
  - the plan:
   - create and automate multiple "nixpkgs-*" python sets in nixpkgs-python repo
   - we already started packaging existing python packages. once we packaged
     enough packages we can consider officially using pypi2nix
   - on "weekly" basis automatically create a PR which copies nixpkgs-* python
     sets into nixpkgs (once hydra supports building PRs)
  - blockers: we need to build with tests enabled to be on pair with current
    functionality