Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Guide for how to create a (minimal) private PyPI repo, just using Apache with directory autoindex, and pip with an extra index URL.

How to set up and use a private PyPI repo

Accompanying blog post:

Splitting a Python codebase into dependencies for fun and profit

Based on:

Create a local PyPi repository using only mod_rewrite

See also:

Local PyPI Options

Setting up a private, team-wide PyPI repository

For a more advanced private PyPI, see:

devpi: PyPI server and packaging/testing/release tool

Create root directory for private PyPI

ssh dude@myserver.com

(On remote server)

mkdir /path/to/pypi.myserver.com
mkdir /path/to/pypi.myserver.com/simple

Create self-signed SSL certificate

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/apache2/ssl/pypi.myserver.com.key \
-out /etc/apache2/ssl/pypi.myserver.com.pem

Set up Apache vhost

sudo htpasswd -c /etc/apache2/passwords_pypi pypi
sudo vi /etc/apache2/sites-available/pypi.myserver.com

(Add these lines)


<VirtualHost *:80>
    ServerName pypi.myserver.com

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>

<VirtualHost *:443>
    ServerName pypi.myserver.com
    DocumentRoot /data/www/pypi.myserver.com

    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/pypi.myserver.com.pem
    SSLCertificateKeyFile /etc/apache2/ssl/pypi.myserver.com.key

    <Directory /data/www/pypi.myserver.com/>
        AllowOverride None
        Options +Indexes
        IndexOptions SuppressColumnSorting
        IndexIgnore ..
        Order deny,allow
        Allow from all

        AuthType Basic
        AuthName "My Server"
        AuthBasicProvider file
        AuthUserFile /etc/apache2/passwords_pypi
        Require valid-user
    </Directory>

    LogLevel warn
    ErrorLog /var/log/apache2/pypi-error.log
    CustomLog /var/log/apache2/pypi-access.log combined
</VirtualHost>

cd /etc/apache2/sites-enabled
sudo ln -s ../sites-available/pypi.myserver.com pypi.myserver.com
sudo apache2ctl graceful

Create directory for new library in private PyPI

mkdir /path/to/pypi.myserver.com/simple/foobar-utils
exit

Update library's code

(On local machine)

cd /path/to/foobar-utils
vi foobar_utils.py

(Add these lines)


__version__ = '0.1.0'

foobar = 'Hey foo, I am a bar!'

vi setup.py

(Add these lines)


import os

import setuptools

module_path = os.path.join(os.path.dirname(__file__), 'foobar_utils.py')
version_line = [line for line in open(module_path)
                if line.startswith('__version__')][0]

__version__ = version_line.split('__version__ = ')[-1][1:][:-2]

setuptools.setup(
    name="foobar-utils",
    version=__version__,
    url="https://git.myserver.com/foobar-utils/",

    author="Mister foo",
    author_email="mister@foo.com",

    description="Utils for handling Foo and Bar.",
    long_description=open('README.rst').read(),

    py_modules=['foobar_utils'],
    zip_safe=False,
    platforms='any',

    install_requires=[],

    classifiers=[
        'Development Status :: 2 - Pre-Alpha',
        'Environment :: Web Environment',
        'Intended Audience :: Developers',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
    ],
)

vi README.rst

(Add these lines)


foobar-utils
============

Utils for handling Foo and Bar.

Upload new version of library code to private PyPI

python setup.py bdist_wheel --universal
scp dist/foobar_utils-0.1.0-py2.py3-none-any.whl \
dude@myserver.com:/path/to/pypi.myserver.com/simple/foobar-utils/

Configure pip to use private PyPI

vi ~/.pip/pip.conf

(Add these lines)

[global]
; Extra index to private pypi dependencies
extra-index-url = https://pypi:pyp1@pypi.myserver.com/simple/
trusted-host = pypi.myserver.com

Use private library in a project's requirements.txt

cd /path/to/projectfoo
virtualenv .
source bin/activate
vi requirements.txt

(Add these lines)


foobar-utils==0.1.0

pip install -r requirements.txt

brettswift commented Aug 17, 2017

any idea how to use this without the user dot files? That kinda sucks when you want to have other people set up on your team right away. Obviously these packages will be sourced from the same spot. I'm new to python - just wondering if it can go into a config file in the project, just no luck finding this on google yet.

Also supplying them with environment variables, so we can do continuous delivery of our modules from a CI server, would be good info :)

https://pypi.python.org/pypi/twine

looks like this is a preferred way of deploying

Could you post the equivalent nginx config?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment