Skip to content

Instantly share code, notes, and snippets.

Last active March 19, 2023 08:56
  • Star 36 You must be signed in to star a gist
  • Fork 9 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Caching PyPi packages locally with nginx
user www-data;
worker_processes 4;
error_log /var/log/nginx/error.log;
pid /var/run/;
events {
worker_connections 1024;
http {
include /etc/nginx/mime.types;
access_log off;
default_type application/octet-stream;
sendfile on;
tcp_nodelay on;
tcp_nopush off;
reset_timedout_connection on;
server_tokens off;
# Cache 100G worth of packages for up to 1 month
proxy_cache_path /var/lib/nginx/pypi levels=1:2 keys_zone=pypi:16m inactive=1M max_size=100G;
# Multiple server definitions makes nginx retry on errors
upstream pypi {
keepalive 16;
gzip on;
gzip_types application/json text/css text/javascript;
gzip_proxied any;
gzip_vary on;
server {
listen 80 default_server;
root /var/www;
proxy_cache pypi;
proxy_cache_key $uri;
proxy_cache_lock on;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_http_version 1.1;
proxy_set_header Host;
proxy_set_header Connection "";
proxy_set_header Accept-Encoding "";
# Rewrite any http redirects to use relative to proxy
proxy_redirect ~https?://*) $1;
location / {
# Replace any reference to actual pypi w/ caching proxy
sub_filter '' $scheme://$host;
sub_filter_once off;
proxy_pass https://pypi;
proxy_cache off;
location ^~ /simple {
# Make sure URI ends with /
rewrite ^(.*[^/])$ $1/ break;
add_header X-Cache2 $upstream_cache_status;
proxy_cache_valid any 5m;
proxy_pass https://pypi;
location ^~ /packages {
add_header X-Cache2 $upstream_cache_status;
proxy_cache_valid any 1M;
proxy_pass https://pypi;
Copy link

cedef commented Nov 9, 2018

You just saved my life with your config snippet dude !
Thanks a lot !!

Copy link

ecerulm commented May 19, 2022

As of 2022 this nginx won't work as is. It requires some changes / additions:

  • All references to (proxy_set_header Host, upstream, etc) should be changed to the current primary domain, otherwise you just will get 301 and pip just thinks there is no package with that name
  • proxy_ssl_server_name should be set to 'on, as` requires SNI.

Copy link

This needs several changes to work in 2023, in addition to the things ecerulm mentions.

Here's what works for me (includes tests to prove it works for pip):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment