Skip to content

Instantly share code, notes, and snippets.

@abdennour
Last active June 19, 2024 10:38
Show Gist options
  • Save abdennour/74c5de79e57a47f3351217d674238da8 to your computer and use it in GitHub Desktop.
Save abdennour/74c5de79e57a47f3351217d674238da8 to your computer and use it in GitHub Desktop.
Nginx Reverse Proxy for Nexus Docker Registries

Overview

This is a solution of a common problem with Nexus Docker repositories. The administrator has to expose port for "pull", another port for "push", other ports for each hosted repository. This solution is about leveraging Nginx reverse proxy to avoid using these ports.

How it works ?

Given :

  • Nexus hostname is "nexus.example.com"
  • Nexus web port is 8081
  • A hosted repository is named "docker-hosted"
  • A group repository is named "docker-group"
  • Your nginx (with the nginx.conf of this gist) will run for example under cregistry.example.com

The following Nginx configuration file is for a reverse proxy without the need to expose connector ports from nexus :

  • docker pull cregistry.example.com/myimage lets Nginx forward the request to "docker-group"
  • docker push cregistry.example.com/myimage lets Nginx forward the request to "docker-hosted"

Notes

  • If you have more than one hosted repository, create another Nginx reverse proxy for it, then aggregate them using a parent Nginx reverse proxy that forwards the request according to certain criteria (.i.e: Host header).

  • All Nexus repositories must have consistent configuration of authentication: Either all require authentication, or all don't.

  • If TLS is enabled with Nexus, change proxy_set_header X-Forwarded-Proto "http"; by proxy_set_header X-Forwarded-Proto "https";

version: "3"
services:
web:
image: nginx:1.15
hostname: cregistry.example.com
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
ports:
- "80:80"
nexus:
image: sonatype/nexus3
hostname: nexus.example.com
volumes:
- "nexus-data:/nexus-data"
ports:
- "8081:8081"
volumes:
nexus-data: {}
events {
worker_connections 1024;
}
http {
proxy_send_timeout 120;
proxy_read_timeout 300;
proxy_buffering off;
keepalive_timeout 5 5;
tcp_nodelay on;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
server {
listen *:80;
location ~ ^/(v1|v2)/[^/]+/?[^/]+/blobs/ {
if ($request_method ~* (POST|PUT|DELETE|PATCH|HEAD) ) {
rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
}
rewrite ^/(.*)$ /repository/docker-group/$1 last;
}
location ~ ^/(v1|v2)/ {
if ($request_method ~* (POST|PUT|DELETE|PATCH) ) {
rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
}
rewrite ^/(.*)$ /repository/docker-group/$1 last;
}
location / {
proxy_pass http://nexus.example.com:8081/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto "http";
}
}
}
@AlexGluck
Copy link

@chaitany010 required code for you (i hope):

  proxy_send_timeout 120;
  proxy_read_timeout 300;
  proxy_buffering    off;
  keepalive_timeout  5 5;
  tcp_nodelay        on;
  client_max_body_size 0;
  chunked_transfer_encoding on;
  proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=nexus:100m inactive=30d  max_size=2g;

  upstream nexus-node {
    server nexus:8081 max_fails=0;
    keepalive 150;
    keepalive_timeout 60s;
    keepalive_time 1h;
    keepalive_requests 1000;
  }

  server {
    listen  80;

    proxy_http_version 1.1;
    proxy_set_header Connection "";

    location ~ ^/api/(.*) {
      proxy_pass http://nexus-node/$1$is_args$args;
    }

    location ~ ^/(v1|v2)/([-_0-9a-z\.]+)/(.*)/blobs/uploads/$ {
      proxy_pass http://nexus-node/repository/$2/$1/$2/$2/$3/blobs/uploads/$is_args$args;
    }

    location ~ ^/(v1|v2)/([-_0-9a-z\.]+)/(blobs/sha256.*|manifests/.*)$ {
      proxy_pass http://nexus-node/repository/library/$1/library/library/$2/$3$is_args$args;
      error_page 404 500 = @fallback2;
      proxy_intercept_errors on;
      recursive_error_pages on;
      proxy_cache            nexus;
      proxy_cache_valid      500 10d;
      proxy_cache_min_uses   3;
      proxy_cache_valid      404 15m;
      proxy_cache_use_stale  http_500;
    }

    location ~ ^/(v1|v2)/$ {
      proxy_pass http://nexus-node/repository/docker-login/$1/$2$is_args$args;
    }

    location ~ ^/(v1|v2)/(_catalog|search)$ {
      proxy_pass http://nexus-node/repository/docker-group/$1/$2$is_args$args;
    }

    location ~ ^/(v1|v2)/([-_0-9a-z\.]+)/(.*)$ {
      proxy_pass http://nexus-node/repository/$2/$1/$2/$3$is_args$args;
      error_page 400 404 500 = @fallback;
      proxy_intercept_errors on;
      recursive_error_pages on;
      proxy_cache            nexus;
      proxy_cache_valid      400 500 10d;
      proxy_cache_min_uses   3;
      proxy_cache_valid      404 15m;
      proxy_cache_use_stale  http_500;
    }

    location @fallback {
      proxy_pass http://nexus-node/repository/$2/$1/$3$is_args$args;
      error_page 404 500 = @fallback2;
      proxy_intercept_errors on;
      recursive_error_pages on;
      proxy_cache            nexus;
      proxy_cache_valid      500 10d;
      proxy_cache_min_uses   3;
      proxy_cache_valid      404 15m;
      proxy_cache_use_stale  http_500;
    }

    location @fallback2 {
      proxy_pass http://nexus-node/repository/docker-group/$1/$2$is_args$args;
    }

    location / {
      proxy_pass http://nexus-node/;
    }
  }

@yokozu777
Copy link

Hi all.
If I activate cache (including proxy_buffering on; )
docker push stops working
"unauthorized: access to the requested resource is not authorized"

Log with cache off:
171.2.1.4 - admin [11/May/2024:20:19:57 +0100] "POST

Log with cache on:
171.2.1.4 - - [11/May/2024:20:22:16 +0100] "POST

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment