Skip to content

Instantly share code, notes, and snippets.

@JosimarCamargo
Last active April 6, 2024 15:39
Show Gist options
  • Save JosimarCamargo/40f8636563c6e9ececf603e94c3affa7 to your computer and use it in GitHub Desktop.
Save JosimarCamargo/40f8636563c6e9ececf603e94c3affa7 to your computer and use it in GitHub Desktop.
How to setup a Postgres replication with docker and docker-compose

How to setup a Postgres replication with docker and docker-compose

How to use

To run you will need docker and docker-compose installed, and run the command docker-compose up on a folder within this file named docker-compose.yml(attached in this Gist)

How the PGAudit is enable, you will see the queries log and in which database is running which query, this is the main purpose of this Gist, along with have a database with replication working out of the box

To add user with just reading rights to access the database on slave replica

I suppose that you are inside the directory with the docker-compose file run the commands:

1 - Access the master container(you read right the user added to master will be replicated to slave)

docker-compose exec postgresql-master bash

2 - Access the psql client

psql -U postgres

3 - Run the SQL commands, creating the user 'reading_user' and giving him permissions

CREATE USER reading_user WITH PASSWORD 'reading_pass';
GRANT CONNECT ON DATABASE my_database TO reading_user;
\connect my_database
GRANT SELECT ON ALL TABLES IN SCHEMA public TO reading_user;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO reading_user;
GRANT USAGE ON SCHEMA public TO reading_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO reading_user;

I suggest restart all containers and possible connections

The docke-compose.yml was heavily based on https://github.com/bitnami/bitnami-docker-postgresql/blob/72183d5623eb92a9781637f6f395f13ee93c5836/docker-compose-replication.yml thanks bitnami for this great work

Questions about the SQL script https://stackoverflow.com/questions/760210/how-do-you-create-a-read-only-user-in-postgresql

version: '2'
services:
postgresql-master:
image: 'docker.io/bitnami/postgresql:11-debian-10'
ports:
- '5432:5432'
volumes:
- 'postgresql_master_data:/bitnami/postgresql'
environment:
- POSTGRESQL_PGAUDIT_LOG=READ,WRITE
- POSTGRESQL_LOG_HOSTNAME=true
- POSTGRESQL_REPLICATION_MODE=master
- POSTGRESQL_REPLICATION_USER=repl_user
- POSTGRESQL_REPLICATION_PASSWORD=repl_password
- POSTGRESQL_USERNAME=postgres
- POSTGRESQL_DATABASE=my_database
- ALLOW_EMPTY_PASSWORD=yes
postgresql-slave:
image: 'docker.io/bitnami/postgresql:11-debian-10'
ports:
- '5433:5432'
depends_on:
- postgresql-master
environment:
- POSTGRESQL_USERNAME=postgres
- POSTGRESQL_PASSWORD=my_password
- POSTGRESQL_MASTER_HOST=postgresql-master
- POSTGRESQL_PGAUDIT_LOG=READ,WRITE
- POSTGRESQL_LOG_HOSTNAME=true
- POSTGRESQL_REPLICATION_MODE=slave
- POSTGRESQL_REPLICATION_USER=repl_user
- POSTGRESQL_REPLICATION_PASSWORD=repl_password
- POSTGRESQL_MASTER_PORT_NUMBER=5432
volumes:
postgresql_master_data:
driver: local
@joshua-quek-sonarsource

Hi there! thanks for this!
I understand that the slave is a read-replica. How can I make it writable too so that an external service can also write to it?

@ronster37
Copy link

@joshua-quek-sonarsource That defeats the whole purpose of a read replica. What you are asking for is multi-master which is a completely different topic.

@joshua-quek-sonarsource

Hi @ronster37 , yes the multi-master is exactly what I am looking for. I would need two instances in the same cluster that are two way writable.

@dibdutta
Copy link

dibdutta commented May 25, 2023

I am getting below warning on slave

docker-compose-postgresql-slave-1  | 2023-05-25 14:14:23.214 GMT [286] LOG:  invalid record length at 0/35E4230: wanted 24, got 0
docker-compose-postgresql-slave-1  | 2023-05-25 14:14:23.352 GMT [384] LOG:  started streaming WAL from primary at 0/3000000 on timeline 1

Also how to connect to slave postgres instance. psql -U postgres seems to be not working with my_password

@OgiBalboa
Copy link

How can I increase the number of slave dbs ?

@alfredriesen
Copy link

alfredriesen commented Oct 4, 2023

Could I use this way also to safely backup a postgres database volume, without downtime? I could stop postgresql-slave service and backup it via https://offen.github.io/docker-volume-backup. When the postgresql-slave gets restart again, will it be synced automatically with the master? And I when want use the backup, can I restore it to postgresql_master_data volume or how would then restoration of a backup work?

@ealcantara22
Copy link

Hi, thanks for sharing! links in the readme file are not working so for those of you interested here's the new repo link: https://github.com/bitnami/containers/tree/main/bitnami/postgresql-repmgr

Best regards!

@asynchroza
Copy link

How do you offload the read queries to the replica in this case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment