How to copy production database on AWS RDS(postgresql) to local development database.

how-to-copy-aws-rds-to-local.md

1. Change your database RDS instance security group to allow your machine to access it.
   • Add your ip to the security group to acces the instance via Postgres.
2. Make a copy of the database using pg_dump
   • $ pg_dump -h <public dns> -U <my username> -f <name of dump file .sql> <name of my database>
   • you will be asked for postgressql password.
   • a dump file(.sql) will be created
3. Restore that dump file to your local database.
   • but you might need to drop the database and create it first
   • $ psql -U <postgresql username> -d <database name> -f <dump file that you want to restore>
   • the database is restored
4. pg_restore -h <host> -U <username> -c -d <database name> <filename to be restored>

#####ref

1. http://stackoverflow.com/questions/31881786/how-to-pg-dump-an-rds-postgres-database
2. http://www.thegeekstuff.com/2009/01/how-to-backup-and-restore-postgres-database-using-pg_dump-and-psql/

There are no such thing as security groups for DB instance.

There is indeed a security group for the VPC your RDS DB is attached to.

Once you're in the admin console, looking at your RDS instance, the third column under the "Connectivity & Security" tab will show and link to the current security group.

If you scroll down just a little more, it'll also actually display to you the current inbound and outbound rules on the current group.

#2-3 worked great, thanks!
What is the #4 for though?

Works perfectly!

Thanks guys. It worked.

Thanks, works perfectly.

has anyone had issues relating to RDSAdmin when importing the dump file ?

Im stuck on "pg_dump: saving search_path =" :(

has anyone had issues relating to RDSAdmin when importing the dump file ?

Have you solved it?

yeah, even tho there is an error it didnt stop the restore. You could manually add it I guess, but seems to be ok for me

Excellent, but how can I do the same for a multi-tenant architecture with DB per tenant?

Just a note: if you're using a custom VPC, make sure you enable "Publicly Accessible" in your DB instance and that your VPC has an internet gateway (by attaching the gateway to the VPC ID).

Future me: also make sure your route table is actually pointing to your internet gateway: https://aws.amazon.com/premiumsupport/knowledge-center/instance-vpc-troubleshoot/ because last time it was pointing to an old one that didn't allow me to connect to anything.

I get this error "pg_dump: error: could not open output file "test_dump_20210121.sql": Permission denied"

Any ideas?

Hey @burakuytun-rightsoft it might be because your current user doesn't have the right permissions. Try running sudo chmod 666 test_dump_20210121.sql.

Thanks for this. #4 was not needed.
The extra step I had to do before restoring the dump, was create the DB, as well as the users and roles associated with it.
Adding a user for rdsadmin is necessary if it's the owner of any objects in your DB. If not, then you should be fine ignoring the error.

PS. Depending on the size (and purpose) of your DB, it might make sense to change the location of postgres' data directory

Thanks @kinduff :)

👍

If you want to restore a dump into a differently named database, make sure not use the --create option on pg_dump as it will force create the DB under that name.

If you use the --format c flag you can use pg_restore and save some bandwidth.

psql -h localhost -U user -d postgres -c "DROP DATABASE IF EXISTS newdb (FORCE)"
psql -h localhost -U user -d postgres -c "CREATE DATABASE newdb WITH OWNER = postgres ENCODING = 'UTF8'"
pg_dump --format c --blobs --no-owner --no-acl --quote-all-identifiers --verbose --dbname=postgresql://user:password@domain.rds.amazonaws.com/databasename --file outputfile.bak
pg_restore --verbose -h localhost -U user -d newdb outputfile.bak

Could add here the step to close all connections to the current running DB, before deletion:
psql -h localhost -U user -d postgres -c "select pg_terminate_backend(pid) from pg_stat_activity where datname='newdb'"

I am getting "AccessShareLock" on a few tables when I try to generate the dump as there are multiple active connections to the db. Is there any way to clone it to a read replica even when the connections are active?