Skip to content

Instantly share code, notes, and snippets.

@sirodoht
Last active April 16, 2024 17:51
Show Gist options
  • Save sirodoht/f598d14e9644e2d3909629a41e3522ad to your computer and use it in GitHub Desktop.
Save sirodoht/f598d14e9644e2d3909629a41e3522ad to your computer and use it in GitHub Desktop.
How to migrate Django from SQLite to PostgreSQL

How to migrate Django from SQLite to PostgreSQL

Dump existing data:

python3 manage.py dumpdata > datadump.json

Change settings.py to Postgres backend.

Make sure you can connect on PostgreSQL. Then:

python3 manage.py migrate --run-syncdb

Run this on Django shell to exclude contentype data

python3 manage.py shell
>>> from django.contrib.contenttypes.models import ContentType
>>> ContentType.objects.all().delete()
>>> quit()

Finally:

python3 manage.py loaddata datadump.json

Source: https://stackoverflow.com/questions/3034910/whats-the-best-way-to-migrate-a-django-db-from-sqlite-to-mysql

@ronaldlangeveld
Copy link

ronaldlangeveld commented Apr 11, 2018

Thank you! Used these steps to migrate my MySQL db to PSQL and it worked perfectly! :D cheers

@gamonsalve
Copy link

gamonsalve commented May 6, 2018

It is not working for me, when I run python3 manage.py migrate --run-syncdb, after change settings.py to Postgresql, I get the following Error: django.db.utils.ProgrammingError: relation "etl_categories" does not exist
Am I making a mistake?
SOLVED:
Yes I was making a mistake. Since some views and forms in my app use some Models, when I run "python manage.py migrate" the ERROR above appear. If you are having the same problem just delete or comment all your views, forms and urls that uses any model. Then Follow the steps (migrate, load again your data) and finally uncomment/restore your files.

@bjarnephys
Copy link

bjarnephys commented Jul 11, 2018

tables created, ok, but data were not inserted!

because datadump.json have no data user data

@Lapicher
Copy link

Thankssssss so much!!! this steps works to me too. To migrate from PostgreSQL to MSSQL Server. Works perfectly

@abitran
Copy link

abitran commented Jul 18, 2018

This method didn't work for me. I encountered a lot of errors. Also loaddata will consume a lot of memory. What worked for me was to run sequel from ruby. Just run the commands:
gem install sequel
You will need to have installed on your system the devel packages for postgres , sqlite and ruby
Run the command:
gem install pg sqlite3
Create an empty database on postgresql, let's say testDB and assign a grant permission to a user
From the command prompt run:
sequel -C sqlite:///path/to/sqlitedb.db postgres://user:password@host/testDB
This will run with no errors.
Change the settings in your django project to work with the postgres database
Run
./manage migrate (not necessary)
Run the server

@gabrielponto
Copy link

gabrielponto commented Aug 18, 2018

To not found error when use loaddata to load fixture, exclude contenttype and auth.Permission modules. So the dump line will be:

python manage.py dumpdata --exclude=contenttypes --exclude=auth.Permission > datadump.json

I found this help on: https://coderwall.com/p/kogbla/django-fixture-dumpdata-loaddata-and-integrity-error

@AhtishamChishti
Copy link

Guys I am getting django.db.utils.IntegrityError can anyone please help.

@bnelson365
Copy link

Thank you @gabriel, that worked for me

@mlazowik
Copy link

mlazowik commented Jan 9, 2019

In case anyone runs into issues like object has no attribute '_something_cache': if you're using custom pre_save or post_save handlers you need to handle the raw=True case in them: https://stackoverflow.com/a/19600001 https://docs.djangoproject.com/en/2.0/ref/django-admin/#what-s-a-fixture

@aniagarwal97
Copy link

Not working for me. I am getting this error

django.db.models.fields.related_descriptors.RelatedObjectDoesNotExist: Problem installing fixture '/home/ubuntu/news-manifest-master/datadump.json': Comment has no content_type.

@mbigatimothy
Copy link

Thanks a bunch. I followed this and it worked perfectly

@omidekz
Copy link

omidekz commented Aug 12, 2019

Thank YOU SO MUUUUUUUCH

@WilliamHH
Copy link

Indeed thank you for this post!

@AlanJui
Copy link

AlanJui commented Sep 28, 2019

😍🤩🥳👍

👏👏👏👏👏........................

@adsk2050
Copy link

adsk2050 commented Feb 9, 2020

Hello @sirodoht ! I have a question. If you remove ContentTypes then what else remains in the database?
I have created many pages in my website in db.sqlite3 database itself and I don't want to recreate everything again. So I need to include ContentTypes in the migration which is causing problems for me and returning ProgrammingError.

@ronaldlangeveld
Copy link

Hello @sirodoht ! I have a question. If you remove ContentTypes then what else remains in the database?
I have created many pages in my website in db.sqlite3 database itself and I don't want to recreate everything again. So I need to include ContentTypes in the migration which is causing problems for me and returning ProgrammingError.

You will add the backed up data after removing contenttypes.. so it won't be lost.

@anefta
Copy link

anefta commented Dec 15, 2020

Guys I am getting django.db.utils.IntegrityError can anyone please help.

Hey, @AhtishamChishti I am getting also this error..Any solution to this?

@aishwarya2202
Copy link

it is asking for login to enter in project.
could plz tell me its username and passward?

@amiaynara
Copy link

tables created, ok, but data were not inserted!

because datadump.json have no data user data

How did you resolve the issue. I carried out the instructions successfully, and it showed [Installed 156 objects from 1 fixture]. But there is no data when i see in the web-app or in the pgadmin4. Any help is appreciated.

@legioz
Copy link

legioz commented May 3, 2021

image

Error migrating from MySQL to MSSQL

@nhridoy
Copy link

nhridoy commented Aug 28, 2021

To not found error when use loaddata to load fixture, exclude contenttype and auth.Permission modules. So the dump line will be:

python manage.py dumpdata --exclude=contenttypes --exclude=auth.Permission > datadump.json

I found this help on: https://coderwall.com/p/kogbla/django-fixture-dumpdata-loaddata-and-integrity-error

This worked for me. Thanks

@kadius
Copy link

kadius commented Sep 10, 2021

image

Error migrating from MySQL to MSSQL

Try this:

> python manage.py shell
from django.contrib.contenttypes.models import ContentType
ContentType.objects.all().delete()

@clondon
Copy link

clondon commented Sep 18, 2021

To not found error when use loaddata to load fixture, exclude contenttype and auth.Permission modules. So the dump line will be:
python manage.py dumpdata --exclude=contenttypes --exclude=auth.Permission > datadump.json
I found this help on: https://coderwall.com/p/kogbla/django-fixture-dumpdata-loaddata-and-integrity-error

This worked for me. Thanks

This works for me also. I was not able to go back and do the data dump in the proper way as most of the web suggests.
I needed to have the loaddata adjusted to work.
Why does every assume you have the original data to do the dump again away?
If you have loaddata issues try the the suggestion at https://coderwall.com/p/kogbla/django-fixture-dumpdata-loaddata-and-integrity-error as stated above.

@safteinzz
Copy link

safteinzz commented Jan 15, 2023

For those with django.db.utils.IntegrityError duplicate entry.

Check if you have any signal creating content post_save, etc.. Comment those and run the process again.

https://stackoverflow.com/questions/75122526/psycopg2-duplicate-key-value-violates-unique-constraint-when-migrating-from-sql

@Know-Thyself
Copy link

It worked well for me to migrate database models and data from sqlite3 to PostgreSQL as promised.
Thank you, @scarabeo7 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment