Skip to content

Instantly share code, notes, and snippets.

@defulmere
Last active April 19, 2024 15:00
Show Gist options
  • Star 16 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save defulmere/8b9695e415a44271061cc8e272f3c300 to your computer and use it in GitHub Desktop.
Save defulmere/8b9695e415a44271061cc8e272f3c300 to your computer and use it in GitHub Desktop.
How to override an old sqlite3 module with pysqlite3 in django settings.py
# ⚠️ USE AT YOUR OWN RISK
# first: pip install pysqlite3-binary
# then in settings.py:
# these three lines swap the stdlib sqlite3 lib with the pysqlite3 package
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
@defulmere
Copy link
Author

defulmere commented Oct 3, 2019

Django won't work with SQLite < 3.8.3 which is a problem on shared web hosts with Linux systems that have Python built against an older version of SQLite (looking at you, CentOS 7).

Some alternatives are:

  • Build a newer SQLite from source and then build Python against that, or...
  • Build a newer SQLite from source and set LD_LIBRARY_PATH to point at the updated libraries

Neither of these are ideal for most shared hosting customers who don't want to maintain their own custom builds, so IMO overriding sqlite3 (from the standard library) with pysqlite3 is a decent workaround.

@adhamselman
Copy link

This was helpful. Thank you!

@becker929
Copy link

Looks like you solved my problem! Thank you so much!

And for those googling:

I am trying to install Wagtail CMS on the host Webfaction while following the tutorial here: https://www.youtube.com/watch?v=1_yA25ZmNkA&list=PLMQHMcNi6ocsS8Bfnuy_IDgJ4bHRRrvub.

I was able to use Python's built-in venv to get around the fact that I couldn't use virtualenv or pipenv. However, I got stuck at
$ python3 manage.py migrate

I had the same error as above, where Django was complaining about needing SQLite version 3.8.3 or greater.

Then I followed the advice that support gave me:
"The latest versions of django 3 do not work with the sqlite3 version installed on our servers.
On our django installers we include a statically linked version of a newer sqlite version, as per https://github.com/coleifer/pysqlite3#building-a-statically-linked-library
You can run something similar within your virtualenv or try using the binary version."

... And I created a new environment, pip3 installed pysqlite3-binary, pip3 installed --upgrade django... but still got the same error when trying to migrate.

However, I went to mysite/settings/base.py and popped in the exact 3 lines you posted above, and voilà-- I can manage.py migrate!!

@aleonsan
Copy link

Thanks!

Worked for me, using pipenv in Debian 9.

@spapas
Copy link

spapas commented Apr 7, 2022

You saved me !!! I was trying for hours to compile Python 3.10 with a new version of sqlite3 on Centos 7 without luck. Using your solution worked in 5 minutes. Thank you so much !

@medhask
Copy link

medhask commented Aug 3, 2023

This was helpful, worked for me as well !

I was getting following error while trying to run chromadb example code using my python3.10.8 venv3.10:
File "~/venv3.10/lib/python3.10/site-packages/chromadb/__init__.py", line 36, in <module> raise RuntimeError( RuntimeError: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.

I executed following steps to resolve this error:

  1. Inside my python3.10.8's virtual environment i.e. venv3.10, installed pysqlite3-binary using command: pip install pysqlite3-binary
  2. Added these 3 lines in venv3.10/lib/python3.10/site-packages/chromadb/__init__.py at the beginning:
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

And my code worked like a charm!

@saifudeennouphal
Copy link

saifudeennouphal commented Aug 20, 2023

Can anyone explain about the BASE_DIR mentioned in the DATABASES
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),

I got an error message -->
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
NameError: name 'BASE_DIR' is not defined

help to get out of this
im on a streamlit project and im using python3.10

@le052301
Copy link

This was helpful, worked for me as well !

I was getting following error while trying to run chromadb example code using my python3.10.8 venv3.10: File "~/venv3.10/lib/python3.10/site-packages/chromadb/__init__.py", line 36, in <module> raise RuntimeError( RuntimeError: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.

I executed following steps to resolve this error:

  1. Inside my python3.10.8's virtual environment i.e. venv3.10, installed pysqlite3-binary using command: pip install pysqlite3-binary
  2. Added these 3 lines in venv3.10/lib/python3.10/site-packages/chromadb/__init__.py at the beginning:
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

And my code worked like a charm!

Thank you all!
I can also confirm that @medhask fix works for fixing ChromaDb when running in a 3.11 Debian DevContainer!

@hpwahyao
Copy link

Sorry guys, after following the steps above. Python complains a new error as below:

File "/usr/local/lib/python3.9/site-packages/chromadb/init.py", line 57, in
if sqlite3.sqlite_version_info < (3, 35, 0):
AttributeError: module 'pysqlite3' has no attribute 'sqlite_version_info'

@Vikho
Copy link

Vikho commented Sep 6, 2023

@hpwahyao hi friends I am also encountering with this error now, has you solved this problem?

@gwc4github
Copy link

I am having the same problem and I think I have tried everything listed here. I'm on a mac with Python 3.8.
I am trying upgrading to Python 3.11 now.
Note that I definitely have the newer version of sqlite3 on my machine. running sqlite3 reports 3.37.0
However, inside my code Chroma finds an older version somehow.

@defulmere
Copy link
Author

@le052301 @hpwahyao @Vikho @gwc4github if you look at the ChromaDB python client code you'll see that it's already equipped to use pysqlite3 but only if the google.colab module is installed:

https://github.com/chroma-core/chroma/blob/747f7c6457fdea29394cdedcfd7cd0bc0541b3eb/chromadb/__init__.py#L50C1-L72C10

You might be able to work around your issue by installing the google-colab and pysqlite-binary Python packages into your environment.

@ivsumitkumar
Copy link

ivsumitkumar commented Sep 13, 2023

Sorry guys, after following the steps above. Python complains a new error as below:

File "/usr/local/lib/python3.9/site-packages/chromadb/init.py", line 57, in if sqlite3.sqlite_version_info < (3, 35, 0): AttributeError: module 'pysqlite3' has no attribute 'sqlite_version_info'

Sorry guys, after following the steps above. Python complains a new error as below:

File "/usr/local/lib/python3.9/site-packages/chromadb/init.py", line 57, in if sqlite3.sqlite_version_info < (3, 35, 0): AttributeError: module 'pysqlite3' has no attribute 'sqlite_version_info'

  1. Go to inti.py file inside the venv3.10/lib/python3.10/site-packages/chromadb/__init__.py.
  2. goto line number 61. if sqlite3.sqlite_version_info < (3, 35, 0):
  3. and paste these three lines below it and make sure to indent it.
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')
  1. Now make sure to comment this:
if IN_COLAB:
        # In Colab, hotswap to pysqlite-binary if it's too old
        import subprocess
        import sys

        subprocess.check_call(
            [sys.executable, "-m", "pip", "install", "pysqlite3-binary"]
        )
        __import__("pysqlite3")
        sys.modules["sqlite3"] = sys.modules.pop("pysqlite3")
    else:
        raise RuntimeError(
            "\033[91mYour system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.\033[0m\n"
            "\033[94mPlease visit https://docs.trychroma.com/troubleshooting#sqlite to learn how to upgrade.\033[0m"
        )

It Worked for me when running in python 3.8 ubuntu 20.04

@selectorseb
Copy link

selectorseb commented Oct 2, 2023

You do not need to modify anything. To bypass the Colab check, you can create an empty folder google/colab in the site packages folder. That way on the Chroma __init__.py when it tries to do the import google.colab check it will pass, and thus replace sqlite3 with pysqlite3

@TheOnlyWayUp
Copy link

You do not need to modify anything. To bypass the Colab check, you can create an empty folder google/colab in the site packages folder. That way on the Chroma __init__.py when it tries to do the import google.colab check it will pass, and thus replace sqlite3 with pysqlite3

Thanks

@A-l-Mahi
Copy link

Sorry guys, after following the steps above. Python complains a new error as below:
File "/usr/local/lib/python3.9/site-packages/chromadb/init.py", line 57, in if sqlite3.sqlite_version_info < (3, 35, 0): AttributeError: module 'pysqlite3' has no attribute 'sqlite_version_info'

Sorry guys, after following the steps above. Python complains a new error as below:
File "/usr/local/lib/python3.9/site-packages/chromadb/init.py", line 57, in if sqlite3.sqlite_version_info < (3, 35, 0): AttributeError: module 'pysqlite3' has no attribute 'sqlite_version_info'

  1. Go to inti.py file inside the venv3.10/lib/python3.10/site-packages/chromadb/__init__.py.
  2. goto line number 61. if sqlite3.sqlite_version_info < (3, 35, 0):
  3. and paste these three lines below it and make sure to indent it.
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')
  1. Now make sure to comment this:
if IN_COLAB:
        # In Colab, hotswap to pysqlite-binary if it's too old
        import subprocess
        import sys

        subprocess.check_call(
            [sys.executable, "-m", "pip", "install", "pysqlite3-binary"]
        )
        __import__("pysqlite3")
        sys.modules["sqlite3"] = sys.modules.pop("pysqlite3")
    else:
        raise RuntimeError(
            "\033[91mYour system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.\033[0m\n"
            "\033[94mPlease visit https://docs.trychroma.com/troubleshooting#sqlite to learn how to upgrade.\033[0m"
        )

It Worked for me when running in python 3.8 ubuntu 20.04

It worked for me also

@tyler-suard-parker
Copy link

This did not solve the error for me.

@cmosta0
Copy link

cmosta0 commented Dec 19, 2023

I tried this couple of months ago and worked, however now it doesn't.
the issue is that I'm not even able to install the dependency:

pip install pysqlite3-binary  

ERROR: Could not find a version that satisfies the requirement pysqlite3-binary (from versions: none)
ERROR: No matching distribution found for pysqlite3-binary

I saw the packages are still with the same name on pypi and they were not removed, but not sure what changed and does not work now.

Any ideas?

@kalebima
Copy link

kalebima commented Jan 1, 2024

Bump, I'm having the same issue as @cmosta0 . Any workaround?

@cmosta0
Copy link

cmosta0 commented Jan 2, 2024

Bump, I'm having the same issue as @cmosta0 . Any workaround?

Hi. In my case, the issue seems to be related the OS. Actually to be able to install using pip, I needed to use --platform on pip install command, otherwise every attempt just returned the same error mentioned above. (I'm using macbook-pro M2)

@wry0313
Copy link

wry0313 commented Jan 11, 2024

@cmosta0 Hey what is the pip install command you used to successfully install the binary? Thanks in advance

@sunshicheng1
Copy link

thinks , u solve my problem, very thinks

@atozharsh
Copy link

@defulmere Thank you very much. This worked for me as well.

@himisir
Copy link

himisir commented Mar 13, 2024

This was helpful, worked for me as well !

I was getting following error while trying to run chromadb example code using my python3.10.8 venv3.10: File "~/venv3.10/lib/python3.10/site-packages/chromadb/__init__.py", line 36, in <module> raise RuntimeError( RuntimeError: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0.

I executed following steps to resolve this error:

  1. Inside my python3.10.8's virtual environment i.e. venv3.10, installed pysqlite3-binary using command: pip install pysqlite3-binary
  2. Added these 3 lines in venv3.10/lib/python3.10/site-packages/chromadb/__init__.py at the beginning:
__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

worked like magic! Thank you!

@cgorricho
Copy link

I tried this couple of months ago and worked, however now it doesn't. the issue is that I'm not even able to install the dependency:

pip install pysqlite3-binary  

ERROR: Could not find a version that satisfies the requirement pysqlite3-binary (from versions: none)
ERROR: No matching distribution found for pysqlite3-binary

I saw the packages are still with the same name on pypi and they were not removed, but not sure what changed and does not work now.

Any ideas?

It also happened to me, until I realized it only works on Linux.

See Chroma's documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment