Skip to content

Instantly share code, notes, and snippets.

@bruno-uy

bruno-uy/install_psycopg2_mac_m1.md

Last active March 19, 2024 10:10
Show Gist options
  • Save bruno-uy/f6c7618e7c3d917ba18fa0f7e2d05426 to your computer and use it in GitHub Desktop.
Save bruno-uy/f6c7618e7c3d917ba18fa0f7e2d05426 to your computer and use it in GitHub Desktop.
Download ZIP
Install psycopg2 in Mac M1
Raw
install_psycopg2_mac_m1.md

Install psycopg2 in Mac M1

Error while installing through pip install psycopg2 looks like this:

Please add the directory containing pg_config to the PATH

or specify the full executable path with the option (...)

Reference to the solution here.

Steps followed in my local

Install PostgreSQL, start the deamon and install openssl:

brew install postgresql
pg_ctl -D /opt/homebrew/var/postgres start
brew install openssl

If you already have openssl installed, run:

brew reinstall openssl

Find the path of pg_config and openssl:

which pg_config
which openssl

In my case those were located in these folders:

/opt/homebrew/Cellar/postgresql/13.3/bin/pg_config
/opt/homebrew/Cellar/openssl@1.1/1.1.1k/bin/openssl

Add that to $PATH:

export PATH=$PATH:/opt/homebrew/Cellar/postgresql/13.3/bin/pg_config
export PATH=$PATH:/opt/homebrew/Cellar/openssl@1.1/1.1.1k/bin/openssl

Then run: pip install psycopg2.

Problems while executing django command

I encountered the following error while running a django command:

Traceback (most recent call last):
  File "/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
    import psycopg2 as Database
  File "/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: dlopen(/Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so, 2): Symbol not found: _PQbackendPID
  Referenced from: /Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Users/user/opt/miniconda3/envs/dat/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-darwin.so

Following this thread, this finally worked:

brew install libpq --build-from-source
export LDFLAGS="-L/opt/homebrew/opt/libpq/lib"
pip install psycopg2-binary

Before using that, uninstall psycopg2 and psycopg2-binary:

pip uninstall psycopg2-binary
pip uninstall psycopg2
@SpencerMarks-IDHA
Copy link

SpencerMarks-IDHA commented Jun 6, 2022 •
edited
Loading

that didn't work for me....

 pip install psycopg2-binary
Collecting psycopg2-binary
  Using cached psycopg2-binary-2.9.3.tar.gz (380 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      running egg_info
      creating /private/var/folders/00/0cr867qs7_lcp7b5h323321r0000gn/T/pip-pip-egg-info-0kxer1_o/psycopg2_binary.egg-info
      writing /private/var/folders/00/0cr867qs7_lcp7b5h323321r0000gn/T/pip-pip-egg-info-0kxer1_o/psycopg2_binary.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/00/0cr867qs7_lcp7b5h323321r0000gn/T/pip-pip-egg-info-0kxer1_o/psycopg2_binary.egg-info/dependency_links.txt
      writing top-level names to /private/var/folders/00/0cr867qs7_lcp7b5h323321r0000gn/T/pip-pip-egg-info-0kxer1_o/psycopg2_binary.egg-info/top_level.txt
      writing manifest file '/private/var/folders/00/0cr867qs7_lcp7b5h323321r0000gn/T/pip-pip-egg-info-0kxer1_o/psycopg2_binary.egg-info/SOURCES.txt'

      Error: pg_config executable not found.

      pg_config is required to build psycopg2 from source.  Please add the directory
      containing pg_config to the $PATH or specify the full executable path with the
      option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      If you prefer to avoid building psycopg2 from source, please install the PyPI
      'psycopg2-binary' package instead.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.```

@SpencerMarks-IDHA
Copy link

My fix was to install postgresql locally (which I didn't realize was a requirement) Nor did I want to have postgresql installed locally since I am using remote instances of it.

@bruno-uy
Copy link
Author

bruno-uy commented Jun 6, 2022

Thanks for the comment @SpencerMarks-IDHA !

@Goldziher
Copy link

Goldziher commented Oct 18, 2022 •
edited
Loading

Thanks for this.

Just an update. I use which pg_config and which openssl - simpler.

@bruno-uy
Copy link
Author

Thanks for this.

Just an update. I use which pg_config and which openssl - simpler.

Way simpler @Goldziher! Thanks for that! I updated the gist with your recommendation.

@J-0RI
Copy link

J-0RI commented Oct 19, 2022

This was wayyyyy simple compared to the others I've seen

@bruno-uy
Copy link
Author

This was wayyyyy simple compared to the others I've seen

I'm glad it helps @J-0RI 🙂

@misterEggroll
Copy link

This is what I did to solve the problem:

1. Reinstall Openssl

% brew reinstall openssl                                                                          
==> Fetching openssl@3
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.1.1
Already downloaded: /Users/<username>/Library/Caches/Homebrew/downloads/a3831d258a8ccc63638c4ae11c47987af956f6db04046977e1583c1410df00a8--openssl@3-3.1.1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:ec0a9f2f744809e81aca2a8953479b490b2c5341238615540c15c74019187ac0
Already downloaded: /Users/<username>/Library/Caches/Homebrew/downloads/10e5960204046ee0ca02040ea350c3d7134158d7b491e148ae41c5ef58b3d475--openssl@3--3.1.1.arm64_ventura.bottle.tar.gz
==> Reinstalling openssl@3 
==> Pouring openssl@3--3.1.1.arm64_ventura.bottle.tar.gz
🍺  /opt/homebrew/Cellar/openssl@3/3.1.1: 6,495 files, 28.4MB
==> Running `brew cleanup openssl@3`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

2. Find out folder locations used to set LDFLAGS

After this, I checked where the bin and include folders are for openssl.

% ls /opt/homebrew/Cellar/openssl@3/3.1.1
AUTHORS.md              INSTALL_RECEIPT.json    NEWS.md                 bin                     lib
CHANGES.md              LICENSE.txt             README.md               include                 share

3. Install psycopg2 using LDFLAGS

Now that you know the folder locations, install psycopg2 using env vars:

% env LDFLAGS="-I/opt/homebrew/Cellar/openssl@3/3.1.1/include -L/opt/homebrew/Cellar/openssl@3/3.1.1/lib" pip install psycopg2
Collecting psycopg2
  Using cached psycopg2-2.9.6.tar.gz (383 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... done
  Created wheel for psycopg2: filename=psycopg2-2.9.6-cp310-cp310-macosx_13_0_arm64.whl size=145782 sha256=701cefbc36bec6bf31751dc21888ee2d96072af99ade9c45a9c7c687d3a7461e
  Stored in directory: /Users/<user>/Library/Caches/pip/wheels/a2/65/83/78e6f42d3b8e22115e894576b71799d96ab5a790b8f7bcfa85
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.9.6

@bruno-uy
Copy link
Author

Thanks @misterEggroll! So the only difference is that you installed psycopg2 with the LDFLAGS, right?

@rafa-munoz
Copy link

rafa-munoz commented Aug 11, 2023 •
edited
Loading

This is what worked for me in a M2 processor, because your instructions didn't work for me, since it was still complaining for not finding pg_config:

brew install postgresql@15

Note that you need to specify the version with @. Otherwise, it won't install.

After installing, you will see the message:

==> Caveats
This formula has created a default database cluster with:
  initdb --locale=C -E UTF-8 /opt/homebrew/var/postgresql@15
For more details, read:
  https://www.postgresql.org/docs/15/app-initdb.html

postgresql@15 is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.

If you need to have postgresql@15 first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"' >> ~/.zshrc

For compilers to find postgresql@15 you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

To start postgresql@15 now and restart at login:
  brew services start postgresql@15
Or, if you don't want/need a background service you can just run:
  LC_ALL="C" /opt/homebrew/opt/postgresql@15/bin/postgres -D /opt/homebrew/var/postgresql@15
==> Summary
🍺  /opt/homebrew/Cellar/postgresql@15/15.4: 3,698 files, 61.5MB

Just copy the export commands and do one by one the export:

export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"

export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"

export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

Now do the install: pip install psycopg2.

@bruno-uy
Copy link
Author

Thanks for all the details @rafa-munoz!

@dhruv2981
Copy link

Just copy export commands and dont forget to run source ~/.zshrc for zsh shell .Use command accordingly for bash.
Now do the install: pip install psycopg2.

@huxzy
Copy link

huxzy commented Oct 14, 2023

This is what worked for me in a M2 processor, because your instructions didn't work for me, since it was still complaining for not finding pg_config:

brew install postgresql@15

Note that you need to specify the version with @. Otherwise, it won't install.

After installing, you will see the message:

==> Caveats
This formula has created a default database cluster with:
  initdb --locale=C -E UTF-8 /opt/homebrew/var/postgresql@15
For more details, read:
  https://www.postgresql.org/docs/15/app-initdb.html

postgresql@15 is keg-only, which means it was not symlinked into /opt/homebrew,
because this is an alternate version of another formula.

If you need to have postgresql@15 first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"' >> ~/.zshrc

For compilers to find postgresql@15 you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

To start postgresql@15 now and restart at login:
  brew services start postgresql@15
Or, if you don't want/need a background service you can just run:
  LC_ALL="C" /opt/homebrew/opt/postgresql@15/bin/postgres -D /opt/homebrew/var/postgresql@15
==> Summary
🍺  /opt/homebrew/Cellar/postgresql@15/15.4: 3,698 files, 61.5MB

Just copy the export commands and do one by one the export:

export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"

export LDFLAGS="-L/opt/homebrew/opt/postgresql@15/lib"

export CPPFLAGS="-I/opt/homebrew/opt/postgresql@15/include"

Now do the install: pip install psycopg2.

This worked for me on M2 processor. Thanks

@cyb3rko
Copy link

cyb3rko commented Mar 19, 2024

Thank you!! I've tried so many different approaches until I've found this here.
The Apple silicon really is a pain in terms of software support...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment