Installation instructions for libicu-dev, PyICU, libpostal, pypostal on Mac OS X 10.14+

Installing PyICU, libpostal, pypostal on Mac OS X 10.14+.md

Installing PyICU, libpostal, pypostal on Mac OS X 10.14+

libicu-dev (PyICU dependency)

brew uninstall --ignore-dependencies icu4c
brew install pkg-config icu4c  # keg-only

PyICU (pypostal dependency)

pip uninstall pyicu

export PATH="/usr/local/opt/icu4c/bin:/usr/local/opt/icu4c/sbin:$PATH"
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:/usr/local/opt/icu4c/lib/pkgconfig"

# ensure system clang is used for proper libstdc++ https://github.com/ovalhub/pyicu/issues/5#issuecomment-291631507
unset CC CXX

pip install --no-binary=:pyicu: pyicu  # avoid wheels from previous runs or PyPI

libpostal (pypostal dependency)

One-liner (using HEAD as v1.1 was released in 2018):

brew install --HEAD robsalasco/libpostal/libpostal

Or manually:

brew install curl autoconf automake libtool pkg-config gcc@8

export CC="/usr/local/bin/gcc-8"
alias gcc="gcc-8"

git clone https://github.com/openvenues/libpostal.git
cd libpostal
./bootstrap.sh
./configure --datadir="${HOME}/.libpostal"
make -j4
sudo make install

pypostal

pip install postal

# test installation by parsing an example address
python -c "from postal.parser import parse_address; my_address = '123 Beech Lake Ct. Roswell, GA 30076'; print({v: k for k, v in dict(parse_address(my_address)).items()})"
# {'house_number': '123', 'road': 'beech lake ct.', 'city': 'roswell', 'state': 'ga', 'postcode': '30076'}

hello from OS X 10.5 followed all the steps and still on import icu gettings this:

Python 3.7.4 (default, Jul  9 2019, 18:13:23)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import icu
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/andi/pyicu/icu/__init__.py", line 37, in <module>
    from _icu import *
ImportError: dlopen(/usr/local/lib/python3.7/site-packages/_icu.cpython-37m-darwin.so, 2): Symbol not found: __ZN6icu_648ByteSink15GetAppendBufferEiiPciPi
  Referenced from: /usr/local/lib/python3.7/site-packages/_icu.cpython-37m-darwin.so
  Expected in: flat namespace
 in /usr/local/lib/python3.7/site-packages/_icu.cpython-37m-darwin.so

Hey @andilabs
I have seen this error before during initial installation, and I'm not 100% sure what causes/solves this issue on MacOS. But after repeated trial & error I managed to get it to work by starting from scratch and I haven't touched the installation since.

There is probably some conflict with the system-installed ICU or some issue with (possibly multiple) icu-config alias(es) on your system (see this comment by the PyICU dev: ovalhub/pyicu#112 (comment)).

Please let me know if you solve this issue, and what solved it. We can adapt the instructions to help others with this strange issue.

For what it's worth, I'm using gcc@8 as system wide default C++ compiler as opposed to the default clang.

Just for a reference, tried the method you mentioned and it won't work for me no matter how many times I start from scratch.
And at last I do pyenv uninstall the whole virtualenv, create a new one, and do the following, and it works! (does not even require to reinstall icu4c).

CFLAGS=-I/usr/local/opt/icu4c/include LDFLAGS=-L/usr/local/opt/icu4c/lib pip install pyicu==2.3.1

Hi @iyabchen, thanks for your reply. I just did a succesfull install of pyicu==2.6.0 on a mint machine, by making sure its setup.py (which attempts to use the deprecated icu-config, but falls back to pkg-config) finds brew's keg-only icu pkgconfig. No more manual setting of compiler flags :)

I've updated the gist with this simplification, and also added brew install robsalasco/libpostal/libpostal.

Just ran into this myself and creating a new virtualenv with pyenv did the trick as well, didn't have to reinstall pkg-config or icu4c after having installed with homebrew.

Running libpostal on Ubuntu: https://github.com/ddelange/libpostal-fastapi

I am having no end of trouble getting postal installed on an M1 Mac running OX 13.1 (Ventura). I have installed and uninstalled lib postal and the BREW command above seems to work flawlessly. However, in every install, I'm getting errors building the wheel. I've tried using the --use-pep517 argument and get the same result.

lovejodh ~ % pip install postal
Collecting postal
Using cached postal-1.1.10.tar.gz (21 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: six in /opt/homebrew/lib/python3.11/site-packages (from postal) (1.16.0)
Building wheels for collected packages: postal
Building wheel for postal (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [40 lines of output]
/opt/homebrew/lib/python3.11/site-packages/setuptools/dist.py:788: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
warnings.warn(
/opt/homebrew/lib/python3.11/site-packages/setuptools/init.py:85: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. Requirements should be satisfied by a PEP 517 installer. If you are using pip, you can try pip install --use-pep517.
dist.fetch_build_eggs(dist.setup_requires)
/opt/homebrew/lib/python3.11/site-packages/setuptools/dist.py:788: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-13-arm64-cpython-311
creating build/lib.macosx-13-arm64-cpython-311/postal
copying postal/token_types.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/dedupe.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/normalize.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/init.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/parser.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/near_dupe.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/tokenize.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/expand.py -> build/lib.macosx-13-arm64-cpython-311/postal
creating build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/test_parser.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/init.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/test_expand.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/_test_near_dupes.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
creating build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/encoding.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/init.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/omitted.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/enum.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/pyutils.h -> build/lib.macosx-13-arm64-cpython-311/postal
running build_ext
building 'postal._expand' extension
creating build/temp.macosx-13-arm64-cpython-311
creating build/temp.macosx-13-arm64-cpython-311/postal
clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -I/Users/lovejodh/libpostal/include -L/Users/lovejodh/libpostal/lib` pip install postal -I/usr/local/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c postal/pyexpand.c -o build/temp.macosx-13-arm64-cpython-311/postal/pyexpand.o -std=c99
clang: error: no such file or directory: 'pip'
clang: error: no such file or directory: 'install'
clang: warning: postal: 'linker' input unused [-Wunused-command-line-argument]
error: command '/usr/bin/clang' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for postal
Running setup.py clean for postal
Failed to build postal
ERROR: Could not build wheels for postal, which is required to install pyproject.toml-based projects
lovejodh@MBP-P32402PG63-US ~ % pip install postal --use-pep517
Collecting postal
Using cached postal-1.1.10.tar.gz (21 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: six in /opt/homebrew/lib/python3.11/site-packages (from postal) (1.16.0)
Building wheels for collected packages: postal
Building wheel for postal (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for postal (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [36 lines of output]
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/dist.py:788: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-13-arm64-cpython-311
creating build/lib.macosx-13-arm64-cpython-311/postal
copying postal/token_types.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/dedupe.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/normalize.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/init.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/parser.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/near_dupe.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/tokenize.py -> build/lib.macosx-13-arm64-cpython-311/postal
copying postal/expand.py -> build/lib.macosx-13-arm64-cpython-311/postal
creating build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/test_parser.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/init.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/test_expand.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
copying postal/tests/_test_near_dupes.py -> build/lib.macosx-13-arm64-cpython-311/postal/tests
creating build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/encoding.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/init.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/omitted.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/utils/enum.py -> build/lib.macosx-13-arm64-cpython-311/postal/utils
copying postal/pyutils.h -> build/lib.macosx-13-arm64-cpython-311/postal
running build_ext
building 'postal._expand' extension
creating build/temp.macosx-13-arm64-cpython-311
creating build/temp.macosx-13-arm64-cpython-311/postal
clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -I/Users/lovejodh/libpostal/include -L/Users/lovejodh/libpostal/lib` pip install postal -I/usr/local/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c postal/pyexpand.c -o build/temp.macosx-13-arm64-cpython-311/postal/pyexpand.o -std=c99
clang: error: no such file or directory: 'pip'
clang: error: no such file or directory: 'install'
clang: warning: postal: 'linker' input unused [-Wunused-command-line-argument]
error: command '/usr/bin/clang' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for postal
Failed to build postal
ERROR: Could not build wheels for postal, which is required to install pyproject.toml-based projects

Hi 👋

This looks weird to me (don't understand why there would be a backtick (`) in that line for instance, also the pip install is weird 🤔

clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -I/Users/lovejodh/libpostal/include -L/Users/lovejodh/libpostal/lib` pip install postal -I/usr/local/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c postal/pyexpand.c -o build/temp.macosx-13-arm64-cpython-311/postal/pyexpand.o -std=c99
clang: error: no such file or directory: 'pip'
clang: error: no such file or directory: 'install'

I can suggest running pip install -U pip setuptools wheel before running pip install postal, but doubt that it'll help. There are also differences in how you installed Python (I see yours is clang based but you can also compile CPython with gcc). I can recommend pyenv, see https://github.com/ddelange/new-mac-setup.

I've ported this gist to a Dockerfile which is ubuntu based.

A 'simple' pip install should be enough also on newer Macs (I'm on 10.15 so can't judge though), because either way, their setup.py is completely platform-agnostic.

Good luck!

Well, I figured it out. I was finally able to get the installer to see the .h file.

CFLAGS="-I/opt/homebrew/Cellar/libpostal/HEAD-8f2066b/include" LDFLAGS="-L/opt/homebrew/Cellar/libpostal/HEAD-8f2066b/lib" pip install postal

If you run into this, note that the include contains the directory name. So fi you set a CFLAG or a symlink that includes the directory name, it's going to miss it.
#include <libpostal/libpostal.h>
^~~~~~~~~~~~~~~~~~~~~~~

Awesome, thanks for reporting back!

Thank you SO much for responding @ddelange. I still can't get the postal scrubbing to work in scrubadub.py, but that's a problem for another day. It's not because it can't find libpostal, at least. The likelihood of there being postal data in this repository is nil, so I'm going to leave it for now.