Skip to content

Instantly share code, notes, and snippets.

@amilstead
amilstead / README.md
Created January 4, 2018 16:35
[Apache Beam] Python pkg_resources incompatibility

Apache Beam's Python SDK does not play nicely with the pkg_resources.require API. More generally, it fails due to badly defined requirement versions between grpcio and protobuf.

This makes using apache-beam in any python project which is consumed as a library itself incapable of using the SDK.

To see the failure, use the code in this gist and run:

virtualenv apache-beam-requirements && source apache-beam-requirements/bin/activate
pip install -e .

python runner.py

Keybase proof

I hereby claim:

  • I am amilstead on github.
  • I am amilstead (https://keybase.io/amilstead) on keybase.
  • I have a public key ASBNBWVhTAF_a8xtqxKki93EQ291wWwQnEhzPfsxmtd_BAo

To claim this, I am signing this object: