Skip to content

Instantly share code, notes, and snippets.

@Fokko
Created April 18, 2024 11:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Fokko/ee94104a98742f4ca699bedccaf34a8f to your computer and use it in GitHub Desktop.
Save Fokko/ee94104a98742f4ca699bedccaf34a8f to your computer and use it in GitHub Desktop.
➜ pyiceberg-0.6.1 docker run -t -i -v `pwd`:/vo python:3.10 bash
root@7984a20cf808:/# cd /vo/
root@7984a20cf808:/vo# make install
pip install poetry==1.7.1
Collecting poetry==1.7.1
Downloading poetry-1.7.1-py3-none-any.whl (235 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 236.0/236.0 kB 10.0 MB/s eta 0:00:00
Collecting keyring<25.0.0,>=24.0.0
Downloading keyring-24.3.1-py3-none-any.whl (38 kB)
Collecting installer<0.8.0,>=0.7.0
Downloading installer-0.7.0-py3-none-any.whl (453 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 453.8/453.8 kB 16.1 MB/s eta 0:00:00
Collecting dulwich<0.22.0,>=0.21.2
Downloading dulwich-0.21.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (516 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 516.7/516.7 kB 19.8 MB/s eta 0:00:00
Collecting pyproject-hooks<2.0.0,>=1.0.0
Downloading pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)
Collecting requests-toolbelt<2,>=0.9.1
Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB 32.3 MB/s eta 0:00:00
Collecting requests<3.0,>=2.26
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 32.5 MB/s eta 0:00:00
Collecting shellingham<2.0,>=1.5
Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Collecting build<2.0.0,>=1.0.3
Downloading build-1.2.1-py3-none-any.whl (21 kB)
Collecting poetry-core==1.8.1
Downloading poetry_core-1.8.1-py3-none-any.whl (306 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 306.6/306.6 kB 25.6 MB/s eta 0:00:00
Collecting poetry-plugin-export<2.0.0,>=1.6.0
Downloading poetry_plugin_export-1.7.1-py3-none-any.whl (10 kB)
Collecting pkginfo<2.0.0,>=1.9.4
Downloading pkginfo-1.10.0-py3-none-any.whl (30 kB)
Collecting pexpect<5.0.0,>=4.7.0
Downloading pexpect-4.9.0-py2.py3-none-any.whl (63 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 kB 28.1 MB/s eta 0:00:00
Collecting tomlkit<1.0.0,>=0.11.4
Downloading tomlkit-0.12.4-py3-none-any.whl (37 kB)
Collecting virtualenv<21.0.0,>=20.23.0
Downloading virtualenv-20.25.3-py3-none-any.whl (3.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 15.2 MB/s eta 0:00:00
Collecting crashtest<0.5.0,>=0.4.1
Downloading crashtest-0.4.1-py3-none-any.whl (7.6 kB)
Collecting trove-classifiers>=2022.5.19
Downloading trove_classifiers-2024.4.10-py3-none-any.whl (13 kB)
Collecting cleo<3.0.0,>=2.1.0
Downloading cleo-2.1.0-py3-none-any.whl (78 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.7/78.7 kB 20.2 MB/s eta 0:00:00
Collecting cachecontrol[filecache]<0.14.0,>=0.13.0
Downloading cachecontrol-0.13.1-py3-none-any.whl (22 kB)
Collecting platformdirs<4.0.0,>=3.0.0
Downloading platformdirs-3.11.0-py3-none-any.whl (17 kB)
Collecting tomli<3.0.0,>=2.0.1
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting fastjsonschema<3.0.0,>=2.18.0
Downloading fastjsonschema-2.19.1-py3-none-any.whl (23 kB)
Collecting packaging>=20.5
Downloading packaging-24.0-py3-none-any.whl (53 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 kB 30.0 MB/s eta 0:00:00
Collecting msgpack>=0.5.2
Downloading msgpack-1.0.8-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (376 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 376.0/376.0 kB 50.1 MB/s eta 0:00:00
Collecting filelock>=3.8.0
Downloading filelock-3.13.4-py3-none-any.whl (11 kB)
Collecting rapidfuzz<4.0.0,>=3.0.0
Downloading rapidfuzz-3.8.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 29.8 MB/s eta 0:00:00
Collecting urllib3>=1.25
Downloading urllib3-2.2.1-py3-none-any.whl (121 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.1/121.1 kB 14.0 MB/s eta 0:00:00
Collecting SecretStorage>=3.2
Downloading SecretStorage-3.3.3-py3-none-any.whl (15 kB)
Collecting jaraco.classes
Downloading jaraco.classes-3.4.0-py3-none-any.whl (6.8 kB)
Collecting jeepney>=0.4.2
Downloading jeepney-0.8.0-py3-none-any.whl (48 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 24.7 MB/s eta 0:00:00
Collecting importlib-metadata>=4.11.4
Downloading importlib_metadata-7.1.0-py3-none-any.whl (24 kB)
Collecting ptyprocess>=0.5
Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting poetry-plugin-export<2.0.0,>=1.6.0
Downloading poetry_plugin_export-1.7.0-py3-none-any.whl (10 kB)
Downloading poetry_plugin_export-1.6.0-py3-none-any.whl (10 kB)
Collecting idna<4,>=2.5
Downloading idna-3.7-py3-none-any.whl (66 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 27.3 MB/s eta 0:00:00
Collecting certifi>=2017.4.17
Downloading certifi-2024.2.2-py3-none-any.whl (163 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.8/163.8 kB 41.5 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2
Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (138 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.2/138.2 kB 34.2 MB/s eta 0:00:00
Collecting distlib<1,>=0.3.7
Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 31.9 MB/s eta 0:00:00
Collecting zipp>=0.5
Downloading zipp-3.18.1-py3-none-any.whl (8.2 kB)
Collecting cryptography>=2.0
Downloading cryptography-42.0.5-cp39-abi3-manylinux_2_28_aarch64.whl (4.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 14.2 MB/s eta 0:00:00
Collecting more-itertools
Downloading more_itertools-10.2.0-py3-none-any.whl (57 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 15.6 MB/s eta 0:00:00
Collecting cffi>=1.12
Downloading cffi-1.16.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (446 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.0/446.0 kB 18.5 MB/s eta 0:00:00
Collecting pycparser
Downloading pycparser-2.22-py3-none-any.whl (117 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 32.7 MB/s eta 0:00:00
Installing collected packages: trove-classifiers, ptyprocess, fastjsonschema, distlib, zipp, urllib3, tomlkit, tomli, shellingham, rapidfuzz, pycparser, poetry-core, platformdirs, pkginfo, pexpect, packaging, msgpack, more-itertools, jeepney, installer, idna, filelock, crashtest, charset-normalizer, certifi, virtualenv, requests, pyproject-hooks, jaraco.classes, importlib-metadata, dulwich, cleo, cffi, requests-toolbelt, cryptography, cachecontrol, build, SecretStorage, keyring, poetry-plugin-export, poetry
Successfully installed SecretStorage-3.3.3 build-1.2.1 cachecontrol-0.13.1 certifi-2024.2.2 cffi-1.16.0 charset-normalizer-3.3.2 cleo-2.1.0 crashtest-0.4.1 cryptography-42.0.5 distlib-0.3.8 dulwich-0.21.7 fastjsonschema-2.19.1 filelock-3.13.4 idna-3.7 importlib-metadata-7.1.0 installer-0.7.0 jaraco.classes-3.4.0 jeepney-0.8.0 keyring-24.3.1 more-itertools-10.2.0 msgpack-1.0.8 packaging-24.0 pexpect-4.9.0 pkginfo-1.10.0 platformdirs-3.11.0 poetry-1.7.1 poetry-core-1.8.1 poetry-plugin-export-1.6.0 ptyprocess-0.7.0 pycparser-2.22 pyproject-hooks-1.0.0 rapidfuzz-3.8.1 requests-2.31.0 requests-toolbelt-1.0.0 shellingham-1.5.4 tomli-2.0.1 tomlkit-0.12.4 trove-classifiers-2024.4.10 urllib3-2.2.1 virtualenv-20.25.3 zipp-3.18.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.0.1 -> 24.0
[notice] To update, run: pip install --upgrade pip
poetry install -E pyarrow -E hive -E s3fs -E glue -E adlfs -E duckdb -E ray -E sql-postgres -E gcsfs -E sql-sqlite -E daft
Creating virtualenv pyiceberg-F02APa8F-py3.10 in /root/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...
Package operations: 151 installs, 0 updates, 0 removals
• Installing pycparser (2.22)
• Installing six (1.16.0)
• Installing attrs (23.2.0)
• Installing cffi (1.16.0)
• Installing jmespath (1.0.1)
• Installing pyasn1 (0.6.0)
• Installing python-dateutil (2.9.0.post0)
• Installing rpds-py (0.18.0)
• Installing urllib3 (2.2.1)
• Installing botocore (1.34.69)
• Installing cachetools (5.3.3)
• Installing certifi (2024.2.2)
• Installing charset-normalizer (3.3.2)
• Installing cryptography (42.0.5)
• Installing idna (3.7)
• Installing protobuf (5.26.1)
• Installing pyasn1-modules (0.4.0)
• Installing referencing (0.34.0)
• Installing rsa (4.9)
• Installing typing-extensions (4.9.0)
• Installing annotated-types (0.6.0)
• Installing frozenlist (1.4.1)
• Installing google-auth (2.29.0)
• Installing googleapis-common-protos (1.56.1)
• Installing jsonschema-specifications (2023.12.1)
• Installing markupsafe (2.1.5)
• Installing multidict (6.0.5)
• Installing pydantic-core (2.18.1)
• Installing pyjwt (2.8.0)
• Installing requests (2.31.0)
• Installing s3transfer (0.10.1)
• Installing aiosignal (1.3.1)
• Installing async-timeout (4.0.3)
• Installing blinker (1.7.0)
• Installing boto3 (1.34.69)
• Installing click (8.1.7)
• Installing google-api-core (2.8.0)
• Installing google-crc32c (1.5.0)
• Installing itsdangerous (2.2.0)
• Installing jinja2 (3.1.3)
• Installing jsonpickle (3.0.4)
• Installing jsonpointer (2.4)
• Installing jsonschema (4.21.1)
• Installing mpmath (1.3.0)
• Installing msal (1.28.0)
• Installing oauthlib (3.2.2)
• Installing packaging (24.0)
• Installing pathable (0.4.3)
• Installing pbr (6.0.0)
• Installing portalocker (2.8.2)
• Installing pydantic (2.7.0)
• Installing pyyaml (6.0.1)
• Installing rfc3339-validator (0.1.4)
• Installing tomli (2.0.1)
• Installing werkzeug (3.0.2)
• Installing yarl (1.9.4)
• Installing aiohttp (3.9.5)
• Installing aioitertools (0.11.0)
• Installing aws-sam-translator (1.87.0)
• Installing azure-core (1.30.1)
• Installing distlib (0.3.8)
• Installing exceptiongroup (1.2.0)
• Installing filelock (3.13.4)
• Installing flask (3.0.3)
• Installing google-cloud-core (2.4.1)
• Installing google-resumable-media (2.7.0)
• Installing iniconfig (2.0.0)
• Installing isodate (0.6.1)
• Installing jschema-to-python (1.2.3)
• Installing jsonpatch (1.33)
• Installing jsonschema-spec (0.1.3)
• Installing junit-xml (1.9)
• Installing lazy-object-proxy (1.10.0)
• Installing mdurl (0.1.2)
• Installing msal-extensions (1.1.0)
• Installing networkx (3.1)
• Installing numpy (1.26.4)
• Installing openapi-schema-validator (0.4.3)
• Installing platformdirs (4.2.0)
• Installing pluggy (1.4.0)
• Installing ply (3.11)
• Installing pyproject-hooks (1.0.0)
• Installing regex (2024.4.16)
• Installing requests-oauthlib (2.0.0)
• Installing sarif-om (1.0.4)
• Installing sympy (1.12)
• Installing wrapt (1.16.0)
• Installing aiobotocore (2.12.3)
• Installing antlr4-python3-runtime (4.13.1)
• Installing aws-xray-sdk (2.13.0)
• Installing azure-datalake-store (0.0.53)
• Installing azure-identity (1.16.0)
• Installing azure-storage-blob (12.19.1)
• Installing build (1.2.1)
• Installing cfgv (3.4.0)
• Installing cfn-lint (0.86.3)
• Installing decorator (5.1.1)
• Installing docker (7.0.0)
• Installing docutils (0.20.1)
• Installing flask-cors (4.0.0)
• Installing fsspec (2023.12.2)
• Installing google-auth-oauthlib (1.2.0)
• Installing google-cloud-storage (2.14.0)
• Installing graphql-core (3.2.3)
• Installing greenlet (3.0.3)
• Installing identify (2.5.35)
• Installing joserfc (0.9.0)
• Installing jsondiff (2.0.0)
• Installing jsonpath-ng (1.6.1)
• Installing markdown-it-py (3.0.0)
• Installing msgpack (1.0.8)
• Installing nodeenv (1.8.0)
• Installing openapi-spec-validator (0.5.5)
• Installing py-partiql-parser (0.5.4)
• Installing py4j (0.10.9.7)
• Installing pyarrow (15.0.2)
• Installing pygments (2.17.2)
• Installing pyparsing (3.1.2)
• Installing pytest (7.4.4)
• Installing pytz (2024.1)
• Installing responses (0.25.0)
• Installing tqdm (4.66.2)
• Installing tzdata (2024.1)
• Installing virtualenv (20.25.3)
• Installing xmltodict (0.13.0)
• Installing adlfs (2024.2.0)
• Installing coverage (7.4.4)
• Installing cython (3.0.8)
• Installing duckdb (0.10.2)
• Installing fastavro (1.9.4)
• Installing gcsfs (2023.12.2.post1)
• Installing getdaft (0.2.21)
• Installing mmhash3 (3.0.1)
• Installing moto (5.0.5)
• Installing mypy-boto3-glue (1.34.84)
• Installing pandas (2.0.3)
• Installing pre-commit (3.5.0)
• Installing psycopg2-binary (2.9.9)
• Installing pyspark (3.5.0)
• Installing pytest-checkdocs (2.10.1)
• Installing pytest-lazy-fixture (0.6.3)
• Installing pytest-mock (3.12.0)
• Installing ray (2.9.3)
• Installing requests-mock (1.11.0)
• Installing rich (13.7.1)
• Installing s3fs (2023.12.2)
• Installing sortedcontainers (2.4.0)
• Installing sqlalchemy (2.0.29)
• Installing strictyaml (1.7.3)
• Installing thrift (0.20.0)
• Installing zstandard (0.22.0)
Writing lock file
Preparing build environment with build-system requirements poetry-core>=1.0.0, wheel, Cython>=3.0.0, setuptools
Compiling pyiceberg/avro/decoder_fast.pyx because it changed.
[1/1] Cythonizing pyiceberg/avro/decoder_fast.pyx
Installing the current project: pyiceberg (0.6.1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment