Skip to content

Instantly share code, notes, and snippets.

@cmdcolin

cmdcolin/gist:7bf3661b24f6baadba37866af885e8a6

Last active September 25, 2024 19:04
Show Gist options
  • Save cmdcolin/7bf3661b24f6baadba37866af885e8a6 to your computer and use it in GitHub Desktop.
Save cmdcolin/7bf3661b24f6baadba37866af885e8a6 to your computer and use it in GitHub Desktop.
Download ZIP
attempts at using Tiberius
Raw
gistfile1.txt
~/Tiberius ❯❯❯ python3 -m venv .venv ✘ 130
~/Tiberius ❯❯❯ .venv/bin/activate
zsh: command not found: .venv/bin/activate
~/Tiberius ❯❯❯ source .venv/bin/activate ✘ 127
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow_probability==0.18.0 transformers pyBigWig bio scikit-learn biopython bcbio-gff requests
Collecting tensorflow_probability==0.18.0
Downloading tensorflow_probability-0.18.0-py2.py3-none-any.whl.metadata (13 kB)
Collecting transformers
Downloading transformers-4.44.2-py3-none-any.whl.metadata (43 kB)
Collecting pyBigWig
Downloading pyBigWig-0.3.23-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting bio
Downloading bio-1.7.1-py3-none-any.whl.metadata (5.7 kB)
Collecting scikit-learn
Downloading scikit_learn-1.5.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB)
Collecting biopython
Downloading biopython-1.84-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting bcbio-gff
Downloading bcbio_gff-0.7.1-py3-none-any.whl.metadata (343 bytes)
Collecting requests
Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting absl-py (from tensorflow_probability==0.18.0)
Downloading absl_py-2.1.0-py3-none-any.whl.metadata (2.3 kB)
Collecting six>=1.10.0 (from tensorflow_probability==0.18.0)
Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting numpy>=1.13.3 (from tensorflow_probability==0.18.0)
Downloading numpy-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Collecting decorator (from tensorflow_probability==0.18.0)
Downloading decorator-5.1.1-py3-none-any.whl.metadata (4.0 kB)
Collecting cloudpickle>=1.3 (from tensorflow_probability==0.18.0)
Downloading cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB)
Collecting gast>=0.3.2 (from tensorflow_probability==0.18.0)
Downloading gast-0.6.0-py3-none-any.whl.metadata (1.3 kB)
Collecting dm-tree (from tensorflow_probability==0.18.0)
Downloading dm_tree-0.1.8-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.9 kB)
Collecting filelock (from transformers)
Downloading filelock-3.16.1-py3-none-any.whl.metadata (2.9 kB)
Collecting huggingface-hub<1.0,>=0.23.2 (from transformers)
Downloading huggingface_hub-0.25.1-py3-none-any.whl.metadata (13 kB)
Collecting packaging>=20.0 (from transformers)
Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pyyaml>=5.1 (from transformers)
Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting regex!=2019.12.17 (from transformers)
Downloading regex-2024.9.11-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
Collecting safetensors>=0.4.1 (from transformers)
Downloading safetensors-0.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting tokenizers<0.20,>=0.19 (from transformers)
Downloading tokenizers-0.19.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
Collecting tqdm>=4.27 (from transformers)
Downloading tqdm-4.66.5-py3-none-any.whl.metadata (57 kB)
Collecting gprofiler-official (from bio)
Downloading gprofiler_official-1.0.0-py3-none-any.whl.metadata (11 kB)
Collecting mygene (from bio)
Downloading mygene-3.2.2-py2.py3-none-any.whl.metadata (10 kB)
Collecting pandas (from bio)
Downloading pandas-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (89 kB)
Collecting pooch (from bio)
Downloading pooch-1.8.2-py3-none-any.whl.metadata (10 kB)
Collecting scipy>=1.6.0 (from scikit-learn)
Downloading scipy-1.14.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Collecting joblib>=1.2.0 (from scikit-learn)
Downloading joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB)
Collecting threadpoolctl>=3.1.0 (from scikit-learn)
Downloading threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
Collecting charset-normalizer<4,>=2 (from requests)
Downloading charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests)
Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests)
Downloading urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests)
Downloading certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Collecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.23.2->transformers)
Downloading fsspec-2024.9.0-py3-none-any.whl.metadata (11 kB)
Collecting typing-extensions>=3.7.4.3 (from huggingface-hub<1.0,>=0.23.2->transformers)
Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting biothings-client>=0.2.6 (from mygene->bio)
Downloading biothings_client-0.3.1-py2.py3-none-any.whl.metadata (9.8 kB)
Collecting python-dateutil>=2.8.2 (from pandas->bio)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas->bio)
Downloading pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas->bio)
Downloading tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting platformdirs>=2.5.0 (from pooch->bio)
Downloading platformdirs-4.3.6-py3-none-any.whl.metadata (11 kB)
Downloading tensorflow_probability-0.18.0-py2.py3-none-any.whl (6.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 11.6 MB/s eta 0:00:00
Downloading transformers-4.44.2-py3-none-any.whl (9.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.5/9.5 MB 31.2 MB/s eta 0:00:00
Downloading pyBigWig-0.3.23-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (188 kB)
Downloading bio-1.7.1-py3-none-any.whl (280 kB)
Downloading scikit_learn-1.5.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.9/12.9 MB 33.8 MB/s eta 0:00:00
Downloading biopython-1.84-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 31.7 MB/s eta 0:00:00
Downloading bcbio_gff-0.7.1-py3-none-any.whl (16 kB)
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
Downloading certifi-2024.8.30-py3-none-any.whl (167 kB)
Downloading charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
Downloading cloudpickle-3.0.0-py3-none-any.whl (20 kB)
Downloading gast-0.6.0-py3-none-any.whl (21 kB)
Downloading huggingface_hub-0.25.1-py3-none-any.whl (436 kB)
Downloading idna-3.10-py3-none-any.whl (70 kB)
Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
Downloading numpy-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.0/16.0 MB 39.1 MB/s eta 0:00:00
Downloading packaging-24.1-py3-none-any.whl (53 kB)
Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 26.2 MB/s eta 0:00:00
Downloading regex-2024.9.11-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (797 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 797.0/797.0 kB 28.0 MB/s eta 0:00:00
Downloading safetensors-0.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (434 kB)
Downloading scipy-1.14.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (40.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 MB 37.1 MB/s eta 0:00:00
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Downloading tokenizers-0.19.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 35.1 MB/s eta 0:00:00
Downloading tqdm-4.66.5-py3-none-any.whl (78 kB)
Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
Downloading absl_py-2.1.0-py3-none-any.whl (133 kB)
Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
Downloading dm_tree-0.1.8-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (152 kB)
Downloading filelock-3.16.1-py3-none-any.whl (16 kB)
Downloading gprofiler_official-1.0.0-py3-none-any.whl (9.3 kB)
Downloading mygene-3.2.2-py2.py3-none-any.whl (5.4 kB)
Downloading pandas-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.7/12.7 MB 28.3 MB/s eta 0:00:00
Downloading pooch-1.8.2-py3-none-any.whl (64 kB)
Downloading biothings_client-0.3.1-py2.py3-none-any.whl (29 kB)
Downloading fsspec-2024.9.0-py3-none-any.whl (179 kB)
Downloading platformdirs-4.3.6-py3-none-any.whl (18 kB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading pytz-2024.2-py2.py3-none-any.whl (508 kB)
Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Downloading tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Installing collected packages: pytz, dm-tree, urllib3, tzdata, typing-extensions, tqdm, threadpoolctl, six, safetensors, regex, pyyaml, pyBigWig, platformdirs, packaging, numpy, joblib, idna, gast, fsspec, filelock, decorator, cloudpickle, charset-normalizer, certifi, absl-py, tensorflow_probability, scipy, requests, python-dateutil, biopython, scikit-learn, pooch, pandas, huggingface-hub, gprofiler-official, biothings-client, bcbio-gff, tokenizers, mygene, transformers, bio
Successfully installed absl-py-2.1.0 bcbio-gff-0.7.1 bio-1.7.1 biopython-1.84 biothings-client-0.3.1 certifi-2024.8.30 charset-normalizer-3.3.2 cloudpickle-3.0.0 decorator-5.1.1 dm-tree-0.1.8 filelock-3.16.1 fsspec-2024.9.0 gast-0.6.0 gprofiler-official-1.0.0 huggingface-hub-0.25.1 idna-3.10 joblib-1.4.2 mygene-3.2.2 numpy-2.1.1 packaging-24.1 pandas-2.2.3 platformdirs-4.3.6 pooch-1.8.2 pyBigWig-0.3.23 python-dateutil-2.9.0.post0 pytz-2024.2 pyyaml-6.0.2 regex-2024.9.11 requests-2.32.3 safetensors-0.4.5 scikit-learn-1.5.2 scipy-1.14.1 six-1.16.0 tensorflow_probability-0.18.0 threadpoolctl-3.5.0 tokenizers-0.19.1 tqdm-4.66.5 transformers-4.44.2 typing-extensions-4.12.2 tzdata-2024.2 urllib3-2.2.3
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --fasta ~/Downloads/ncbi_dataset/data/GCF_027595985.1/..
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --fasta ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 11, in <module>
import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow-gpu==2.10.* ✘ 1
zsh: no matches found: tensorflow-gpu==2.10.*
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow-gpu ✘ 1
Collecting tensorflow-gpu
Downloading tensorflow-gpu-2.12.0.tar.gz (2.6 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [58 lines of output]
Traceback (most recent call last):
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/requirements.py", line 36, in __init__
parsed = _parse_requirement(requirement_string)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/_parser.py", line 62, in parse_requirement
return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/_parser.py", line 80, in _parse_requirement
url, specifier, marker = _parse_requirement_details(tokenizer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/_parser.py", line 124, in _parse_requirement_details
marker = _parse_requirement_marker(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/_parser.py", line 145, in _parse_requirement_marker
tokenizer.raise_syntax_error(
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/_tokenizer.py", line 167, in raise_syntax_error
raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after name and no valid version specifier)
python_version>"3.7"
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
super().run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
exec(code, locals())
File "<string>", line 40, in <module>
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 116, in setup
_install_setup_requires(attrs)
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 87, in _install_setup_requires
dist.parse_config_files(ignore_option_errors=True)
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 610, in parse_config_files
self._finalize_requires()
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 344, in _finalize_requires
self._normalize_requires()
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 359, in _normalize_requires
self.install_requires = list(map(str, _reqs.parse(install_requires)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-mp61psez/overlay/lib/python3.12/site-packages/setuptools/_vendor/packaging/requirements.py", line 38, in __init__
raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected end or semicolon (after name and no valid version specifier)
python_version>"3.7"
^
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
(.venv) ~/Tiberius ❯❯❯ python --version ✘ 1
Python 3.12.6
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow
Collecting tensorflow
Downloading tensorflow-2.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)
Requirement already satisfied: absl-py>=1.0.0 in ./.venv/lib/python3.12/site-packages (from tensorflow) (2.1.0)
Collecting astunparse>=1.6.0 (from tensorflow)
Downloading astunparse-1.6.3-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting flatbuffers>=24.3.25 (from tensorflow)
Downloading flatbuffers-24.3.25-py2.py3-none-any.whl.metadata (850 bytes)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in ./.venv/lib/python3.12/site-packages (from tensorflow) (0.6.0)
Collecting google-pasta>=0.1.1 (from tensorflow)
Downloading google_pasta-0.2.0-py3-none-any.whl.metadata (814 bytes)
Collecting h5py>=3.10.0 (from tensorflow)
Downloading h5py-3.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting libclang>=13.0.0 (from tensorflow)
Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting ml-dtypes<0.5.0,>=0.3.1 (from tensorflow)
Downloading ml_dtypes-0.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Collecting opt-einsum>=2.3.2 (from tensorflow)
Downloading opt_einsum-3.3.0-py3-none-any.whl.metadata (6.5 kB)
Requirement already satisfied: packaging in ./.venv/lib/python3.12/site-packages (from tensorflow) (24.1)
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow)
Downloading protobuf-4.25.5-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Requirement already satisfied: requests<3,>=2.21.0 in ./.venv/lib/python3.12/site-packages (from tensorflow) (2.32.3)
Collecting setuptools (from tensorflow)
Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)
Requirement already satisfied: six>=1.12.0 in ./.venv/lib/python3.12/site-packages (from tensorflow) (1.16.0)
Collecting termcolor>=1.1.0 (from tensorflow)
Downloading termcolor-2.4.0-py3-none-any.whl.metadata (6.1 kB)
Requirement already satisfied: typing-extensions>=3.6.6 in ./.venv/lib/python3.12/site-packages (from tensorflow) (4.12.2)
Collecting wrapt>=1.11.0 (from tensorflow)
Downloading wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow)
Downloading grpcio-1.66.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.9 kB)
Collecting tensorboard<2.18,>=2.17 (from tensorflow)
Downloading tensorboard-2.17.1-py3-none-any.whl.metadata (1.6 kB)
Collecting keras>=3.2.0 (from tensorflow)
Downloading keras-3.5.0-py3-none-any.whl.metadata (5.8 kB)
Collecting numpy<2.0.0,>=1.26.0 (from tensorflow)
Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting wheel<1.0,>=0.23.0 (from astunparse>=1.6.0->tensorflow)
Downloading wheel-0.44.0-py3-none-any.whl.metadata (2.3 kB)
Collecting rich (from keras>=3.2.0->tensorflow)
Downloading rich-13.8.1-py3-none-any.whl.metadata (18 kB)
Collecting namex (from keras>=3.2.0->tensorflow)
Downloading namex-0.0.8-py3-none-any.whl.metadata (246 bytes)
Collecting optree (from keras>=3.2.0->tensorflow)
Downloading optree-0.12.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (47 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow) (2024.8.30)
Collecting markdown>=2.6.8 (from tensorboard<2.18,>=2.17->tensorflow)
Downloading Markdown-3.7-py3-none-any.whl.metadata (7.0 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.18,>=2.17->tensorflow)
Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.18,>=2.17->tensorflow)
Downloading werkzeug-3.0.4-py3-none-any.whl.metadata (3.7 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.18,>=2.17->tensorflow)
Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting markdown-it-py>=2.2.0 (from rich->keras>=3.2.0->tensorflow)
Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich->keras>=3.2.0->tensorflow)
Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich->keras>=3.2.0->tensorflow)
Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Downloading tensorflow-2.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (601.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 601.4/601.4 MB 34.8 MB/s eta 0:00:00
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Downloading flatbuffers-24.3.25-py2.py3-none-any.whl (26 kB)
Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
Downloading grpcio-1.66.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 35.8 MB/s eta 0:00:00
Downloading h5py-3.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 34.4 MB/s eta 0:00:00
Downloading keras-3.5.0-py3-none-any.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 28.7 MB/s eta 0:00:00
Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl (24.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.5/24.5 MB 34.7 MB/s eta 0:00:00
Downloading ml_dtypes-0.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 24.5 MB/s eta 0:00:00
Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.0/18.0 MB 33.4 MB/s eta 0:00:00
Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Downloading protobuf-4.25.5-cp37-abi3-manylinux2014_x86_64.whl (294 kB)
Downloading tensorboard-2.17.1-py3-none-any.whl (5.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 36.8 MB/s eta 0:00:00
Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB)
Downloading termcolor-2.4.0-py3-none-any.whl (7.7 kB)
Downloading wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (87 kB)
Downloading Markdown-3.7-py3-none-any.whl (106 kB)
Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 34.0 MB/s eta 0:00:00
Downloading werkzeug-3.0.4-py3-none-any.whl (227 kB)
Downloading wheel-0.44.0-py3-none-any.whl (67 kB)
Downloading namex-0.0.8-py3-none-any.whl (5.8 kB)
Downloading optree-0.12.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)
Downloading rich-13.8.1-py3-none-any.whl (241 kB)
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Downloading pygments-2.18.0-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 49.4 MB/s eta 0:00:00
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Installing collected packages: namex, libclang, flatbuffers, wrapt, wheel, termcolor, tensorboard-data-server, setuptools, pygments, protobuf, optree, numpy, mdurl, MarkupSafe, markdown, grpcio, google-pasta, werkzeug, opt-einsum, ml-dtypes, markdown-it-py, h5py, astunparse, tensorboard, rich, keras, tensorflow
Attempting uninstall: numpy
Found existing installation: numpy 2.1.1
Uninstalling numpy-2.1.1:
Successfully uninstalled numpy-2.1.1
Successfully installed MarkupSafe-2.1.5 astunparse-1.6.3 flatbuffers-24.3.25 google-pasta-0.2.0 grpcio-1.66.1 h5py-3.11.0 keras-3.5.0 libclang-18.1.1 markdown-3.7 markdown-it-py-3.0.0 mdurl-0.1.2 ml-dtypes-0.4.1 namex-0.0.8 numpy-1.26.4 opt-einsum-3.3.0 optree-0.12.1 protobuf-4.25.5 pygments-2.18.0 rich-13.8.1 setuptools-75.1.0 tensorboard-2.17.1 tensorboard-data-server-0.7.2 tensorflow-2.17.0 termcolor-2.4.0 werkzeug-3.0.4 wheel-0.44.0 wrapt-1.16.0
(.venv) ~/Tiberius ❯❯❯ rg tensorflow
README.md
35:- tensorflow==2.10.*
36:- tensorflow_probability==0.18.0
45:pip install tensorflow_probability==0.18.0 transformers pyBigWig bio scikit-learn biopython bcbio-gff requests
47:Tensorflow should be installed with GPU support. If you are using conda, you can install it with these [instructions](docs/install_tensorflow.md).
51:pip install tensorflow-gpu==2.10.*
56:python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
test_data/Panthera_pardus/example_train_full.ipynb
20: "import tensorflow as tf\n",
21: "from tensorflow.keras.optimizers import Adam\n",
22: "import tensorflow.keras as keras\n",
test_data/val_model.py
9:import tensorflow as tf
11:import tensorflow.keras as keras
test_data/test_vit.py
26: import tensorflow as tf
test_data/Panthera_pardus/example_prediction.ipynb
15: "2024-07-20 08:13:08.768505: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA\n",
17: "2024-07-20 08:13:08.837031: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
18: "2024-07-20 08:13:09.135135: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/conda/envs/cuda-11.2/lib\n",
19: "2024-07-20 08:13:09.135305: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/conda/envs/cuda-11.2/lib\n",
20: "2024-07-20 08:13:09.135312: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n",
21: "2024-07-20 08:13:09.802324: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
22: "2024-07-20 08:13:09.807364: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
23: "2024-07-20 08:13:09.807509: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
24: "2024-07-20 08:13:09.807844: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA\n",
26: "2024-07-20 08:13:09.809049: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
27: "2024-07-20 08:13:09.809203: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
28: "2024-07-20 08:13:09.809332: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
29: "2024-07-20 08:13:10.199331: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
30: "2024-07-20 08:13:10.199526: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
31: "2024-07-20 08:13:10.199655: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
32: "2024-07-20 08:13:10.199779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 22118 MB memory: -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9\n",
42: "# !pip install tensorflow==2.10.1\n",
43: "# !pip install tensorflow_probability==0.18.0\n",
51: "import tensorflow as tf\n",
52: "from tensorflow.keras.optimizers import Adam\n",
53: "import tensorflow.keras as keras\n",
202: "2024-07-20 08:13:17.082403: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:428] Loaded cuDNN version 8100\n",
203: "2024-07-20 08:13:17.554684: W tensorflow/compiler/xla/stream_executor/gpu/asm_compiler.cc:234] Falling back to the CUDA driver for PTX compilation; ptxas does not support CC 8.9\n",
204: "2024-07-20 08:13:17.554710: W tensorflow/compiler/xla/stream_executor/gpu/asm_compiler.cc:237] Used ptxas at ptxas\n",
205: "2024-07-20 08:13:17.554798: W tensorflow/compiler/xla/stream_executor/gpu/redzone_allocator.cc:318] UNIMPLEMENTED: ptxas ptxas too old. Falling back to the driver to compile.\n",
209: "2024-07-20 08:13:18.154053: I tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:630] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.\n"
test_data/Panthera_pardus/example_train_lstm.ipynb
13: "2024-07-20 08:29:05.755801: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA\n",
15: "2024-07-20 08:29:05.802750: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
16: "2024-07-20 08:29:06.776396: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:980] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
17: "2024-07-20 08:29:06.777858: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: /home/jovyan/tfenv/tf_env/lib/python3.10/site-packages/tensorflow/python/../../../../libcublas.so.11: undefined symbol: cublasLt_for_cublas_HSS, version libcublasLt.so.11; LD_LIBRARY_PATH: :/opt/conda/envs/cuda-11.2/lib\n",
18: "2024-07-20 08:29:06.796111: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n",
20: "2024-07-20 08:29:06.796421: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA\n",
38: "import tensorflow as tf\n",
39: "from tensorflow.keras.optimizers import Adam\n",
40: "import tensorflow.keras as keras\n",
docs/install_tensorflow.md
3:Adapted from https://gretel.ai/blog/install-tensorflow-with-cuda-cdnn-and-gpu-support-in-4-easy-steps
22: python3 -m pip install tensorflow==2.10 numpy==1.24
28: python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
docs/README_old.md
16:- tensorflow==2.10.1
17:- tensorflow_probability==0.18.0
25:pip install tensorflow==2.10.1 tensorflow_probability==0.18.0 transformers pyBigWig bio scikit-learn biopython bcbio-gff
32:mamba install -c conda-forge cudatoolkit ipykernel cudnn tensorflow==2.10.*
bin/write_tfrecord_species.py
8:import tensorflow as tf
bin/models.py
1:import tensorflow as tf
2:from tensorflow.keras.models import Model
3:from tensorflow.keras.layers import (Conv1D, SimpleRNN, Conv1DTranspose, MaxPool1D, LSTM,
9:from tensorflow.keras import backend as K
14:from tensorflow.keras.callbacks import Callback
bin/data_generator.py
11:import tensorflow as tf
bin/tiberius.py
11:import tensorflow as tf
bin/gene_pred_hmm_transitioner.py
1:import tensorflow as tf
bin/nuc_trans.py
2:import tensorflow as tf
8:from tensorflow.keras.models import Model
9:from tensorflow.keras.layers import (Conv1D, SimpleRNN, Conv1DTranspose, LSTM, GRU, Dense, Bidirectional, Dropout, Activation, Input, BatchNormalization, LSTM, Reshape, Embedding, Add, LayerNormalization,
12:from tensorflow.keras.callbacks import CSVLogger, ModelCheckpoint
13:from tensorflow.keras.optimizers import Adam
bin/validation_from_tfrecords.py
11:import tensorflow as tf
bin/create_tf_records.py
3:# import tensorflow records
4:import tensorflow as tf
bin/kmer.py
1:import tensorflow as tf
bin/eval_model_class.py
14:import tensorflow as tf
15:import tensorflow.keras as keras
17:from tensorflow.keras.models import Model
23:from tensorflow.keras.layers import (Conv1D, SimpleRNN, Conv1DTranspose, LSTM, GRU, Dense, Bidirectional, Dropout, Activation, Input, BatchNormalization, LSTM, Reshape, Embedding, Add, LayerNormalization,
bin/gene_pred_hmm_emitter.py
1:import tensorflow as tf
bin/train.py
9:# tensorflow_probability 0.18.0
20:import tensorflow as tf
22:from tensorflow.keras.callbacks import ModelCheckpoint
24:from tensorflow.keras.optimizers import Adam, SGD
25:import tensorflow.keras as keras
26:from tensorflow.keras.callbacks import CSVLogger
32:from tensorflow.keras.callbacks import LearningRateScheduler
bin/unit_tests.py
9:import tensorflow as tf
bin/gene_pred_hmm.py
1:import tensorflow as tf
271: # tell tensorflow that some config items have been renamed
(.venv) ~/Tiberius ❯❯❯ rg tensorflow-gpu
README.md
51:pip install tensorflow-gpu==2.10.*
(.venv) ~/Tiberius ❯❯❯ python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
2024-09-25 01:11:20.608452: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:11:20.612718: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:11:20.623111: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:11:20.638269: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:11:20.642118: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:11:20.651425: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:11:21.825571: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[]
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --fasta ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:11:30.168331: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:11:30.172597: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:11:30.185418: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:11:30.203375: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:11:30.207889: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:11:30.218174: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:11:30.991539: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
usage: tiberius.py [-h] [--model_lstm MODEL_LSTM] [--model_hmm MODEL_HMM] [--model MODEL]
[--out OUT] [--genome GENOME] [--parallel_factor PARALLEL_FACTOR]
[--learnMSA LEARNMSA] [--no_softmasking] [--clamsa CLAMSA]
[--codingseq CODINGSEQ] [--protseq PROTSEQ] [--emb] [--strand STRAND]
[--seq_len SEQ_LEN] [--batch_size BATCH_SIZE]
tiberius.py: error: unrecognized arguments: --fasta /home/cdiesh/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:12:00.906833: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:12:00.909874: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:12:00.920066: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:12:00.938429: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:12:00.944234: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:12:00.956575: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:12:01.799014: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
ModuleNotFoundError: No module named 'learnMSA'
(.venv) ~/Tiberius ❯❯❯ rg TensorRT ✘ 1
test_data/Panthera_pardus/example_prediction.ipynb
20: "2024-07-20 08:13:09.135312: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n",
(.venv) ~/Tiberius ❯❯❯ pip install "tensorflow-gpu==2.10.*"
ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.10.* (from versions: 2.12.0)
ERROR: No matching distribution found for tensorflow-gpu==2.10.*
(.venv) ~/Tiberius ❯❯❯ pip install "tensorflow-gpu==2.10.\*" ✘ 1
ERROR: Invalid requirement: 'tensorflow-gpu==2.10.\\*': Expected end or semicolon (after version specifier)
tensorflow-gpu==2.10.\*
~~~~~~^
(.venv) ~/Tiberius ❯❯❯ pip install "tensorflow-gpu==2.10.*" ✘ 1
ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.10.* (from versions: 2.12.0)
ERROR: No matching distribution found for tensorflow-gpu==2.10.*
(.venv) ~/Tiberius ❯❯❯ pip install "tensorflow-gpu==2.8.*" ✘ 1
ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.8.* (from versions: 2.12.0)
ERROR: No matching distribution found for tensorflow-gpu==2.8.*
(.venv) ~/Tiberius ❯❯❯ pip install "tensorflow-gpu==2.8.*" ✘ 1
ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==2.8.* (from versions: 2.12.0)
ERROR: No matching distribution found for tensorflow-gpu==2.8.*
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:15:22.531396: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:15:22.534977: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:15:22.549269: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:15:22.568688: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:15:22.573074: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:15:22.584867: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:15:23.367821: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
ModuleNotFoundError: No module named 'learnMSA'
(.venv) ~/Tiberius ❯❯❯ ll ✘ 1
total 28
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 01:12 bin
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 00:47 docs
drwxrwxr-x 2 cdiesh cdiesh 4096 Sep 25 00:47 figures
drwxrwxr-x 4 cdiesh cdiesh 4096 Sep 25 00:47 test_data
-rw-rw-r-- 1 cdiesh cdiesh 1069 Sep 25 00:47 LICENSE
-rw-rw-r-- 1 cdiesh cdiesh 8110 Sep 25 00:47 README.md
(.venv) ~/Tiberius ❯❯❯ git clone https://github.com/Gaius-Augustus/learnMSA
cd learnMSA
git checkout parallel
Cloning into 'learnMSA'...
remote: Enumerating objects: 5119, done.
remote: Counting objects: 100% (646/646), done.
remote: Compressing objects: 100% (292/292), done.
remote: Total 5119 (delta 434), reused 535 (delta 349), pack-reused 4473 (from 1)
Receiving objects: 100% (5119/5119), 109.18 MiB | 35.70 MiB/s, done.
Resolving deltas: 100% (3396/3396), done.
branch 'parallel' set up to track 'origin/parallel'.
Switched to a new branch 'parallel'
(.venv) ~/T/learnMSA ❯❯❯ cd ..
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:17:28.169946: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:17:28.173841: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:17:28.186937: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:17:28.205388: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:17:28.210722: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:17:28.224437: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:17:29.187499: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
ModuleNotFoundError: No module named 'learnMSA'
(.venv) ~/Tiberius ❯❯❯ cd learnMSA ✘ 1
(.venv) ~/T/learnMSA ❯❯❯ ll
total 844
drwxrwxr-x 5 cdiesh cdiesh 4096 Sep 25 01:17 learnMSA
drwxrwxr-x 2 cdiesh cdiesh 4096 Sep 25 01:17 logo
drwxrwxr-x 4 cdiesh cdiesh 4096 Sep 25 01:17 test
-rw-rw-r-- 1 cdiesh cdiesh 770092 Sep 25 01:17 learnMSA_demo.ipynb
-rw-rw-r-- 1 cdiesh cdiesh 77 Sep 25 01:17 learnMSA.py
-rw-rw-r-- 1 cdiesh cdiesh 56682 Sep 25 01:17 learnMSA_with_language_model_demo.ipynb
-rw-rw-r-- 1 cdiesh cdiesh 1069 Sep 25 01:17 LICENSE.md
-rw-rw-r-- 1 cdiesh cdiesh 80 Sep 25 01:17 pyproject.toml
-rw-rw-r-- 1 cdiesh cdiesh 3788 Sep 25 01:17 README.md
-rw-rw-r-- 1 cdiesh cdiesh 1465 Sep 25 01:17 setup.py
(.venv) ~/T/learnMSA ❯❯❯ pip install .
Processing /home/cdiesh/Tiberius/learnMSA
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
INFO: pip is looking at multiple versions of learnmsa to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement tensorflow<2.11,>=2.5.0 (from learnmsa) (from versions: 2.16.0rc0, 2.16.1, 2.16.2, 2.17.0rc0, 2.17.0rc1, 2.17.0)
ERROR: No matching distribution found for tensorflow<2.11,>=2.5.0
(.venv) ~/T/learnMSA ❯❯❯ vim pyproject.toml ✘ 1
(.venv) ~/T/learnMSA ❯❯❯ pip install .
Processing /home/cdiesh/Tiberius/learnMSA
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: tensorflow in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from learnMSA==2.0.1) (2.17.0)
Requirement already satisfied: tensorflow-probability<=0.18.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from learnMSA==2.0.1) (0.18.0)
Collecting networkx (from learnMSA==2.0.1)
Downloading networkx-3.3-py3-none-any.whl.metadata (5.1 kB)
Collecting logomaker (from learnMSA==2.0.1)
Downloading logomaker-0.8-py2.py3-none-any.whl.metadata (1.0 kB)
Collecting seaborn (from learnMSA==2.0.1)
Downloading seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Requirement already satisfied: biopython>=1.69 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from learnMSA==2.0.1) (1.84)
Collecting pyfamsa (from learnMSA==2.0.1)
Downloading pyfamsa-0.5.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Requirement already satisfied: transformers in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from learnMSA==2.0.1) (4.44.2)
Collecting imageio (from learnMSA==2.0.1)
Downloading imageio-2.35.1-py3-none-any.whl.metadata (4.9 kB)
Collecting SentencePiece (from learnMSA==2.0.1)
Downloading sentencepiece-0.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.7 kB)
Requirement already satisfied: numpy in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from biopython>=1.69->learnMSA==2.0.1) (1.26.4)
Requirement already satisfied: absl-py in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (2.1.0)
Requirement already satisfied: six>=1.10.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (1.16.0)
Requirement already satisfied: decorator in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (5.1.1)
Requirement already satisfied: cloudpickle>=1.3 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (3.0.0)
Requirement already satisfied: gast>=0.3.2 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (0.6.0)
Requirement already satisfied: dm-tree in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow-probability<=0.18.0->learnMSA==2.0.1) (0.1.8)
Collecting pillow>=8.3.2 (from imageio->learnMSA==2.0.1)
Downloading pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.2 kB)
Collecting matplotlib (from logomaker->learnMSA==2.0.1)
Downloading matplotlib-3.9.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Requirement already satisfied: pandas in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from logomaker->learnMSA==2.0.1) (2.2.3)
Collecting scoring-matrices~=0.2.0 (from pyfamsa->learnMSA==2.0.1)
Downloading scoring_matrices-0.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.8 kB)
Requirement already satisfied: astunparse>=1.6.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (1.6.3)
Requirement already satisfied: flatbuffers>=24.3.25 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (24.3.25)
Requirement already satisfied: google-pasta>=0.1.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (0.2.0)
Requirement already satisfied: h5py>=3.10.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (3.11.0)
Requirement already satisfied: libclang>=13.0.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (18.1.1)
Requirement already satisfied: ml-dtypes<0.5.0,>=0.3.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (0.4.1)
Requirement already satisfied: opt-einsum>=2.3.2 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (3.3.0)
Requirement already satisfied: packaging in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (24.1)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (4.25.5)
Requirement already satisfied: requests<3,>=2.21.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (2.32.3)
Requirement already satisfied: setuptools in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (75.1.0)
Requirement already satisfied: termcolor>=1.1.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (4.12.2)
Requirement already satisfied: wrapt>=1.11.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (1.16.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (1.66.1)
Requirement already satisfied: tensorboard<2.18,>=2.17 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (2.17.1)
Requirement already satisfied: keras>=3.2.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorflow->learnMSA==2.0.1) (3.5.0)
Requirement already satisfied: filelock in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (3.16.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (0.25.1)
Requirement already satisfied: pyyaml>=5.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (2024.9.11)
Requirement already satisfied: safetensors>=0.4.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (0.4.5)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (0.19.1)
Requirement already satisfied: tqdm>=4.27 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from transformers->learnMSA==2.0.1) (4.66.5)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from astunparse>=1.6.0->tensorflow->learnMSA==2.0.1) (0.44.0)
Requirement already satisfied: fsspec>=2023.5.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers->learnMSA==2.0.1) (2024.9.0)
Requirement already satisfied: rich in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow->learnMSA==2.0.1) (13.8.1)
Requirement already satisfied: namex in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow->learnMSA==2.0.1) (0.0.8)
Requirement already satisfied: optree in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow->learnMSA==2.0.1) (0.12.1)
Collecting contourpy>=1.0.1 (from matplotlib->logomaker->learnMSA==2.0.1)
Downloading contourpy-1.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib->logomaker->learnMSA==2.0.1)
Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib->logomaker->learnMSA==2.0.1)
Downloading fonttools-4.54.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (163 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib->logomaker->learnMSA==2.0.1)
Downloading kiwisolver-1.4.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.3 kB)
Collecting pyparsing>=2.3.1 (from matplotlib->logomaker->learnMSA==2.0.1)
Downloading pyparsing-3.1.4-py3-none-any.whl.metadata (5.1 kB)
Requirement already satisfied: python-dateutil>=2.7 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from matplotlib->logomaker->learnMSA==2.0.1) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from pandas->logomaker->learnMSA==2.0.1) (2024.2)
Requirement already satisfied: tzdata>=2022.7 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from pandas->logomaker->learnMSA==2.0.1) (2024.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow->learnMSA==2.0.1) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow->learnMSA==2.0.1) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow->learnMSA==2.0.1) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow->learnMSA==2.0.1) (2024.8.30)
Requirement already satisfied: markdown>=2.6.8 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow->learnMSA==2.0.1) (3.7)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow->learnMSA==2.0.1) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow->learnMSA==2.0.1) (3.0.4)
Requirement already satisfied: MarkupSafe>=2.1.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from werkzeug>=1.0.1->tensorboard<2.18,>=2.17->tensorflow->learnMSA==2.0.1) (2.1.5)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from rich->keras>=3.2.0->tensorflow->learnMSA==2.0.1) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from rich->keras>=3.2.0->tensorflow->learnMSA==2.0.1) (2.18.0)
Requirement already satisfied: mdurl~=0.1 in /home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.2.0->tensorflow->learnMSA==2.0.1) (0.1.2)
Downloading imageio-2.35.1-py3-none-any.whl (315 kB)
Downloading logomaker-0.8-py2.py3-none-any.whl (11.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 20.4 MB/s eta 0:00:00
Downloading networkx-3.3-py3-none-any.whl (1.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 27.3 MB/s eta 0:00:00
Downloading pyfamsa-0.5.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 25.9 MB/s eta 0:00:00
Downloading seaborn-0.13.2-py3-none-any.whl (294 kB)
Downloading sentencepiece-0.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 20.4 MB/s eta 0:00:00
Downloading matplotlib-3.9.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.3/8.3 MB 35.0 MB/s eta 0:00:00
Downloading pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl (4.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 33.4 MB/s eta 0:00:00
Downloading scoring_matrices-0.2.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (461 kB)
Downloading contourpy-1.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (320 kB)
Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)
Downloading fonttools-4.54.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 36.0 MB/s eta 0:00:00
Downloading kiwisolver-1.4.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 30.2 MB/s eta 0:00:00
Downloading pyparsing-3.1.4-py3-none-any.whl (104 kB)
Building wheels for collected packages: learnMSA
Building wheel for learnMSA (pyproject.toml) ... done
Created wheel for learnMSA: filename=learnMSA-2.0.1-py3-none-any.whl size=2998767 sha256=c87524ca8195def382742f5c869b72b939cfbd1dcfd7011c5af43f0fb31c3e8a
Stored in directory: /tmp/pip-ephem-wheel-cache-8s9bal5j/wheels/72/16/fa/9a765835e3aa2524dffdcf70ac5c60795e1580ed8e304717ee
Successfully built learnMSA
Installing collected packages: SentencePiece, scoring-matrices, pyparsing, pillow, networkx, kiwisolver, fonttools, cycler, contourpy, pyfamsa, matplotlib, imageio, seaborn, logomaker, learnMSA
Successfully installed SentencePiece-0.2.0 contourpy-1.3.0 cycler-0.12.1 fonttools-4.54.1 imageio-2.35.1 kiwisolver-1.4.7 learnMSA-2.0.1 logomaker-0.8 matplotlib-3.9.2 networkx-3.3 pillow-10.4.0 pyfamsa-0.5.2 pyparsing-3.1.4 scoring-matrices-0.2.2 seaborn-0.13.2
(.venv) ~/T/learnMSA ❯❯❯ cd ..
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:19:49.233738: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:19:49.237073: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:19:49.247915: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:19:49.264887: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:19:49.269683: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:19:49.280699: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:19:50.101403: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Viterbi.py", line 3, in <module>
import learnMSA.msa_hmm.Training as train
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Training.py", line 5, in <module>
from learnMSA.msa_hmm.MsaHmmCell import MsaHmmCell
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/MsaHmmCell.py", line 4, in <module>
import learnMSA.msa_hmm.Emitter as emit
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Emitter.py", line 2, in <module>
import tensorflow_probability as tfp
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/__init__.py", line 20, in <module>
from tensorflow_probability import substrates
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/substrates/__init__.py", line 17, in <module>
from tensorflow_probability.python.internal import all_util
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/__init__.py", line 138, in <module>
dir(globals()[pkg_name]) # Forces loading the package from its lazy loader.
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 57, in __dir__
module = self._load()
^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 40, in _load
module = importlib.import_module(self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/experimental/__init__.py", line 31, in <module>
from tensorflow_probability.python.experimental import bijectors
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/experimental/bijectors/__init__.py", line 17, in <module>
from tensorflow_probability.python.bijectors.ldj_ratio import forward_log_det_jacobian_ratio
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/bijectors/__init__.py", line 19, in <module>
from tensorflow_probability.python.bijectors.absolute_value import AbsoluteValue
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/bijectors/absolute_value.py", line 19, in <module>
from tensorflow_probability.python.bijectors import bijector
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/bijectors/bijector.py", line 26, in <module>
from tensorflow_probability.python.internal import batch_shape_lib
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/batch_shape_lib.py", line 23, in <module>
from tensorflow_probability.python.internal import prefer_static as ps
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/prefer_static.py", line 368, in <module>
ones_like = _copy_docstring(tf.ones_like, _ones_like)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/prefer_static.py", line 85, in _copy_docstring
raise ValueError(
ValueError: Arg specs do not match: original=FullArgSpec(args=['input', 'dtype', 'name', 'layout'], varargs=None, varkw=None, defaults=(None, None, None), kwonlyargs=[], kwonlydefaults=None, annotations={}), new=FullArgSpec(args=['input', 'dtype', 'name'], varargs=None, varkw=None, defaults=(None, None), kwonlyargs=[], kwonlydefaults=None, annotations={}), fn=<function ones_like_v2 at 0x7af55bfc9bc0>
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow_probability ✘ 1
Requirement already satisfied: tensorflow_probability in ./.venv/lib/python3.12/site-packages (0.18.0)
Requirement already satisfied: absl-py in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (2.1.0)
Requirement already satisfied: six>=1.10.0 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.16.0)
Requirement already satisfied: numpy>=1.13.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.26.4)
Requirement already satisfied: decorator in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (5.1.1)
Requirement already satisfied: cloudpickle>=1.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (3.0.0)
Requirement already satisfied: gast>=0.3.2 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.6.0)
Requirement already satisfied: dm-tree in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.1.8)
(.venv) ~/Tiberius ❯❯❯ ll
total 32
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 01:12 bin
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 00:47 docs
drwxrwxr-x 2 cdiesh cdiesh 4096 Sep 25 00:47 figures
drwxrwxr-x 8 cdiesh cdiesh 4096 Sep 25 01:19 learnMSA
drwxrwxr-x 4 cdiesh cdiesh 4096 Sep 25 00:47 test_data
-rw-rw-r-- 1 cdiesh cdiesh 1069 Sep 25 00:47 LICENSE
-rw-rw-r-- 1 cdiesh cdiesh 8110 Sep 25 00:47 README.md
(.venv) ~/Tiberius ❯❯❯ ll .venv
total 20
drwxrwxr-x 2 cdiesh cdiesh 4096 Sep 25 01:19 bin
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 00:49 include
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 00:49 lib
lrwxrwxrwx 1 cdiesh cdiesh 3 Sep 25 00:49 lib64 -> lib
drwxrwxr-x 3 cdiesh cdiesh 4096 Sep 25 01:19 share
-rw-rw-r-- 1 cdiesh cdiesh 293 Sep 25 00:49 pyvenv.cfg
(.venv) ~/Tiberius ❯❯❯ pip update tensorflow_probability
ERROR: unknown command "update"
(.venv) ~/Tiberius ❯❯❯ pip upgrade tensorflow_probability ✘ 1
ERROR: unknown command "upgrade"
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow_probability ✘ 1
Requirement already satisfied: tensorflow_probability in ./.venv/lib/python3.12/site-packages (0.18.0)
Requirement already satisfied: absl-py in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (2.1.0)
Requirement already satisfied: six>=1.10.0 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.16.0)
Requirement already satisfied: numpy>=1.13.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.26.4)
Requirement already satisfied: decorator in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (5.1.1)
Requirement already satisfied: cloudpickle>=1.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (3.0.0)
Requirement already satisfied: gast>=0.3.2 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.6.0)
Requirement already satisfied: dm-tree in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.1.8)
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow_probability -U
Requirement already satisfied: tensorflow_probability in ./.venv/lib/python3.12/site-packages (0.18.0)
Collecting tensorflow_probability
Downloading tensorflow_probability-0.24.0-py2.py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: absl-py in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (2.1.0)
Requirement already satisfied: six>=1.10.0 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.16.0)
Requirement already satisfied: numpy>=1.13.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.26.4)
Requirement already satisfied: decorator in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (5.1.1)
Requirement already satisfied: cloudpickle>=1.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (3.0.0)
Requirement already satisfied: gast>=0.3.2 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.6.0)
Requirement already satisfied: dm-tree in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.1.8)
Downloading tensorflow_probability-0.24.0-py2.py3-none-any.whl (6.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 34.3 MB/s eta 0:00:00
Installing collected packages: tensorflow_probability
Attempting uninstall: tensorflow_probability
Found existing installation: tensorflow-probability 0.18.0
Uninstalling tensorflow-probability-0.18.0:
Successfully uninstalled tensorflow-probability-0.18.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
learnmsa 2.0.1 requires tensorflow-probability<=0.18.0, but you have tensorflow-probability 0.24.0 which is incompatible.
Successfully installed tensorflow_probability-0.24.0
(.venv) ~/Tiberius ❯❯❯ cd learnMSA
(.venv) ~/T/learnMSA ❯❯❯ vim setup.py
(.venv) ~/T/learnMSA ❯❯❯ cd ..
(.venv) ~/Tiberius ❯❯❯ pip install tensorflow_probability -U
Requirement already satisfied: tensorflow_probability in ./.venv/lib/python3.12/site-packages (0.24.0)
Requirement already satisfied: absl-py in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (2.1.0)
Requirement already satisfied: six>=1.10.0 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.16.0)
Requirement already satisfied: numpy>=1.13.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (1.26.4)
Requirement already satisfied: decorator in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (5.1.1)
Requirement already satisfied: cloudpickle>=1.3 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (3.0.0)
Requirement already satisfied: gast>=0.3.2 in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.6.0)
Requirement already satisfied: dm-tree in ./.venv/lib/python3.12/site-packages (from tensorflow_probability) (0.1.8)
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:21:30.816245: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:21:30.819670: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:21:30.830581: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:21:30.848059: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:21:30.853331: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:21:30.864197: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:21:31.640178: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Failed to import TF-Keras. Please note that TF-Keras is not installed by default when you install TensorFlow Probability. This is so that JAX-only users do not have to install TensorFlow or TF-Keras. To use TensorFlow Probability with TensorFlow, please install the tf-keras or tf-keras-nightly package.
This can be be done through installing the tensorflow-probability[tf] extra.
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Viterbi.py", line 3, in <module>
import learnMSA.msa_hmm.Training as train
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Training.py", line 5, in <module>
from learnMSA.msa_hmm.MsaHmmCell import MsaHmmCell
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/MsaHmmCell.py", line 4, in <module>
import learnMSA.msa_hmm.Emitter as emit
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Emitter.py", line 2, in <module>
import tensorflow_probability as tfp
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/__init__.py", line 22, in <module>
from tensorflow_probability.python import * # pylint: disable=wildcard-import
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/__init__.py", line 152, in <module>
dir(globals()[pkg_name]) # Forces loading the package from its lazy loader.
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 57, in __dir__
module = self._load()
^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 37, in _load
self._on_first_access()
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/tensorflow_probability/python/__init__.py", line 79, in _validate_tf_environment
import tf_keras # pylint: disable=unused-import
^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'tf_keras'
(.venv) ~/Tiberius ❯❯❯ pip install tf_keras ✘ 1
Collecting tf_keras
Downloading tf_keras-2.17.0-py3-none-any.whl.metadata (1.6 kB)
Requirement already satisfied: tensorflow<2.18,>=2.17 in ./.venv/lib/python3.12/site-packages (from tf_keras) (2.17.0)
Requirement already satisfied: absl-py>=1.0.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (2.1.0)
Requirement already satisfied: astunparse>=1.6.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (1.6.3)
Requirement already satisfied: flatbuffers>=24.3.25 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (24.3.25)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (0.6.0)
Requirement already satisfied: google-pasta>=0.1.1 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (0.2.0)
Requirement already satisfied: h5py>=3.10.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (3.11.0)
Requirement already satisfied: libclang>=13.0.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (18.1.1)
Requirement already satisfied: ml-dtypes<0.5.0,>=0.3.1 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (0.4.1)
Requirement already satisfied: opt-einsum>=2.3.2 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (3.3.0)
Requirement already satisfied: packaging in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (24.1)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (4.25.5)
Requirement already satisfied: requests<3,>=2.21.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (2.32.3)
Requirement already satisfied: setuptools in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (75.1.0)
Requirement already satisfied: six>=1.12.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (4.12.2)
Requirement already satisfied: wrapt>=1.11.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (1.16.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (1.66.1)
Requirement already satisfied: tensorboard<2.18,>=2.17 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (2.17.1)
Requirement already satisfied: keras>=3.2.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (3.5.0)
Requirement already satisfied: numpy<2.0.0,>=1.26.0 in ./.venv/lib/python3.12/site-packages (from tensorflow<2.18,>=2.17->tf_keras) (1.26.4)
Requirement already satisfied: wheel<1.0,>=0.23.0 in ./.venv/lib/python3.12/site-packages (from astunparse>=1.6.0->tensorflow<2.18,>=2.17->tf_keras) (0.44.0)
Requirement already satisfied: rich in ./.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (13.8.1)
Requirement already satisfied: namex in ./.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (0.0.8)
Requirement already satisfied: optree in ./.venv/lib/python3.12/site-packages (from keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (0.12.1)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow<2.18,>=2.17->tf_keras) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow<2.18,>=2.17->tf_keras) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow<2.18,>=2.17->tf_keras) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.12/site-packages (from requests<3,>=2.21.0->tensorflow<2.18,>=2.17->tf_keras) (2024.8.30)
Requirement already satisfied: markdown>=2.6.8 in ./.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow<2.18,>=2.17->tf_keras) (3.7)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in ./.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow<2.18,>=2.17->tf_keras) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in ./.venv/lib/python3.12/site-packages (from tensorboard<2.18,>=2.17->tensorflow<2.18,>=2.17->tf_keras) (3.0.4)
Requirement already satisfied: MarkupSafe>=2.1.1 in ./.venv/lib/python3.12/site-packages (from werkzeug>=1.0.1->tensorboard<2.18,>=2.17->tensorflow<2.18,>=2.17->tf_keras) (2.1.5)
Requirement already satisfied: markdown-it-py>=2.2.0 in ./.venv/lib/python3.12/site-packages (from rich->keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ./.venv/lib/python3.12/site-packages (from rich->keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (2.18.0)
Requirement already satisfied: mdurl~=0.1 in ./.venv/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.2.0->tensorflow<2.18,>=2.17->tf_keras) (0.1.2)
Downloading tf_keras-2.17.0-py3-none-any.whl (1.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 23.1 MB/s eta 0:00:00
Installing collected packages: tf_keras
Successfully installed tf_keras-2.17.0
(.venv) ~/Tiberius ❯❯❯ python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna --out output.gtf
2024-09-25 01:21:43.063368: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:21:43.066281: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-09-25 01:21:43.076585: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-25 01:21:43.094727: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-25 01:21:43.099308: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-25 01:21:43.109149: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-25 01:21:43.882388: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 315, in <module>
main()
File "/home/cdiesh/Tiberius/bin/tiberius.py", line 90, in main
from eval_model_class import PredictionGTF
File "/home/cdiesh/Tiberius/bin/eval_model_class.py", line 16, in <module>
from learnMSA.msa_hmm.Viterbi import viterbi
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Viterbi.py", line 3, in <module>
import learnMSA.msa_hmm.Training as train
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Training.py", line 5, in <module>
from learnMSA.msa_hmm.MsaHmmCell import MsaHmmCell
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/MsaHmmCell.py", line 4, in <module>
import learnMSA.msa_hmm.Emitter as emit
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Emitter.py", line 5, in <module>
import learnMSA.msa_hmm.Initializers as initializers
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Initializers.py", line 39, in <module>
model = dm.load_mixture_model(model_path, 1, 20, trainable=False, dtype=tf.float32)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/DirichletMixture.py", line 164, in load_mixture_model
model = make_model(dm)
^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/DirichletMixture.py", line 155, in make_model
prob_vectors = tf.keras.Input(shape=(dirichlet_mixture_layer.alphabet_size),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/layers/core/input_layer.py", line 154, in Input
layer = InputLayer(
^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/layers/core/input_layer.py", line 47, in __init__
shape = backend.standardize_shape(shape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/backend/common/variables.py", line 515, in standardize_shape
raise ValueError(f"Cannot convert '{shape}' to a shape.")
ValueError: Cannot convert '20' to a shape.
(.venv) ~/Tiberius ❯❯❯
@cmdcolin
Copy link
Author

cmdcolin commented Sep 25, 2024 •
edited
Loading

we are reading https://github.com/Gaius-Augustus/Tiberius for journal club

  • Tiberius wants specifically tensorflow-gpu, which is fully pulled from pypi (so installing any version of https://pypi.org/project/tensorflow-gpu/ does not work) and then installing the "newer version" tensorflow without -gpu works

  • but then it complains learnMSA is not installed, but there are no git submodules as claimed in the README, so i manually clone it, and "pip install ." in learnMSA directory (not instructed to do as such, but intuited it)

  • then the --fasta flag in Tiberius readme does not exist, needs to be --genome

@cmdcolin
Copy link
Author

short list of commands i ran

git clone --recursive https://github.com/Gaius-Augustus/Tiberius
cd Tiberius
python3 -m venv  .venv # create venv
source .venv/bin/activate
pip install tensorflow_probability==0.18.0 transformers pyBigWig bio scikit-learn biopython bcbio-gff requests # get base requirements
pip install tensorflow-gpu==2.10.* # run command from readme to get tensorflow, non conda approach, doesn't work
pip install tensorflow-gpu # not specifiyng version also doesn't work
pip install tensorflow # worked, this is supposedly the newer version of tensorflow
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"  # print's warnings and outputs an empty array
pip install "tensorflow-gpu==2.10.\*" # doesn't work
pip install "tensorflow-gpu==2.10.*" "# doesn't work
pip install "tensorflow-gpu==2.8.*" # doesn't work
git clone https://github.com/Gaius-Augustus/learnMSA
cd learnMSA
git checkout parallel # instructions to get the learnMSA module
vim setup.py # remove the tensorflow version pinning to allow the non-tensorflow-gpu dep
pip install .
python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna  --out output.gtf # gives error related to something with tensorflow_probability in the stack trace
pip install tensorflow_probability  -U # try upgrading tensorflow_probability to counteract...
pip install tf_keras # also install tf_keras because now it gives an error for that otherwise
python bin/tiberius.py --genome ~/Downloads/ncbi_dataset/data/GCF_027595985.1/GCF_027595985.1_mSorAra2.pri_genomic.fna  --out output.gtf # finally try running

after all that it gives an error

@cmdcolin
Copy link
Author

template for me submitting an issue to Tiberius team....

Hi there,
I'm pretty new to using machine learning packages but was attempting the pip installation of Tiberius. I tried to sort of work through some of the issues but here is a short summary

  1. the tensorflow-gpu dependency that seems impossible to install because they removed tensorflow-gpu from pypi and replaced it with just tensorflow. google suggests "As of December 2022, tensorflow-gpu has been removed and has been replaced with this new, empty package that generates an error upon installation.". the readme also refers to getting "tensorflow==2.10." but attempting to pip install this gets ERROR: Could not find a version that satisfies the requirement tensorflow==2.10. (from versions: 2.16.0rc0, 2.16.1, 2.16.2, 2.17.0rc0, 2.17.0rc1, 2.17.0) ERROR: No matching distribution found for tensorflow==2.10.*
  2. the --fasta argument doesn't exist for tiberius.py, so should be --genome in the README.md
  3. then after that, there are some miscellaneous errors that i tried to work through by upgrading various dependencies, but i ended up with keras v3 (which is automatically what you get after using the newer tensorflow>=2.16.0) and then this is unable to load the "older model weights" (the .pb file and such?)
git clone --recursive https://github.com/Gaius-Augustus/Tiberius
cd Tiberius
python3 -m venv  .venv # create venv
source .venv/bin/activate
pip install tensorflow_probability==0.18.0 transformers pyBigWig bio scikit-learn biopython bcbio-gff requests # get base requirements
pip install tensorflow-gpu==2.10.* # run command from readme to get tensorflow, non conda approach, doesn't work
pip install tensorflow-gpu # not specifiyng version also doesn't work
pip install tensorflow # worked, this is supposedly the newer version of tensorflow
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"  # print's warnings and outputs an empty array
git clone https://github.com/Gaius-Augustus/learnMSA # get the learnMSA submodule manually (it is not available as a true github sub module so i think instructions in readme related to this may be outdated)
cd learnMSA
git checkout parallel # instructions to get the learnMSA module
vim setup.py # remove the tensorflow version pinning to allow the newer non-tensorflow-gpu depenedency
pip install . # i think this is needed, but not mentioned in README(?)
python bin/tiberius.py --genome testgenome.fa  --out output.gtf # gives error related to something with tensorflow_probability in the stack trace
pip install tensorflow_probability  -U # try upgrading tensorflow_probability to counter-act above seen error
pip install tf_keras # also install tf_keras because now it gives an error for that otherwise
python bin/tiberius.py --genome testgenome.fa  --out output.gtf # finally try running, but it gives an error like this

this got farther, but then gave an error about shape...

 python bin/tiberius.py --genome testgenome.fa  --out output.gtf
...
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/Initializers.py", line 39, in <module>
    model = dm.load_mixture_model(model_path, 1, 20, trainable=False, dtype=tf.float32)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/DirichletMixture.py", line 164, in load_mixture_model
    model = make_model(dm)
            ^^^^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/DirichletMixture.py", line 155, in make_model
    prob_vectors = tf.keras.Input(shape=(dirichlet_mixture_layer.alphabet_size),
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/layers/core/input_layer.py", line 154, in Input
    layer = InputLayer(
            ^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/layers/core/input_layer.py", line 47, in __init__
    shape = backend.standardize_shape(shape)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/backend/common/variables.py", line 515, in standardize_shape
    raise ValueError(f"Cannot convert '{shape}' to a shape.")
ValueError: Cannot convert '20' to a shape.

to fix this shape bug, i edited the learnMSA code to

@ learnMSA/msa_hmm/DirichletMixture.py:155 @ class DirichletMixtureLayer(tf.keras.layers.Layer):
def make_model(dirichlet_mixture_layer):
    """Utility function that constructs a keras model over a DirichletMixtureLayer.
    """
-   prob_vectors = tf.keras.Input(shape=(dirichlet_mixture_layer.alphabet_size),
+   prob_vectors = tf.keras.Input(shape=(dirichlet_mixture_layer.alphabet_size,), <-- added comma

this fixed the above error but then it gave an error about not being to load the weights file

  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/DirichletMixture.py", line 165, in load_mixture_model
    model.load_weights(model_path).expect_partial()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/keras/src/saving/saving_api.py", line 273, in load_weights
    raise ValueError(
ValueError: File format not supported: filepath=/home/cdiesh/Tiberius/.venv/lib/python3.12/site-packages/learnMSA/msa_hmm/trained_prior/1_True_float32__dirichlet/ckpt.index. Keras 3 only supports V3 `.keras` and `.weights.h5` files, or legacy V1/V2 `.h5` files.

so i got close i think but i think ultimately we end up that "tensorflow >= 2.16.0" uses keras 3 which can't load the old(?) weights file.

so i guess at this point my curiosity is

  • if possible, upgrading Tiberius to use the latest versions of dependencies like tensorflow would be great because i have trouble installing the older versions. i did not try any containerized solutions or conda yet though
  • if possible, provide newer weights files compatible with keras v3 (?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment