matteoferla/fixes-found.md

Last active December 20, 2024 09:49

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/matteoferla/2001c5578c24651bb04ef910004c7d52.js"></script>
Save matteoferla/2001c5578c24651bb04ef910004c7d52 to your computer and use it in GitHub Desktop.

Download ZIP

I am going to write a comment for every damn IT issue/fix I get...

Raw

fixes-found.md

This is an experiment. Is a GitHub Gist comment a good way to log publically how I fixed an installation or similar issue?

See comments

Author

matteoferla commented Dec 10, 2023

I keep having issues with libmamba solver for conda. This time its actually not libmamba, but libmambapy.
libmambapy is the Pythonic reimplementation of the C++ library libmamba.

Error while loading conda entry point: conda-libmamba-solver (/data/xchem-fragalysis/mferla/waconda/envs/cuda118/lib/python3.11/site-packages/../../libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /data/xchem-fragalysis/mferla/waconda/lib/python3.11/site-packages/libmambapy/../../../libmamba.so.2))

Unsurprisingly, this was not directly installed by me, but by jax. Ie. a random config requirement that really ought to not be there —the award for that goes to tensorflow whose requirement are effectly a conda environment export.

To fix this I did:

ldd --version

2.34
and then

CONDA_OVERRIDE_GLIB=2.34 conda install --solver classic libmambapy --force-reinstall

Author

matteoferla commented Dec 20, 2023 •

edited

Loading

Installation of MaSIF via conda. 🚧 Not finished.
GitHub: https://github.com/lpdi-epfl/masif has no conda, because of its dozen requirements, some old.

# ## Start with creation
export CONDA_YES=true
# I believe this ancient prepandemic python is due to pymesh:
conda create -n masif python=3.6
conda activate masif
# I don't want stuff in $HOME/.local and PIP_NO_USERBASE:
export PYTHONUSERBASE=$CONDA_PREFIX
conda env config vars set PYTHONUSERBASE=$PYTHONUSERBASE

# ## APBS
#conda install -c conda-forge apbs #No: this does not install multivalue
wget https://github.com/Electrostatics/apbs/releases/download/v3.4.1/APBS-3.4.1.Linux.zip -O /tmp/APBS-3.4.1.Linux.zip
unzip /tmp/APBS-3.4.1.Linux.zip -d /tmp/APBS-3.4.1.Linux
rsync -ar /tmp/APBS-3.4.1.Linux/ $CONDA_PREFIX
export APBS_BIN=$CONDA_PREFIX/bin/apbs
conda env config vars set APBS_BIN=$APBS_BIN
export MULTIVALUE_BIN=$CONDA_PREFIX/share/apbs/tools/bin/multivalue
conda env config vars set MULTIVALUE_BIN=$MULTIVALUE_BIN

# ## MSMS
conda install -c bioconda msms
export MSMS_BIN=$CONDA_PREFIX/bin/msms
export PDB2XYZRN=$CONDA_PREFIX/bin/pdb_to_xyzrn
conda env config vars set MSMS_BIN=$MSMS_BIN
conda env config vars set PDB2XYZRN=$PDB2XYZRN

# ## PDB2PQR
conda install -c schrodinger pdb2pqr
export PDB2PQR_BIN=$CONDA_PREFIX/bin/pdb2pqr
conda env config vars set PDB2PQR_BIN=$PDB2PQR_BIN

# ## Open3D
conda install -c open3d-admin open3d

# ## Reduce
conda install -c ostrokach-forge reduce
conda env config vars set PATH=$PATH:$CONDA_PREFIX/bin/reduce
# forgive me for this transgression:
export PATH=$PATH
# reduce dict needs adding:
curl https://github.com/rlabduke/reduce/blob/master/reduce_wwPDB_het_dict.txt -o $CONDA_PREFIX/var/reduce_wwPDB_het_dict.txt
export REDUCE_HET_DICT=$CONDA_PREFIX/var/reduce_wwPDB_het_dict.txt

# ## PyMesh
conda install -c conda-forge pymesh2
export PYMESH_PATH=$CONDA_PREFIX/lib/python3.6/site-packages/pymesh
conda env config vars set PYMESH_PATH=$PYMESH_PATH

# MaSIF itself
git clone https://github.com/lpdi-epfl/masif $CONDA_PREFIX/masif_repo
export PATH=$CONDA_PREFIX/masif_repo/data/masif:$PATH

NB. masif_repo/requirements.txt is not a valid Python pip requirements.txt it's just rubbish.

The repo is not installed but simply dumped. That would need to be fixed especially the shell scripts

I have not looked into why conda install -c conda-forge pymesh2 seems fine while this https://github.com/PyMesh/PyMesh is lengthy

Author

matteoferla commented Dec 21, 2023 •

edited

Loading

Installing ColabFold with a 11.x driver not compatible with cuda-compat in December 2023 (in singularity without root conda-toolkit where /.singularity.d/libs do not match).

conda create -n colabfold python=3.10
conda activate colabfold
conda env config vars set PYTHONUSERBASE=$CONDA_PREFIX
conda install ipykernel
conda install git
$CONDA_PREFIX/bin/python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
conda install -c conda-forge python=3.10 cudnn==8.2.1.32 cudatoolkit==11.6.0 openmm==7.7.0 pdbfixer -y
$CONDA_PREFIX/bin/python -m pip install --upgrade pip
$CONDA_PREFIX/bin/python -m pip install --no-warn-conflicts "colabfold[alphafold-minus-jax] @ git+https://github.com/sokrypton/ColabFold" tensorflow==2.12.0
$CONDA_PREFIX/bin/python -m pip install https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.3.25+cuda11.cudnn82-cp310-cp310-manylinux2014_x86_64.whl
$CONDA_PREFIX/bin/python -m pip install jax==0.3.25 chex==0.1.6 biopython==1.79
$CONDA_PREFIX/bin/python -m pip install -y jax==0.3.25 chex==0.1.6
conda install -c "nvidia/label/cuda-11.6.0" cuda-toolkit
export LD_LIBRARY=$CONDA_PREFIX
rm -rf $CONDA_PREFIX/lib/python3.10/site-packages/nvidia

The $LD_LIBRARY is not assigned /.singularity.d/libs as those mismatch in DLS.
Normally the values /usr/local/cuda/compat, /usr/local/nvidia/lib, /usr/local/nvidia/lib64 and $CONDA_PREFIX/lib may be used. Some instructions add /proc/driver/nvidia, which is 100% comedically wrong. Specifying a path to specific packages might do something but effort (eg. $CONDA_PREFIX/lib/python3.11/site-packages/nvidia/cudnn/lib.

The removal of the package by rm is because CuDNN is being installed via multiple packages: use find $CONDA_PREFIX -name "libdevice.*" to find the conflict locations.

The problem child is tensorflow, to test it:

import os

#os.environ['LD_LIBRARY_PATH'] = os.environ['CONDA_PREFIX']
import tensorflow as tf
import sys, os

print(sys.version_info)
print(tf.config.list_physical_devices('GPU'))
print('CUDA build:', tf.test.is_built_with_cuda())
print("CUDA version:", tf.sysconfig.get_build_info()["cuda_version"])
print("cuDNN version:", tf.sysconfig.get_build_info()["cudnn_version"])
print("CUDA library paths:", tf.sysconfig.get_lib())
a = tf.constant([[1.0, 2.0], [3.0, 4.0]])
b = tf.constant([[1.0, 1.0], [0.0, 1.0]])
c = tf.matmul(a, b)
print(c)

Note, Pytorch is easier to please while tensorflow does seem to not respect $LD_LIBRARY_PATH. It is possible to have a env with a working pytorch but broken tensorflow. In which case loading torch then tf will work, so be aware of that.

Author

matteoferla commented Jan 18, 2024

Colab installation of Pymol

import sys
!wget -qnc https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
!bash Mambaforge-Linux-x86_64.sh -bfp /usr/local
sys.path.append('/usr/local/lib/python3.10/site-packages')
!clear
!/usr/local/condabin/mamba install -y conda-forge::pymol-open-source

Works with

import os
os.environ['LD_LIBRARY_PATH'] = '/usr/local/lib:/usr/local/lib64:/usr/local/bin/openssl:/usr/local/nvidia/lib:/usr/local/nvidia/lib64'

import pymol2

Author

matteoferla commented Jan 29, 2024

git clone https://github.com/👾👾👾/👾👾👾.git gave: 'file:///lib64/libldap.so.2: undefined symbol: EVP_md2, version OPENSSL_3.0.0'
Fixed with conda install anaconda::openldap

Author

matteoferla commented Feb 1, 2024

Pyrosetta cannot read mmCIF/PDBx files. The following seems to work: pose: pyrosetta.Pose = prc.import_pose.pose_from_file('crysalin_lattice.cif', read_fold_tree=True, type=prc.import_pose.FileType.CIF_file) but it does not.
However, atom indices are ignored so I will just convert into PDB with over >9_999 atoms. 🤷

Author

matteoferla commented Feb 9, 2024

Installation of the RDKit Postgres catridge

conda create -n py310 python==3.10 ipykernel -y
conda activate py310
conda env config vars set PYTHONUSERBASE=$CONDA_PREFIX
export PYTHONUSERBASE=$CONDA_PREFIX

# https://www.rdkit.org/docs/Cartridge.html
conda install -y -c conda-forge -c rdkit rdkit-postgresql postgresql
# lets be a traditionalist
conda env config vars set PGDATA=$CONDA_PREFIX/var/lib/pgsql/data
export PGDATA=$CONDA_PREFIX/var/lib/pgsql/data
# starting rdkit db
initdb
echo "shared_buffers = 2048MB
work_mem = 128MB
synchronous_commit = off
full_page_writes = off" >> "$PGDATA/postgresql.conf"
pg_ctl -l $HOME2/logs/postgres.log start
createdb test
# dropdb test
psql -c 'create extension rdkit'test
psql -c 'create table raw_data (id SERIAL, smiles text, emol_id integer, parent_id integer)' test
...

Author

matteoferla commented Feb 28, 2024 •

edited

Loading

Getting the Python API of the CCDC CSD working is not too straightforward —the older documentation does not help.
The installation of CCDC is obviously needed so: ./CSDInstallerOffline-2023.3-linux --accept-licenses install uk.ac.cam.ccdc --root $CCDC_HOME -c

cleaner environment variables:

export CCDC_HOME=$HOME/CCDC
conda activate base
# conda install --channel=https://conda.ccdc.cam.ac.uk csd-python-api # does not work for me
cp -r $CCDC_HOME/ccdc-software/csd-python-api.minoconda $CONDA_PREFIX/envs/CDS
conda activate CSD
# following is an alternative to `$CCDC_HOME/ccdc-utilities/software-activation/bin/ccdc_activator -a -k 👾👾👾`
conda env config vars set CCDC_LICENSING_CONFIGURATION='la-code;👾👾👾'
conda env config vars set CCDC_TOOLKIT_ASER_DATABASE=$CCDC_HOME/ccdc-data/csd/as544be
conda env config vars set CCDC_TOOLKIT_SQLITE_DATABASE=$CCDC_HOME/ccdc-data/csd/as544be_ASER.sqlite
conda env config vars set CCDC_ISOSTAR_DATA_DIRECTORY=$CCDC_HOME/ccdc-data/isostar
conda env config vars set CCDC_MOGUL_DATA=$CCDC_HOME/ccdc-data/mogul
conda env config vars set CCDC_MOGUL_INITIALISATION_FILE=$CONDA_PREFIX/lib/python3.9/site-packages/ccdc/parameter_files/mogul.ini
conda env config vars set CCDC_CROSSMINER_DATABASE=$CCDC_HOME/ccdc-data/crossminer/csd_pdb_crossminer.feat
conda env config vars set CCDC_CROSSMINER_FEATURE_DEFINITIONS=$CCDC_HOME/ccdc-data/crossminer/feature_definitions
conda env config vars set GOLD_DIR=$CCDC_HOME/ccdc-software/gold/GOLD
# always good:
conda env config vars set PYTHONUSERBASE=$CONDA_PREFIX
conda deactivate
conda activate CSD
# this applied to my system only (Mac OS), was fine in Fedora
# First error: `Symbol not found: _png_create_info_struct` (Mac OS 14)
conda install anaconda::libpng
# also for my notebook
conda install ipykernel

Author

matteoferla commented Mar 16, 2024

Module file for CCDC CSD.

#%Module1.0
##
## csd23-module
## Module file for setting up environment for CSD23
## Written by Matteo Ferla — matteo.ferla@stats.ox.ac.uk

proc ModulesHelp { } {

    puts stderr "\tAdds CSD23 CCDC Utilities and Software to your environment variables"
}

module-whatis "Adds CSD23 CCDC Utilities and Software to your environment variables"
set CCDC_HOME /vols/opig/apps/CSD23
prepend-path PATH /vols/opig/apps/CSD23/ccdc-utilities
prepend-path PATH /vols/opig/apps/CSD23/ccdc-software
setenv CCDC_LICENSING_CONFIGURATION "la-code;👾👾👾"
setenv CCDC_HOME /vols/opig/apps/CSD23
setenv CCDC_TOOLKIT_ASER_DATABASE $CCDC_HOME/ccdc-data/csd/as544be
setenv CCDC_TOOLKIT_SQLITE_DATABASE $CCDC_HOME/ccdc-data/csd/as544be_ASER.sqlite
setenv CCDC_ISOSTAR_DATA_DIRECTORY $CCDC_HOME/ccdc-data/isostar
setenv CCDC_MOGUL_DATA $CCDC_HOME/ccdc-data/mogul
setenv CCDC_MOGUL_INITIALISATION_FILE $CCDC_HOME/lib/python3.9/site-packages/ccdc/parameter_files/mogul.ini
setenv CCDC_CROSSMINER_DATABASE $CCDC_HOME/ccdc-data/crossminer/csd_pdb_crossminer.feat
setenv CCDC_CROSSMINER_FEATURE_DEFINITIONS $CCDC_HOME/ccdc-data/crossminer/feature_definitions
setenv GOLD_DIR $CCDC_HOME/ccdc-software/gold/GOLD

# Configure CONDA_ENVS_PATH
if { [info exists env(CONDA_ENVS_PATH)] } {
    setenv CONDA_ENVS_PATH $env(CONDA_ENVS_PATH):/vols/opig/apps/conda310/envs/CSD
} else {set
    setenv CONDA_ENVS_PATH /vols/opig/apps/conda310/envs/CSD
""

(Test code: https://github.com/matteoferla/Fragment-hit-follow-up-chemistry/blob/main/fragment_elaboration_scripts/mogul_zscore.py)

Author

matteoferla commented Jun 1, 2024

RDKit to OpenBabel
[10:11:18] Cannot convert ' 3.' to unsigned int on line 4
This is caused by no title line in the Mol_block.

smiles = 'CN(C)CCCN1c2ccccc2Sc3c1cccc3'
promazine = AllChem.AddHs(Chem.MolFromSmiles(smiles))
AllChem.EmbedMolecule(promazine)
# REQUIRED: promazine.SetProp('_Name', 'promazine')
conv = ob.OBConversion()
conv.SetInAndOutFormats('mol', 'mol')
obmol = ob.OBMol()
conv.ReadString(obmol, Chem.MolToMolBlock(promazine))
roundtrip = Chem.MolFromMolBlock(conv.WriteString(obmol).strip())

Author

matteoferla commented Aug 30, 2024

Jupyter decided to stop starting kernels after a pip install in conda base env. Cause: prompt-toolkit-3.0.42 giving error ModuleNotFoundError: No module named 'prompt_toolkit.output' —ipython run, but import raised error. Fix was simply to pip install prompt-toolkit -U to version 3.0.47.
Oddly, this applied to all conda envs.
One environment's kernel was still effe'ed, but fixed when disinstalled and reinstalled possibly due to lack of conda env config vars set PYTHONUSERBASE=$CONDA_PREFIX

Author

matteoferla commented Oct 2, 2024

APBS does not work out of the box on Rocky Linux 8.10

https://github.com/Electrostatics/apbs/releases/download/v3.4.1/APBS-3.4.1.Linux.zip
$ unzip APBS-3.4.1.Linux.zip
...REDACTED...
$ ./APBS-3.4.1.Linux/bin/apbs pentakaihemimer.relax.pdb 
./APBS-3.4.1.Linux/bin/apbs: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by ./APBS-3.4.1.Linux/bin/apbs)
./APBS-3.4.1.Linux/bin/apbs: /lib64/libpthread.so.0: version `GLIBC_2.30' not found (required by ./APBS-3.4.1.Linux/bin/apbs)
$ ldd --version
ldd (GNU libc) 2.28
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
$ source /etc/os-release 
$ echo $PRETTY_NAME 
Rocky Linux 8.10 (Green Obsidian)

Author

matteoferla commented Dec 20, 2024

MacCoot

CCP4 has coot, but why install the whole, when you need a part?
The following is more icky than the value of zeta(3), but CCP4 installation on Mac can be worse:
installing XQuartz traditionally has made my OS glitchier than a Windows machine running pirated software.

brew tap brewsci/bio
brew install coot
 
# find /opt/homebrew -iname '*coot*' --> /opt/homebrew/Cellar/coot/1.1.11_1.reinstall/bin/coot
# but when run it thinks itself in coot/1.1.11_1/
ln -s /opt/homebrew/Cellar/coot/1.1.11_1.reinstall /opt/homebrew/Cellar/coot/1.1.11_1
export COOT_PATH='/opt/homebrew/Cellar/coot/1.1.11_1'
 
# link binaries to folder in PATH
ln -s $COOT_PATH/bin/coot /opt/homebrew/bin/coot
ln -s $COOT_PATH/libexec/MacCoot /opt/homebrew/bin/MacCoot
 
# boost error? Boost installed                                                       
#brew install boost-python3
#brew upgrade boost-python3
# linking it and crossing fingers...
ln -s /opt/homebrew/Cellar/boost-python3/1.87.0_1/lib/libboost_python313.dylib \
      /opt/homebrew/opt/boost-python3/lib/libboost_python312.dylib

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment