Skip to content

Instantly share code, notes, and snippets.

@prerakmody

prerakmody/ASAP-CentOS.md

Last active July 16, 2023 21:27
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save prerakmody/d80ec865c0ffbb156a7c2bdcd2a9bb5c to your computer and use it in GitHub Desktop.
Save prerakmody/d80ec865c0ffbb156a7c2bdcd2a9bb5c to your computer and use it in GitHub Desktop.
Download ZIP
ASAP Installation (Whole Slide Images in Digital Pathology)
Raw
ASAP-CentOS.md

Install ASAP on CentOS Stream (within a conda environment as a non-root user)

Preliminary points

  1. The approach here is to first build/install dependencies of ASAP and then define their paths in the cmake command for ASAP. The order in which I build the dependencies is in the same order as defined using the find_package() function in ASAP/CMakeLists.txt.

  2. Assumptions

    • Note 1: I assume you have your conda environment installed and activated
    • Note 2: These instruction assume you have cmake installed. Find a basic tutorial here and a quick intuition here.
    • Note 3: These instruction also assume that you gcc --version is >=9 since otherwise it leads to issues with compiling #include<filesystem> in some .cpp files like core/filetools.cpp.

  3. My packages and their versions

    • Library Version
      conda 4.9.2
      python 3.8.13
      numpy 1.23.1
      cmake 3.20.2
      gcc 112.0

  4. My System

    • cat /etc/os-release 
      • NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"

  5. Terminals commands to check if libs exist via cmake

    • OpenJPEG : cmake --find-package -DNAME=OpenJPEG -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • TIFF : cmake --find-package -DNAME=TIFF -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • JPEG : cmake --find-package -DNAME=JPEG -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • ZLIB : cmake --find-package -DNAME=ZLIB -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • PugiXML : cmake --find-package -DNAME=PugiXML -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • OPENSLIDE : cmake --find-package -DNAME=OPENSLIDE -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • OpenCV : cmake --find-package -DNAME=OpenCV -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • Qt : cmake --find-package -DNAME=Qt5Core -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK
    • Qt : cmake --find-package -DNAME=Qt5Gui -DCOMPILER_ID=GNU -DLANGUAGE=C -DMODE=LINK

  6. Install missing libs (as non-root user)

    • Results from my academic cluster (I already had OpenJPEG, TIFF, JPEG and ZLIB)
    • Define export PATH_ASAP_EXTLIBS="" as the directory where you install the missing libraries
    • Then do mkdir -p $PATH_ASAP_EXTLIBS && cd $PATH_ASAP_EXTLIBS
    • Define export PATH_ASAP="" as the directory where you install ASAP
    • PugiXML 
      • cd $PATH_ASAP_EXTLIBS
git clone git@github.com:zeux/pugixml.git && cd pugixml
cmake -S . -B build # makes this file: pugixml/build/CMakeFiles
cmake --build build -j 4 # compiles .cpp files
cd build
make DESTDIR=../install install # uses pugixml/build/CMakeFiles to copy the build files into "proper" locations
cp $PATH_ASAP_EXTLIBS/pugixml/src/pugixml.cpp $PATH_ASAP_EXTLIBS/pugixml/install/usr/local/include/ # needed during ASAP building
    • OpenSlide (uses a different build system, not cmake) 
      • cd $PATH_ASAP_EXTLIBS
git clone git@github.com:openslide/openslide.git && cd openslide
autoreconf -i
./configure
make DESTDIR=$PATH_ASAP_EXTLIBS/openslide/install install
    • OpenCV (we dont install here, dunno why!) 
      • cd $PATH_ASAP_EXTLIBS
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip
unzip opencv.zip && rm opencv.zip && cd opencv-4.x/ # time consuming
cmake -S . -B build # time consuming
cmake --build build -j 4 # ultra time consuming
    • DCMTK (Dicom toolkit, apparently only needed for VSI support, specified in find_package() of the CMakeLists.txt belonging to multiresolutionimageinterface folder) 
      • cd $PATH_ASAP_EXTLIBS
wget https://github.com/DCMTK/dcmtk/archive/refs/tags/DCMTK-3.6.7.tar.gz
tar -xvzf DCMTK-3.6.7.tar.gz && rm DCMTK-3.6.7.tar.gz && cd dcmtk-DCMTK-3.6.7
cmake -S . -B build -DCMAKE_POSITION_INDEPENDENT_CODE=ON # time consuming
cmake --build build -j 4 # ultra time consuming
cd $PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/build && make DESTDIR=$PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/install install 
cp -r $PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/dcmjpeg/libijg8 $PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/install/usr/local/include/dcmtk/dcmjpeg/
      • Check if all required files have been put in the $PATH_ASAP_EXTLIBS/dcmtk/install dir
        • tree --prune -P 'jconfig8.h|jpeglib8.h|djdecode.h|djencode.h'

  7. Build ASAP

    cd $PATH_ASAP
wget https://github.com/computationalpathologygroup/ASAP/archive/refs/tags/ASAP-2.1.tar.gz
tar -xvzf ASAP-2.1.tar.gz && rm ASAP-2.1.tar.gz && cd ASAP-ASAP-2.1

# Need to define many macros here. 
cmake -S . -B build 
-DPugiXML_INCLUDE_DIR="$PATH_ASAP_EXTLIBS/pugixml/install/usr/local/include/"
-DOPENSLIDE_INCLUDE_DIR="$PATH_ASAP_EXTLIBS/openslide/install/usr/local/include/openslide"
-DOPENSLIDE_LIBRARY="$PATH_ASAP_EXTLIBS/openslide/install/usr/local/lib/libopenslide.so.0.4.1"
-DOpenCV_DIR="$PATH_ASAP_EXTLIBS/opencv-4.x/build"
-DDCMTK_DIR="$PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/build"
-DDCMTKJPEG_INCLUDE_DIR="$PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/install/usr/local/include"
-DDCMTKJPEG_LIBRARY="$PATH_ASAP_EXTLIBS/dcmtk-DCMTK-3.6.7/install/usr/local/lib64/libijg8.a"
-DBUILD_EXECUTABLES=ON -DBUILD_ASAP=ON -DBUILD_IMAGEPROCESSING=ON -DPACKAGE_ON_INSTALL=ON  
-DBUILD_MULTIRESOLUTIONIMAGEINTERFACE_VSI_SUPPORT=ON 
-DBUILD_MULTIRESOLUTIONIMAGEINTERFACE_DICOM_SUPPORT=ON
-DWRAP_MULTIRESOLUTIONIMAGEINTERFACE_PYTHON=ON
-DPYTHON_LIBRARY=$CONDA_PREFIX/lib/libpython3.8.so

cd $PATH_ASAP/ASAP-ASAP-2.1/ && cmake --build build -j 4

cd $PATH_ASAP/ASAP-ASAP-2.1/build && make DESTDIR=$PATH_ASAP/ASAP-ASAP-2.1/install install
    • Output of cmake -S . -B build ... can be checked here
    • Note: Here VSI_SUPPORT is probably some proprietary imaging format (link)
    • Note: The final command here should create the file multiresolutionimageinterface.py
      • Find it: find $PATH_ASAP/ASAP-ASAP-2.1/ -iname 'multiresolutionimageinterface.py'

  8. Import ASAP in python

    • import os
import sys
from pathlib import Path
path_add1 = Path.cwd().joinpath(os.environ['PATH_ASAP'], 'ASAP-ASAP-2.1', 'install', 'usr', 'local', 'bin') # you may have modify this option depending on where 'multiresolutionimageinterface.py' installs
sys.path.append(str(path_add1))
assert Path(sys.path[0]).exists()
import multiresolutionimageinterface as mir

Other Notes

  1. If you wish to print within a .cmake file 
    • function(PRINT_VAR VARNAME)
  message(STATUS "${VARNAME}: ${${VARNAME}}")
endfunction()
PRINT_VAR("CMAKE_CXX_COMPILER")
  2. References
Raw
ASAP.md

(Failed) attempt to use precompiled Ubuntu binaries of ASAP on a CentOS (RedHat) system

  • Note: This is for a non-root user (a situation common in academic environments)

  1. Check the flavor of unix (to be sure that it is CentOS)

    • cat /etc/os-release

  2. Download Alien

    • Note: Alien helps convert .deb (linux) to .rpm (centos)
    • The script below will download alien and then using rpm2cpio will convert the rpm files to cpio files. This is done instead of yum install alien as these instructions are for a non-root user 
      • cd ~/
yumdownloader --destdir ~/rpm --resolve alien rpmrebuild opencv libtiff openjpeg2 dcmtk swig openslide pugixml zlib
cd ~/rpm 
for i in *.rpm; do rpm2cpio $i | cpio -idv; done
export PATH=~/rpm/usr/bin:~/usr/sbin:${PATH}
export PERL5LIB=~/rpm/usr/share/perl5/vendor_perl
    • Check the path using echo $PATH and echo $PERL5LIB
    • The above commands will create folders like ~/rpm/bin, ~/rpm/sbin, ... etc
    • Note: If you use cpio -uidv, it will force the installation, else it does not install if the files already exist.
    • You can also use yum search {} to find newer modules to install

  3. Download precompiled ASAP binary for Ubuntu

    • I used v2.1 with py3.8
    • cd ~/rpm
wget https://github.com/computationalpathologygroup/ASAP/releases/download/ASAP-2.1/ASAP-2.1-py38-Ubuntu2004.deb
alien --to-rpm --scripts ASAP-2.1-py38-Ubuntu2004.deb
rpm2cpio asap-2.1-2.x86_64.rpm > asap-2.1-2.x86_64.cpio
cpio -uidv < asap-2.1-2.x86_64.cpio

  4. Open python interface to load multiresolutionimageinterface, the library within ASAP that lets you programatically access whole-slide-images (.tif format)

    • cd ~/rpm
    • import sys
from pathlib import Path
path_add1 = Path.cwd().joinpath('opt', 'ASAP', 'bin')
sys.path.append(str(path_add1))
assert Path(sys.path[0]).exists()
import multiresolutionimageinterface as mir
    • The above will give you an error ImportError: libijg8.so.14: cannot open shared object file: No such file or directory

Some errors I came across

  1. Issue with PERL paths
    • Cant locate Alien/Package/Deb.pm in @INC (you may need to install the Alien::Package::Deb module)
      • Solution: export PERL5LIB=~rpm/usr/share/perl5/vendor_perl
  2. Issue with dcmtk files
    • ImportError: libijg8.so.14: cannot open shared object file: No such file or directory
      • yumdownloader --destdir ~/rpm --resolve dcmtk
        • find ./ -iname '*libijg8*' --> You will find them in ~/rpm/usr/lib64
      • Following a hint from here, create a symlink
        • ln -s ~/rpm/usr/lib64/libijg8.so.14 ~/rpm/usr/lib/libijg8.so.14

Some other points

  1. If you want to find out the space occupied by all these installations
  • du ~/rpm -sh -- its around 350MB

External Refs

  1. https://unix.stackexchange.com/questions/61283/yum-install-in-user-home-for-non-admins
  2. https://wolfecomputer.wordpress.com/2017/07/14/how-to-install-a-deb-package-on-centos-linux-7/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment