Skip to content

Instantly share code, notes, and snippets.

View full-sized avatar

André F. Rendeiro afrendeiro

View GitHub Profile
@afrendeiro
afrendeiro / test_gpus_accelerate.py
Created July 24, 2023 08:11
Multi-GPU training with hf-accelerate
View test_gpus_accelerate.py
#!/usr/bin/env python
"""
A simple example of how to use the Accelerator API
to train a vision model on a dummy dataset.
Accelerator enables training on a single or multiple GPUs.
Run once `accelerate config` to set up your configuration file.
@afrendeiro
afrendeiro / quantization_experiments.py
Last active March 9, 2023 15:36
Run experiments on different quantization settings and report speedup and error.
View quantization_experiments.py
"""
Run the experiments for the quantization speedup and plot results.
"""
import subprocess
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
@afrendeiro
afrendeiro / kde_2d_weighted.py
Last active October 17, 2022 07:33
2D weighted kernel density estimation (KDE)
View kde_2d_weighted.py
import numpy as np
import matplotlib.pyplot as plt
# class from here: http://nbviewer.ipython.org/gist/tillahoffmann/f844bce2ec264c1c8cb5
class gaussian_kde(object):
"""Representation of a kernel-density estimate using Gaussian kernels.
Kernel density estimation is a way to estimate the probability density
function (PDF) of a random variable in a non-parametric way.
@afrendeiro
afrendeiro / ngs_101.md
Last active September 26, 2022 13:19
NGS for dummies
View ngs_101.md

Introduction to next-generation sequencing (NGS)

General workflow

The current used technology for next generation sequencing is Illumina sequencing - all others cannot compete with its speed, price and output power - they have therefore specialized in niche applications (not discussed here).

Nevertheless, no sequencing technology cannot simply start sequencing one end of a chromosome until the other end.

The approach therefore is:

@afrendeiro
afrendeiro / fix_indica_tiff.py
Last active June 30, 2022 15:26
Read IndicaLabs TIFF files (exported from HALO) and write TIFF file.
View fix_indica_tiff.py
"""
Convert a TIFF file generated by IndicaLabs (HALO) to a usable TIFF file.
"""
import typing as tp
from pathlib import Path
from xml.etree import ElementTree
import tifffile
from tifffile.tiffcomment import tiffcomment
View macsima_liver.py
from pathlib import Path
import tifffile
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages
from stardist.models import StarDist2D
from csbdeep.utils import normalize
_dir = Path("HumanLiverH35")
files = list(
@afrendeiro
afrendeiro / markers_to_cell_type_labels.yaml
Last active May 9, 2022 14:39
Annotation of marker epitopes for various cell types
View markers_to_cell_type_labels.yaml
# This annotation of cell type markers is tailored for surface proteins
# normally profilid with multiplexed imaging (IMC, CODEX).
# It is a simple reference for myself and may be more detailed in some parts than others.
# This reference is hierarchical.
# This annotation is generally pan-tissue.
# Author: Andre Rendeiro
# License: CC BY-SA 2.0
cell_types:
structural:
@afrendeiro
afrendeiro / mask_to_labelme_json.py
Created July 7, 2021 17:23
Convert a labeled image mask to a JSON file compatible with labelme. Useful to export predictions and then fine tune by hand with labelme.
View mask_to_labelme_json.py
import pathlib
import numpy as np
def mask_to_labelme(
labeled_image: np.ndarray,
filename: pathlib.Path,
overwrite: bool = False,
simplify: bool = True,
simplification_threshold: float = 5.0,
View triangular_clustermap.py
import string
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
# generate some data
n_groups = 4
n_per_group = 5