natefoo

Distribution Detection

I am working on adding support for building and distributing (via PyPI) Python Wheels with C Extensions to the Python wheel and pip packages. The discussion on Distutils-SIG continues, but I believe it is fairly certain that some effort to correctly identify Linux distributions will need to be made. I've begun efforts to add this support to wheel.

How you can help

If you have a Linux distribution or version of a listed distribution not in this gist, or one of the ones I have not directly verified, I could use the following:

• The contents of /etc/os-release, if it exists

natefoo

data_managers:
- data_table_reload:
  - all_fasta
  - __dbkeys__
  id: toolshed.g2.bx.psu.edu/repos/devteam/data_manager_fetch_genome_dbkeys_all_fasta/data_manager_fetch_genome_all_fasta_dbkey/0.0.4
  params:
  - dbkey_source|dbkey: dm6
  - reference_source|reference_source_selector: ucsc
  - reference_source|requested_dbkey: dm6

natefoo

Still working on it

Retest in progress

None

Fixing tool data then retest

None

Fixing tool resources then retest

natefoo

Manually mulling container images

This is occasionally necessary e.g. when trying to build images for old software or old versions that won't be supported by BioContainers, custom channels, etc.

Another common use case is for when you are trying to run a Galaxy tool that is missing a requirement specification for something it depends on (this is common especially with older tools that assumed Python would be present) or that requires older conda packages that did not fully specify their dependencies.

Prerequisites

You will need Docker on whatever host you plan to run on. Singularity is not required (even if building Singularity images) since the Singularity build occurs in Docker.

natefoo

I upgraded Mailman3 from <3.3.1 to >= 3.3.1 without heeding the instructions and as a result, had a ton of my list members erroneously set to disabled due to bounces.

These scripts scan your bounce log file and re-enable anyone who was disabled. You could also easily modify it to just re-enable everyone, but I didn't want to do it unilaterally just in case.

natefoo

---
- hosts: galaxyservers
  become: true
  become_user: root
  vars_files:
    - group_vars/secret.yml
  pre_tasks:
    - name: Install Dependencies
      apt:
        #name: ['acl', 'bzip2', 'git', 'make', 'python3-psycopg2', 'tar', 'virtualenv']

natefoo

# https://github.com/natefoo/total-perspective-vortex/tree/resource-params
global:
  default_inherits: default
    
tools:
  default:
    cores: 1
    mem: cores * 3.9
    env: {}
    scheduling:

natefoo

global:
  default_inherits: default

tools:
  default:
    cores: 1
    mem: cores * 3.9
    env: {}
    context:
      partition: normal

natefoo

I created this to search BioContainers images for https://github.com/NCSC-NL/OpenSSL-2022 before it turned out that the bug is probably practically unexploitable, but should be useful in the future if we have another such case:

$ $HOME/bin/scan-singularity.sh -d 2021-09-01 -i /singularity -l ./log -s $HOME '/usr/local/conda-meta/openssl-3\..*'

For my own personal notes, I ran this on a manually-created LX-Brand CentOS 7 VM (alias: singularity-scan) on rochefort with the singularity zfs dataset lofs-mounted in. From there you can install epel-release and singularity-runtime, which can work with sif files (even though it can't run them in LX). tmpfs created with:

# useradd -d /home/scan -m -s /bin/bash scan

natefoo

[program:galaxy_test_uwsgi]
command         = /usr/bin/sg G-803372 '/srv/galaxy/test/venv/bin/uwsgi --ini /srv/galaxy/test/config/uwsgi.ini'
directory       = /srv/galaxy/test/server
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 15
user            = g2test
environment     = PYTHON_EGG_CACHE=/srv/galaxy/test/var/python-egg-cache-test_uwsgi,GALAXY_EGGS_PATH=/srv/galaxy/test/eggs
numprocs        = 1