Skip to content

Instantly share code, notes, and snippets.

View natefoo's full-sized avatar

Nate Coraor natefoo

View GitHub Profile

Multi-host Galaxy Server

usegalaxy.org uses a five host setup:

  • 2 x hosts for serving web requests
  • 2 x hosts for handling jobs and Pulsar staging
  • 1 x host for the database

Web requests are balanced only with a DNS round-robin (i.e. there are two A records for usegalaxy.org), which is not ideal since it does not do any load balancing or health checks. A better setup would include a 6th host with nginx proxying the web hosts.

@natefoo
natefoo / 00-README.md
Last active February 27, 2020 20:08
uWSGI Zerg Mode + Mules

Background

Two commonly used [Galaxy][galaxy] server configurations are the use of [uWSGI Zerg Mode][uwsgi-zerg-mode] and [uWSGI Mules][uwsgi-mules] as [Galaxy job handlers][galaxy-scaling]. These features are not easily compatible because Galaxy job handlers rely heavily on having unique server names, and handlers' server names must be persistent across restarts. Because zerg mode results in running two Galaxy servers simultaneously (however briefly), using mules with zerg mode would necessarily mean running mules with overlapping server names.

Solution

In a typical Galaxy zerg mode setup, the newly started zergling (B) terminates the old zergling (A) once B is ready to serve requests. Zergling B then continues to serve requests until another zergling (C) is started and terminates B.

It is possible to get both zerg mode and mules working together by configuring zergling B to start without mules, and perform a double zerg dance on each restart:

@danmack
danmack / gist:6d69a6d2ab08e6dbf8a1727e9c2267a0
Created December 5, 2018 16:32
SmartOS add disk to bhyve linux container
Adding a new disk to a bhyve vm takes a few steps.
1. increase the quota on the zfs dataset to accomodate the additional
space.
2. create a zvol inside the zone
3. stop the vm and update its configuration to reflect the change
4. start the vm, login and use the disk
# update quota
galaxy_main=> EXPLAIN ANALYZE SELECT job.id AS job_id,
job.create_time AS job_create_time,
job.update_time AS job_update_time,
job.history_id AS job_history_id,
job.library_folder_id AS job_library_folder_id,
job.tool_id AS job_tool_id,
job.tool_version AS job_tool_version,
job.state AS job_state,
job.info AS job_info,
job.command_line AS job_command_line,
@natefoo
natefoo / installers.md
Last active March 17, 2017 17:56
How to install tools on Test and Main