Config files @Clinical-Genomics

config-conventions.md

General config file

• Format: toml
• Name: .clinicalgenomicsrc or clinicalgenomics.cfg
• Location: $HOME/ (default)
• Install: pip install toml (python)

Downside: it's not obvious how you would read a TOML file in Bash.

Solution 1

[parameters.ini]
database_user    = "user"
database_version = 20110611142248

$ awk -F '=' '{if (! ($0 ~ /^#/) && $0 ~ /database_version/) print $2}' parameters.toml

Solution 2

Use something like SHYAML which handles more complex cases: https://github.com/0k/shyaml.

Solution 3

Rely on config files that can be sourced to set environment variables and read them from various scripts.

This adds some limitations to the structure of the config file. It will also lead to the use of very specific and long variable names to avoid conflicts.

It's also inherently insecure even though we should be able to trust each other. More information in this article. The article suggests extracting variables, writing to a different file and sourcing it. That might make it possible to convert a TOML file.

$ cat variables.cfg
ENV_VARIABLE=rastapopoulos
$ source variables.cfg
$ echo "${ENV_VARIABLE}.scilifelab.se"
rastapopoulos.scilifelab.se

Conventions

General variables

Some variables might be useful for a number of scripts. For these, we need to define some conventions to keep consistent.

The most obvious variables could go in the "global" scope. They would in that case represent reserved keywords.

# this is a TOML document. Boom.

server = "clinical-db.scilifelab.se"

... which converts to:

{
  "server": "clinical-db.scilifelab.se"
}

Scoping variables

Each specific application should claim and use it's own scope:

# this is a TOML document.

[clinical_db]
# these are used to access clinical-db through an ssh tunnel
user = "remoteuser"
password = "some-password"

In this case, [clinical_db] is the scope and will be converted to a node in a hash table/dictionary when loaded in your favorite programming language.

{
  "clinical_db": {
    "user": "remoteuser",
    "password": "some-password"
  }
}

Example

A real world-like example might look like this:

# general config stuff
this = "thalamus.scilifelab.se"
config_dir = "/home/clinical/CONFIG/"
runbase = "/home/hiseq.clinical/Runs/"
old_runbase = "/home/hiseq.clinical/oldRuns/"

[backup]
copied = "/home/clinical/BACKUPCOPIED/"
copied_runs = "/home/hiseq.clinical/oldoncluster"
dir = "/home/clinical/BACKUP/"

server = "rastapopoulos.scilifelab.se"
server_dir = "/mnt/hds/proj/bioinfo/BACKUP/"

[preproc]
runbase = "/home/clinical/RUNS/"
old_runbase = "/home/clinical/OLDRUNS/"