Skip to content

Instantly share code, notes, and snippets.

View kinow's full-sized avatar
💭
😬

Bruno P. Kinoshita kinow

💭
😬
View GitHub Profile
@kinow
kinow / expdef_a002.yml
Last active February 6, 2024 12:33
Based on dummy experiment created with autosubmit expid -H local -d "bla"
View expdef_a002.yml
# What I modified
EXPERIMENT:
DATELIST: '20000101 20000102'
@kinow
kinow / flow.cylc
Created November 8, 2023 22:41
Data Notifier example WIP
View flow.cylc
#!Jinja2
{% if MEMBERS is not defined %}
{% set MEMBERS = 4 %}
{% endif %}
[scheduler]
UTC mode = true
allow implicit tasks = True
[scheduling]
@kinow
kinow / coverage.xml
Created May 19, 2023 20:50
cylc pytest-cov output
View coverage.xml
This file has been truncated, but you can view the full file.
<?xml version="1.0" ?>
<coverage version="7.2.5" timestamp="1684529350876" lines-valid="23787" lines-covered="17741" line-rate="0.7458" branches-valid="10117" branches-covered="6379" branch-rate="0.6305" complexity="0">
<!-- Generated by coverage.py: https://coverage.readthedocs.io/en/7.2.5 -->
<!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
<sources>
<source>/home/kinow/Development/python/workspace/cylc-flow/cylc</source>
</sources>
<packages>
<package name="flow" line-rate="0.7415" branch-rate="0.6293" complexity="0">
<classes>
View gist:ce0435d4ffd1e4a2fcfede53735cd03e
This file has been truncated, but you can view the full file.
---
Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
Maven home: /opt/apache-maven-3.8.5
Java version: 17.0.6, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-69-generic", arch: "amd64", family: "unix"
---
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
View INSTALL_ECFLOW.md
# git clone ...ecflow
# git checkout 5.8.4
ln -s /home/kinow/Development/python/workspace/ecflow/ /tmp/ecflow_build/ecFlow-5.8.4-Source
export WK=/tmp/ecflow_build/ecFlow-5.8.4-Source
export BOOST_ROOT=/tmp/ecflow_build/boost_1_71_0
cd $BOOST_ROOT
./bootstrap.sh --prefix=/home/kinow/Development/python/workspace/ecflow/boost
./b2 install
# boost_build.sh failed to locate /usr/bin/libcrypt.so ?
@kinow
kinow / RUN_AS_WORKFLOW.md
Last active September 3, 2022 07:59
First complete run of an Autosubmit workflow
View RUN_AS_WORKFLOW.md

Managed to complete a run of an Autosubmit workflow today for the first time, running the commands that I wanted. Here's where I was erring, and what I did to fix it.

Ubuntu sshd configurations

When you use the PS platform type, you also need to enable an SSH server. Autosubmit reads the SSH config file to load the host configuration.

@kinow
kinow / README.md
Last active September 1, 2022 04:33
Re-creating streamflow-fl with other engines
View README.md

Autosubmit

Install Autosubmit with pip install autosubmit==3.*. Follow their tutorial to set up the workflow engine (basically autosubmit configure and autosubmit install to create the directories, files, and databases).

Create an experiment with autosubmit expid --HPC local --description "autosubmit-fl...", and edit the files expdef_a000.conf and jobs_a000.conf in the ~/autosubmit/a000/ directory. The important entries to add or modify in expdef_a000.conf are:

@kinow
kinow / FIRST_EXPERIMENT.md
Last active June 26, 2022 05:58
Running my first experiment with Autosubmit
View FIRST_EXPERIMENT.md

Here's the step by step that I used to have a running workflow.

  1. Synced master branch
  2. Re-installed dependencies in venv, pip install -U -e .
  3. Removed old files, rm -rf ~/.autosubmit* and rm -rf ~/autosubmit
  4. Configured and installed Autosubmit, autosubmit configure, followed by autosubmit install
  5. Created a new dummy experiment with autosubmit expid -H local --dummy -d DummyExp
  6. Updated my experiment's platform file
@kinow
kinow / blast.wdl
Created June 2, 2022 03:38
blast.wdl
View blast.wdl
version 1.0
workflow blast {
input {
String? blast_docker_override
String blast_docker= select_first([blast_docker_override,"swr.cn-south-1.myhuaweicloud.com/cngbdb/blast:1.2"])
File queryfa
String fname = '/sfs/blastdb/2019_ncov/nucl/v6/ncov'
String method = 'blastn'
Int outfmt = 7
@kinow
kinow / app.py
Created April 27, 2022 12:29
setuptools/discussions/3288
View app.py
def hello():
print("hello")
__version__ = '0.1'