delagoya

test page

delagoya

#!/usr/bin/env python
from collections import Counter
import socket
import time

import ray

ray.init()

print('''This cluster consists of

delagoya

#!/bin/bash

# Specify the desired volume size in GiB as a command line argument. If not specified, default to 20 GiB.
SIZE=${1:-20}

# Get the ID of the environment host Amazon EC2 instance.
# IMDSv2 enabled
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
INSTANCEID=$(curl  -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
REGION=$(curl  -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/\(.*\)[a-z]/\1/')

delagoya

Description: Create a VS code-server instance with an Amazon CloudFront distribution for development of Nextflow pipelines. Version 4.0.0

Parameters:
  VSCodeUser:
    Type: String
    Description: UserName for VS code-server
    Default: ec2-user
  GitUserName:
    Type: String
    Description: Global Git user.name for repo commits

delagoya

{
  "jobDefinitionName": "dask-mnp-hello-world",
  "type": "multinode",
  "platformCapabilities": [
    "EC2"
  ],
  "nodeProperties": {
    "numNodes": 3,
    "mainNode": 0,
    "nodeRangeProperties": [

delagoya

#!/bin/sh
cd /tmp
git clone https://github.com/mpitutorial/mpitutorial
cd mpitutorial/tutorials/mpi-hello-world/code
# grab the mpicc PATH locations
source /etc/profile.d/efa.sh
make all
cp mpi_hello_world /shared/mpi_hello_world

delagoya

Map of Task Execution Service (TES) to AWS

This document is an overview of how concepts from TES map to concepts in AWS Batch.

AWS Batch - Basic Concepts

AWS Batch ("Batch") has a few basic concepts that need to be understood before we can make a comparison to concepts in TES. Some relate directly to TES and others do not.

Job

delagoya

FROM rocker/r-base

RUN apt-get update
# install python tools
RUN apt-get install -y python-dev python-pip
# install AWS CLI
RUN pip install -U awscli

delagoya

Port 22 is BLOCKED! HALP!

The MCW auditorium wifi blocks the standard SSH network port, port 22. If you are to successfully connect to the EC2 instance from the MCW guest wifi, you will need to change the default port that the SSH service provides for connections. You will also need to allow network traffic to that port on the EC2 instance.

Step 1: the userdata script

The following Bash shell script will be used to add a port to the SSH service.

delagoya

Implementing a simple AWS Lambda function

In this lab, we will use the chalice Python serverless web application framework for AWS to create a REST API endpoint to reverse complement a DNA string using AWS Lambda and Amazon API Gateway.

# install chalice
$ pip install chalice