Skip to content

Instantly share code, notes, and snippets.

@glennklockwood
glennklockwood / privatelink.md
Created April 13, 2023 19:51
Private Link, Private Endpoint
@glennklockwood
glennklockwood / .gitignore
Last active May 13, 2021 19:00
Create a file on Lustre with explicitly set OSTs
lustre_createfile_layout
@glennklockwood
glennklockwood / .gitignore
Last active April 7, 2021 05:13
IOR Incompressibility Testing
.ipynb_checkpoints
@glennklockwood
glennklockwood / darshan2parquet.ipynb
Created February 7, 2021 01:28
Convert Darshan log to Parquet
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@glennklockwood
glennklockwood / cudf_ARM64_jetson.md
Last active February 4, 2021 06:57 — forked from abishekmuthian/cudf_ARM64_jetson.md
Installing cudf on ARM(aarch64)[Jetson Nano]

Installing cuDF on Jetson Nano

These instructions move in the direction of building a cuDF docker image that works on Jetson Nano.

My setup

I'm using Nvidia Jetson nano.

  • Quad-core ARM® Cortex®-A57 MPCore processor
@glennklockwood
glennklockwood / df.py
Last active December 23, 2020 02:01
File system fullness watcher
#!/usr/bin/env python3
"""Creates JSON representation of file system fullness
"""
import re
import time
import json
import subprocess
WATCHED_FS = re.compile(r'(^(dev)?tmpfs|:/cvmfs(/|\s)|/dev/|overlay)')
INTEGER_COLS = {'1K-blocks', 'Used', 'Available', 'Inodes', 'IUsed', 'IFree'}
@glennklockwood
glennklockwood / restamp-notability-exports.sh
Created September 20, 2020 22:34
restamp-notability-exports.sh
#!/usr/bin/env bash
#
# Update the created/modified dates of PDF files to match the Notability .note
# native format's metadata. Used to preserve date ordering of notes exported
# from Notability.
#
# Step 1. Back up notability to a shared drive in the native "Note" format
# Step 2. Change the backup format to PDF and create PDF versions
# Step 3. Run this script against each .note file
#
@glennklockwood
glennklockwood / ior-summary.md
Created June 3, 2020 00:38
NERSC Cori IOR Acceptance

Acceptance Test Options

The Cori acceptance test for both the Lustre file system (cscratch) and the burst buffer used IOR to obtain the peak numbers that were advertised.

DataWarp Phase I

DataWarp Phase I used 4480 processes (ppn=4) with the following IOR command-line options:

  • ./IOR -a MPIIO -g -t 512k -b 8g -o $DW_JOB_STRIPED/IOR_file -v
  • ./IOR -a POSIX -F -e -g -t 512k -b 8g -o $DW_JOB_STRIPED/IOR_file -v
@glennklockwood
glennklockwood / poke_esnet_graphql.sh
Last active October 21, 2021 23:27
Demonstrate accessing ESnet's SNMP GraphQL service
#!/usr/bin/env bash
CSRF_TOK=$(curl --verbose -D - https://my.es.net/ 2>/dev/null | grep -o 'csrftoken=[^;]*' | cut -d= -f2)
echo "CSRF token = [$CSRF_TOK]"
curl \
--referer https://my.es.net/nersc-400g \
-X POST \
-b "csrftoken=$CSRF_TOK" \
-H "X-CSRFToken: $CSRF_TOK" \