Skip to content

Instantly share code, notes, and snippets.

Stephen Herbein SteVwonder

Block or report user

Report or block SteVwonder

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View t2400-job-exec-test.valgrind.log
==7612== Memcheck, a memory error detector
==7612== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==7612== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==7612== Command: /home/sherbein/Repositories/flux-framework/flux-core/src/broker/.libs/flux-broker --setattr=rundir=/tmp/flux-316-FepHh4 --setattr=tbon.endpoint=ipc://%B/req -Sbroker.rc1_path=/home/sherbein/Repositories/flux-framework/flux-core/t/rc/rc1-job -Sbroker.rc3_path=/home/sherbein/Repositories/flux-framework/flux-core/t/rc/rc3-job -Slog-filename=t2400-job-exec-test.broker.log -Slog-forward-level=7 -Sinit.rc2_timeout=300 sh\ ./t2400-job-exec-test.t\
==7612==
==7612== Thread 13:
==7612== Invalid read of size 8
==7612== at 0xC65A6B8: kill_timer_cb (job-exec.c:338)
==7612== by 0x4898142: ev_invoke_pending (ev.c:3322)
==7612== by 0x489B2F2: ev_run (ev.c:3726)
View ids.txt
→ jq '.graph.nodes[] | .id' ./test.jgf
"146337"
"146338"
"146339"
"146340"
"146341"
"146342"
"146343"
"146344"
"215384"
View check.sh
#!/bin/bash
# checkout LGPL branch
if [[ "$(git rev-parse --abbrev-ref HEAD)" != "lgpl2_libflux" ]]; then
git checkout lgpl2_libflux
fi
# get all the files changing to LGPL
git diff --name-only upstream/master | grep 'src/common' > full_path_changed_to_lgpl.txt
# get all the headers included by the files changing to LGPL
@SteVwonder
SteVwonder / R-notes.org
Created Oct 2, 2018
Notes from discussion on R
View R-notes.org

Identify the requirements on R

Consumers/Produces

  • Definition: express a resource set
  • R_local: subset of R per broker rank
    • IMP consumes this
  • Producers + Consumers
    • Scheduler produces
    • Scheduler consumes
      • Resources incrementally handed back from the exec
    • Exec consumes
View frozen-backtrace
→ pgrep -af Wavebox
27094 /opt/wavebox/Wavebox --mailto > /dev/null
27096 /opt/wavebox/Wavebox --type=zygote --no-sandbox
27128 /opt/wavebox/Wavebox --type=renderer --service-pipe-token=DC3DAE745D3E9434C0739F346A1565A0 --lang=en-US --app-path=/opt/wavebox/resources/app.asar --enable-plugins --node-integration=true --webview-tag=true --no-sandbox --background-color=#f2f2f2 --context-id=2 --enable-pinch --num-raster-threads=4 --enable-main-frame-before-activation --content-image-texture-target=0,0,3553;0,1,3553;0,2,3553;0,3,3553;0,4,3553;0,5,3553;0,6,3553;0,7,3553;0,8,3553;0,9,3553;0,10,3553;0,11,3553;0,12,3553;0,13,3553;0,14,3553;0,15,3553;0,16,3553;0,17,3553;1,0,3553;1,1,3553;1,2,3553;1,3,3553;1,4,3553;1,5,3553;1,6,3553;1,7,3553;1,8,3553;1,9,3553;1,10,3553;1,11,3553;1,12,3553;1,13,3553;1,14,3553;1,15,3553;1,16,3553;1,17,3553;2,0,3553;2,1,3553;2,2,3553;2,3,3553;2,4,3553;2,5,3553;2,6,3553;2,7,3553;2,8,3553;2,9,3553;2,10,3553;2,11,3553;2,12,3553;2,13,3553;2,14,3553;2,15,3553;2,16,3553;2,17,3553;3,0,3553;3,1,355
@SteVwonder
SteVwonder / dump_py_bindings.py
Last active Feb 15, 2019
Statistics and Raw Output on Coverage of Python Bindings
View dump_py_bindings.py
#!/user/bin/env python
import re
import inspect
from collections import defaultdict
import flux
from _flux import _core
from _flux._core import ffi
from flux import jsc
@SteVwonder
SteVwonder / job-event-log.org
Last active Aug 30, 2018
Notes from the Flux Team Brainstorming Sessions
View job-event-log.org

Use-Cases

User-initiated

Handled by ingest module

  • Submitting a job for immediate scheduler processing (not held)
    • Ingest module creates the first event to “ingested” (maybe “submitted”)
  • Submitting a job as held

Proxied through job manager module

  • Cancelling a job
    • From submitted
    • From allocated
View sierra-one-socket.hwloc (cores 0-87)
# herbein1 at sierra4361 in /nfs/tmp2/herbein1/sierra-test [13:42:17]
→ bsub -nnodes 1 -G guests -Is jsrun -n1 --bind=none taskset -c 0-87 bash -c 'CUDA_VISIBLE_DEVICES=0,1,2,3 CUDA_DEVICE_ORDER=PCI_BUS_ID hwloc-ls --restrict binding'
Job <143722> is submitted to default queue <pbatch>.
<<Waiting for dispatch ...>>
<<Starting on sierra4369>>
Machine (128GB total) + Group0 L#0 + NUMANode L#0 (P#0 128GB) [80/18667]
Package L#0
L3 L#0 (10MB) + L2 L#0 (512KB)
L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
PU L#0 (P#0)
@SteVwonder
SteVwonder / parse_podcast.py
Created Apr 10, 2018
Count words in the transcript of Sam & Ezra's podcast
View parse_podcast.py
#!/usr/bin/env python
import re
sam_wc = 0
ezra_wc = 0
speaker = None
with open('podcast-transcript.txt', 'r') as infile:
View hierarchy_config.json
{
"levels": [
{
"cores_per_child": 36,
"num_children": 8
},
{
"cores_per_child": 1,
"num_children": 36
}
You can’t perform that action at this time.