Skip to content

Instantly share code, notes, and snippets.

View SteVwonder's full-sized avatar

Stephen Herbein SteVwonder

21 followers · 3 following
View GitHub Profile
@SteVwonder
SteVwonder / get_jobs_from_kvs.py
Created June 8, 2016 23:59
import syslog
import flux
import json
import os
from flux import core, kvs
from flux.core.inner import ffi
def get_job_times(h):
jobs_times = []
with kvs.get_dir(h, "lwj") as lwj_dir:
@SteVwonder
SteVwonder / rc1.d_sim_start
Created June 13, 2016 18:03
Launching the sim under gdb
#!/bin/bash
#
# In addition, users can set FLUX_SCHED_OPTIONS if they want flux
# to load in the sched module with non-default options.
# (e.g., FLUX_SCHED_OPTIONS="plugin=sched.backfill" will cause
# flux to load the sched module with the backfil scheduling plugin.
#
# Finally, if FLUX_SCHED_RC_NOOP=1, flux-core
# won't load in or remove sched as part of runlevel 1 and 3.
#
@SteVwonder
SteVwonder / 395-cont.md
Created June 14, 2016 20:47
Copy of our email exchange about Issue #395

@Stevwonder:

During the group meeting (either this week or last week), you mentioned that you think there is a better way to construct/wire-up the hierarchy of Flux instances than what is currently being done in the hierarchical emulator. I was wondering if you could expand on that a bit. I didn’t fully comprehend what you meant during the meeting.

My understanding of the current code is that each Flux instance stores a copy of its handle as well as a handle to its parent (and/or the root instance of the hierarchy). I’m not 100% sure how the child gets the handle to the parent, but I believe the parent’s URI is passed as an argument to the child (who then uses that URI to create a handle).

@garlick (in reply to @stevwonder):

My understanding is that Suraj implemented a dedicated scheduler to scheduler communication mechanism for hierarchical scheduling.

@SteVwonder
SteVwonder / spack-build.out
Created July 11, 2016 18:04
Flux build log under spack
==> '/g/g17/herbein1/local/active/bash-4.3.30/bin/bash' './autogen.sh'
Running aclocal ...
Running libtoolize ...
Running autoheader ...
Running automake ...
configure.ac:26: installing 'config/compile'
configure.ac:9: installing 'config/config.guess'
configure.ac:9: installing 'config/config.sub'
configure.ac:15: installing 'config/install-sh'
configure.ac:15: installing 'config/missing'
@SteVwonder
SteVwonder / tree.log
Created September 2, 2016 18:09
Flux install directory tree
/opt/flux-0.0.1
├── bin
│   └── flux
├── etc
│   ├── flux
│   │   ├── rc1
│   │   └── rc3
│   └── wreck
│   └── lua.d
│   ├── 01-env.lua
@SteVwonder
SteVwonder / gist:825129d9dc097dd9eb507acc87bd0abe
Created September 7, 2016 14:47
Copyright (c) "2016, by University of Delaware
Contributors: Robert Searles and Stephen Herbein
Affiliation: University of Delaware"
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
@SteVwonder
SteVwonder / spack-build.out
Created October 24, 2016 16:52
==> '/bin/bash' './autogen.sh'
Running libtoolize --automake --copy ...
Running autoreconf --verbose --install -I config
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal -I config -I config
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy
autoreconf: running: /usr/bin/autoconf --include=config
autoreconf: running: /usr/bin/autoheader --include=config
@SteVwonder
SteVwonder / gist:00a84f2c29139e0fea54a66d895a3a0c
Created October 25, 2016 16:31
2016-10-25T16:29:28.971952Z broker.info[0]: pmi: bootstrap time 0.0s
2016-10-25T16:29:28.982407Z broker.info[0]: wireup: 1/1 (complete) 0.0s
2016-10-25T16:29:28.982445Z broker.info[0]: Run level 1 starting
2016-10-25T16:29:28.998158Z broker.debug[0]: insmod barrier
2016-10-25T16:29:29.011427Z broker.debug[0]: insmod content-sqlite
2016-10-25T16:29:29.012058Z broker.debug[0]: content backing store: enabled content-sqlite
2016-10-25T16:29:29.024753Z broker.debug[0]: insmod kvs
2016-10-25T16:29:29.066510Z broker.debug[0]: insmod aggregator
2016-10-25T16:29:29.090461Z broker.debug[0]: insmod resource-hwloc
2016-10-25T16:29:29.097656Z broker.debug[0]: insmod job
@SteVwonder
SteVwonder / sched_proxy_c.c
Created August 18, 2017 22:53
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libgen.h>
#include <errno.h>
#include <libgen.h>
#include <czmq.h>
#include <dlfcn.h>
#include <stdbool.h>
#include <flux/core.h>
@SteVwonder
SteVwonder / module output
Created August 18, 2017 23:00
ROOT LEVEL
→ grep "sched_proxy_c" 0-broker.out
2017-08-18T22:56:31.738387Z init_prog.debug[0]: Loading sched_proxy_c module: ['flux', 'module', 'load', 'sched_proxy_c', 'root=1']
2017-08-18T22:56:31.813909Z broker.debug[0]: insmod sched_proxy_c
2017-08-18T22:56:31.817743Z sched_proxy_c.debug[0]: sched_proxy_c module is running on rank 0
2017-08-18T22:56:33.391903Z sched_proxy_c.debug[0]: child_needs_a_job_cb: received a request for work
2017-08-18T22:56:33.391932Z sched_proxy_c.debug[0]: child_needs_a_job_cb: Jobs are queued, responding immediately to request for work
2017-08-18T22:56:40.834900Z sched_proxy_c.debug[0]: new_job_cb: received a new job
2017-08-18T22:56:40.834956Z sched_proxy_c.debug[0]: new_job_cb: no outstanding requests for work, queueing job