Skip to content

Instantly share code, notes, and snippets.

View dangunter's full-sized avatar
🏠
Working from home

Dan Gunter dangunter

🏠
Working from home
View GitHub Profile
@dangunter
dangunter / stampede_timetest.py
Created March 14, 2012 09:24
Timing test of NetLogger Stampede DB functions
import sys, time
from netlogger.analysis.workflow.stampede_statistics import *
st = StampedeStatistics("sqlite:///gp-0.stampede.db")
w="89a25dd1-e8ae-4f15-9dea-367bbf598779"
st.initialize(w)
sys.stdout.write("method,time\n")
for fn in filter(lambda s: s.startswith('get_'), dir(st)):
sys.stdout.write("{},".format(fn))
sys.stdout.flush()
t = time.time()
@dangunter
dangunter / stampede-timetest-results.txt
Created March 14, 2012 09:50
Results from a couple of runs of stampede_timetest.py
gp-0.stampede.db
method,time
get_condor_q_time,0.000005
get_descendant_workflow_ids,0.001646
get_failed_job_instances,0.000004
get_invocation_by_time,2.845388
get_invocation_by_time_per_host,4.467101
get_invocation_info,0.000007
@dangunter
dangunter / taskgraph.py
Created March 15, 2012 16:13
task graph thoughts
"""
Task graph
==========
Some toy code to play with the aspects
of the task-graph problem. Note that this is
all in-memory, but really want to implement so it
can directly manipulate mongodb data.
Initial problem statement
--------------------------
@dangunter
dangunter / dump_blipp_as_csv.py
Created April 6, 2012 21:35
Dump from Mongo database of (old-style) Blipp Periscope data to CSV
"""
Given x,y,z MIDs, dump the data values in range a to b to a CSV
"""
import argparse
import os
import sys
#
import pymongo
from pymongo_mr import base as mr_base
@dangunter
dangunter / mg-run-history
Last active December 15, 2015 03:28
Generate run history from materials project database
## Generate graph of run history
Generating the graph requires 2 scripts
1) A Python script to extract the dates of each compound from the
database. Replace DB, USER and PASSWORD below with appropriate values:
./mg_run_history.py DB USER PASSWORD --port 27018 > runs.csv
#!/usr/bin/env python
"""
Generate many fireworks workflows.
"""
import argparse
import os
import sys
def main():
p = argparse.ArgumentParser(description="Generate fireworks")
@dangunter
dangunter / Mapping.ipynb
Created January 24, 2015 03:34
SEED mapping example
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@dangunter
dangunter / crd-skills
Last active August 29, 2015 14:18
CRD skills list
AJAX
AMR
AngularJS
Apache HTTPD
Assembly
BLAST - Basic Local Alignment Search Tool
C/C++
CGI
cloud computing
CSS
@dangunter
dangunter / wbnotes-data-1apr2015
Last active August 29, 2015 14:18
data redesign discussion notes 4/1/2015
# Weaknesses
* Sharing/granularity
* Versioning
* Narratives together with bio objs
* References
- shock vs. W.S. vs. CDS
- permissions relaxed on refs
- all refs hierarchical/one-way
- data copies may not be complete