Help Wanted...

FeeChart_Task.md

Task: starting with a bunch of .json files (read from stdin), generate a Google Charts javascript file on stdout.

Reward: CLAIMED!

Spec:

json-to-google-chart utility written in Python 2.7; will be released as public-domain, open source.

Input will be a series of JSON files, one after another, each of the form:

{"timestamp": 1402503121, "priority_estimates": [54187268400.0, 7974115018.0, 2321397999.0, 1258815207.0, 794378941.0, 583741516.0, 427508991.0, 338266118.0, 269237500.0, 227228318.0, 191546029.0, 168449108.0, 150728972.0, 135565268.0, 123221976.0, 117162662.0, 108344079.0, 103956121.0, 99205128.0], "fee_estimates": [0.00089285, 0.00044843, 0.00044444, 0.00044247, 0.00038759, 0.0003231, 0.00024875, 0.00021186, 0.00016207, 0.00013801, 0.00012422, 0.00011413, 0.00010395, 0.0001002, 8.62e-05, 7.457e-05, 6.016e-05, 4.464e-05, 3.891e-05]}

Command-line options will be:

--key=which_key_to_use :  e.g. --key=fee_estimates  or --key=priority_estimates
--values=comma-separated-list-of-indices  :  e.g. --values=0,5  to plot just 1-confirmation and 6-confirmation
--value_multiply=1.0 : e.g. --value_multiply=1e6 --key=fee_estimates to plot fees in micro-bitcoins 

Output will be a complete .html file that uses the Google Charts API (see https://developers.google.com/chart/ ) to draw values-over-time as a LineChart. X axis will be dates (taken from "timestamp"); the unix timestamp must be converted into human-friendly date. Y axis will be floating point, starting at 0.0 (I believe Google Charts will auto-scale based on the data).

Sample input:

{"timestamp": 1402148967, "priority_estimates": [60435736872.0, 3182878368.0, 897328767.0, 303933454.0, 144259603.0, 104767493.0, 90490508.0, 79065418.0, 77081538.0, 76151857.0, 75395385.0, 73863636.0, 69082255.0, 69082255.0, 68306779.0, 68306779.0, 68306779.0, 64671619.0, 62335560.0], "fee_estimates": [0.00044843, 0.00044247, 0.00034129, 0.00022831, 0.00016207, 0.00014925, 0.00012763, 0.00012131, 0.00011441, 0.00010384, 0.00010224, 7.8e-05, 6.734e-05, 6.734e-05, 6.596e-05, 6.596e-05, 6.006e-05, 5.984e-05, 4.444e-05]}
{"timestamp": 1402372801, "priority_estimates": [62661404587.0, 10391270193.0, 2290839703.0, 1051282051.0, 587819590.0, 407959578.0, 322331912.0, 251882594.0, 222342466.0, 194832821.0, 175968997.0, 157038509.0, 147859907.0, 140718205.0, 133333333.0, 123221976.0, 115736154.0, 105463889.0, 100925000.0], "fee_estimates": [0.00052356, 0.00044642, 0.00044444, 0.00042801, 0.00038759, 0.00026809, 0.00022883, 0.00017271, 0.00012562, 0.00010604, 8.62e-05, 6.738e-05, 5.324e-05, 4.464e-05, 4.078e-05, 3.891e-05, 3.875e-05, 3.875e-05, 3.875e-05]}
{"timestamp": 1402459202, "priority_estimates": [52270908580.0, 8721998124.0, 2413703275.0, 1298293039.0, 784526998.0, 544021154.0, 397573818.0, 314609660.0, 256976166.0, 214276550.0, 187041255.0, 161243169.0, 146404234.0, 134471399.0, 123087844.0, 115736154.0, 107971270.0, 102150103.0, 95968912.0], "fee_estimates": [0.0005541, 0.00044642, 0.00044444, 0.00041493, 0.00038759, 0.00026809, 0.00024752, 0.00020831, 0.00016207, 0.00013856, 0.00012515, 0.00011764, 0.00010976, 0.00010373, 8.904e-05, 8.282e-05, 6.851e-05, 5.321e-05, 4.196e-05]}

I took your challenge to do it in bash. I also don't know Python. My solution massages the JSON in the browser.

if you still need it in Python too then poke me on IRC and I'll get on it ;-)

@berndjendrissek : does not work with the sample input I gave.

I'm starting with (potentially hundreds of) individual .json files, one created per day. Passing the names of all of those files on the command line will not scale (will run into command-line limits), so I want to cat *.json | make_a_graph > graph.html.

@gavinandresen : If the shell can't expand the command "jsontocharts *.json" due to running into argument limits, then "cat *.json" is not far behind. How about "find . -name *.json |jsontocharts @/dev/stdin >estimates.html" or even "ls *.json |jsontocharts @/dev/stdin" (though that's vulnerable to argument limits again)? New version uploaded to implement @ATFILE syntax. But hey, if you insist, I'll make the script split on }{

@berndjendrissek : no need to get snippy, helpful suggestions on how I'm being an idiot are always welcome. I outsourced this exactly because I didn't want to spend time thinking much about it.

Reward sent, thanks!