Skip to content

Instantly share code, notes, and snippets.

@stuart-warren
Last active December 16, 2015 01:19
Show Gist options
  • Save stuart-warren/5354116 to your computer and use it in GitHub Desktop.
Save stuart-warren/5354116 to your computer and use it in GitHub Desktop.
Takes a URL to the ascii output of opentsdb and outputs json in a format easily used by the NVD3.js graph library (at least lineWithFocusChart) Will probably build on this to create a nicer frontend / dashboard NVD3 (http://nvd3.org/) - D3 js library OpenTSDB (http://opentsdb.net/) - Metric collection tool
<!DOCTYPE html>
<meta charset="utf-8">
<!-- this works from within the examples folder -->
<link href="../src/nv.d3.css" rel="stylesheet" type="text/css">
<style>
body {
overflow-y:scroll;
}
text {
font: 12px sans-serif;
}
svg {
display: block;
}
#chart1 svg {
height: 500px;
min-width: 100px;
min-height: 100px;
/*
margin: 50px;
Minimum height and width is a good idea to prevent negative SVG dimensions...
For example width should be =< margin.left + margin.right + 1,
of course 1 pixel for the entire chart would not be very useful, BUT should not have errors
*/
}
</style>
<body>
<div id="chart">
<svg style="height: 500px;"></svg>
</div>
<script src="../lib/d3.v2.js"></script>
<script src="../nv.d3.js"></script>
<script src="../src/tooltip.js"></script>
<script src="../src/utils.js"></script>
<script src="../src/models/legend.js"></script>
<script src="../src/models/axis.js"></script>
<script src="../src/models/scatter.js"></script>
<script src="../src/models/line.js"></script>
<script src="../src/models/lineWithFocusChart.js"></script>
<script src="stream_layers.js"></script>
<script>
nv.addGraph(function() {
var chart = nv.models.lineWithFocusChart();
// fix x-axis to show real dates from epoch seconds (rather than milliseconds)
chart.xAxis
.tickFormat(function(d) { return d3.time.format('%c')(new Date(d * 1000)) });
chart.x2Axis
.tickFormat(function(d) { return d3.time.format('%c')(new Date(d * 1000)) });
chart.yAxis
.tickFormat(d3.format(',.2f'));
chart.y2Axis
.tickFormat(d3.format(',.2f'));
d3.select('#chart svg')
.datum(testData())
.transition().duration(500)
.call(chart);
nv.utils.windowResize(chart.update);
return chart;
});
function testData() {
return [] //your data to go here
}
</script>
#!/usr/bin/env python
#
# Author: https://github.com/stuart-warren
# Take ascii data from OpenTSDB and output JSON for NVD3.js graphs
#
import csv
import urllib2
# Your url to ascii here
url = 'http://opentsdb.example.com:4242/q?start=2013/04/08-18:05:00&ignore=2860&m=sum:rate:tsd.hbase.rpcs%7Bhost=*,type=*%7D&o=&m=sum:10m-avg:rate:tsd.rpc.received%7Bhost=*,type=*%7D&o=axis%20x1y2&yrange=%5B1:%5D&y2range=%5B1:%5D&ylog&y2log&key=out%20right%20top%20box&wxh=1627x758&ascii'
serv_req = urllib2.Request(url=url)
serv_resp = urllib2.urlopen(serv_req)
allmetrics = []
metricreader = csv.DictReader(serv_resp,
fieldnames=['metric', 'time', 'value'],
restkey='tags',
delimiter=' ')
for row in metricreader:
# Create a unique key
key = ''.join([row['metric'], '{', ', '.join(row['tags']), '}'])
# Make a copy of row['tags']
tags = row['tags']
# Replace it with a dict version
row['tags'] = dict()
for tag in tags:
t, v = tag.split('=')
row['tags'].update({t: v})
new = True
# Check to see if we have this metric type already
for m in allmetrics:
if m['key'] == key:
new = False
# If so, just add the data points
m['values'].append({'x': int(row['time']),
'y': float(row['value'])
})
if new:
# If not, Add a new metric type
allmetrics.append({'key': key,
'metric': row['metric'],
'tags': row['tags'],
'values': [{'x': int(row['time']),
'y': float(row['value'])
}]
})
print allmetrics
@l1x
Copy link

l1x commented Dec 20, 2013

Hey,

How do you display the generated JSON with D3/NVD3 actually? It is not clear based on looking at your HTML.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment