Skip to content

Instantly share code, notes, and snippets.

@welch
welch / installing-opencv-2.4.5-macosx-10.8.4-anaconda-1.6.0
Created September 6, 2013 19:17
How to to build OpenCV 2.4.5 from the distribution tarball using cmake, on Mac OSX 10.8.4, linked to an anaconda installation.
It is a rite of passage to post one's successful build instructions for OpenCV on a Mac
after you've tried all the other conflicting instructions out there and still failed.
brew failed for me (was this because I could never get a happy brew doctor situation?
I'll never know). macports? nope. build-from-source recipes? I didn't find one that
worked for me.
Here's what did work to build OpenCV 2.4.5 from the distribution tarball using cmake,
on Mac OSX 10.8.4, linked to an anaconda installation rather than the system python.
It is a mashup of various bits of advice out there. If you're already comfortable with
build/install from source, all you need to read is the cmake invocation in step 3 and
@welch
welch / README.md
Last active April 27, 2016 20:44
exponential smoothing with de-seasonalization

Forecast

Holt-Winters smoother/forecaster for anomaly detection

Forecast threads a smooth curve through a noisy timeseries in a way that lets you visualize trends, cycles, and anomalies. It can be used as part of an automatic anomaly detection system for metric timeseries (that is, sequences of timestamped numerical values).

It accomplishes this using a variation of Holt-Winters forecasting -- more generally known as exponential smoothing. Forecast decomposes a noisy signal into level, trend, repetitive "seasonal" effects, and unexplained variation or noise. In this example, a "season" is a day long, and we model repeated day-over-day variation. The result is a smoothed version of the signal which may be used to forecast future values or detect unexpected variation. This approach has been successfully used for [network anomaly detection](https://www.usenix.org/legacy/event/lisa2000/full_papers/br

@welch
welch / predict.juttle
Last active October 20, 2015 21:13
predict: return the unsurprising portion of a metric stream by estimating trend and seasonal effects
// Predict:
// Trend, seasonality, and noise estimation for metric streams.
//
// Predict is a sub which consumes a metric stream and emits a
// prediction stream based on estimated trend, seasonality, and a
// smoothed version of the input. Prediction errors (the difference
// between predict's output and the input) can signal anomalies in the
// stream.
//
// usage: predict [options]
@welch
welch / README.md
Last active October 7, 2015 19:54
ingredients for the predict proc, in juttle

The juttle predict proc combines trend, seasonal, and level prediction. Although it is native javascript, many of its components can also be written in juttle, and those examples are here.

@welch
welch / alert-context.juttle
Last active October 1, 2015 01:24
annotate an alert event with recent lines from a logging stream
// Alerting with context:
//
// This example shows how to annotate an alert event with recent lines from
// a logging stream, to give context for the alert. This is accomplished with
// a custom reducer that tails the log stream, and a join of the current tail
// against each alert event as it occurs.
//
// The log-tailing stream is joined using the -table option, which means the
// join will fire only when the alert stream gets a new point, but not fire
// when the log stream gets new points.
@welch
welch / cust-info.json
Last active September 28, 2015 19:41
customer info for join -table examples
[
{"time":"2016-01-01T00:00:00.000Z", "cust_id":1, "email":"lou@grainger.com"},
{"time":"2016-01-01T00:00:02.000Z", "cust_id":2, "email":"bubba@nytimes.com"},
{"time":"2016-01-01T00:00:03.000Z", "cust_id":3, "email":"sam@hotmail.com"}
]
@welch
welch / add-streams.juttle
Created September 28, 2015 18:18
example of collectd ingest in which several metrics are combined
// example of ingesting multiple metrics at a regular 20s pulse from
// collectd (timestamps are aligned across streams, simulated here),
// and merging them onto a single point so relative percentages can be computed.
//
// Useful patterns in add_streams:
// put *name = value applied to points having {name=foo, value=bar} layout,
// gives the values unique names in preparation for joining
// use of single-stream join to combine a variable number of input streams
// reduce -groupby by, which is the same as reduce by, but is convenient inside a sub.
// reduce time=first(time) gives reducer results same timestamp as their inputs
@welch
welch / random-demo.juttle
Created September 23, 2015 22:59
demo basic stdlib.random functions
// stdlib.random demos:
//
import "random.juttle" as random;
export sub poissonHisto() {
// render draws from 3 poisson distributions as a scatter chart.
// see https://en.wikipedia.org/wiki/Poisson_distribution
emit -limit 10000 -from 0
| put lambda_1=random.poisson(1), lambda_4=random.poisson(4), lambda_10=random.poisson(10)
| split lambda_1, lambda_4, lambda_10
@welch
welch / cpuZ-demo.juttle
Created September 23, 2015 22:43
cpuZ: visualize z-scores for a cpu metric.
// stats demos: statistics 101 for juttle data flows
//
// Each of the stream standardization approaches offered in stats is
// appropriate in a different setting. The demos here show some good
// and some bad choices.
//
import "stats.juttle" as stats;
// cpuZ: visualize z-scores for a cpu metric.
//
@welch
welch / wtf.spec.md
Created August 19, 2015 15:39
why does the second test fail?

WTF?

The basic set, fed a steady stream of two groups; A group

Juttle

emit -from Date.new(0) -limit 10
| put T = Duration.seconds(time - Date.new(0))
| ( put name = "A" ; put name = "B", T = -T)

| (