Skip to content

Instantly share code, notes, and snippets.

View markwellis's full-sized avatar

Mark Ellis markwellis

8 followers · 1 following
View GitHub Profile
@markwellis
markwellis / gist:95f6bbc42abff3e94596
Created December 8, 2014 22:34
streaming advent
=head1 Callbacks and streaming to reduce memory usage or something
At $work we were looking into some memory issues when we noticed that every now and then we'd spot one of the workers had gone from the normal 120mb memory usage to 1.2gb
We did some investigating and it turned out our 8mb csv feeds were using a ridiculous amount of memory to be generated. I went and made a brew, and came back with a plan.
=head2 Quick overview
We have a custom view that outputs csv that's pretty much as follows
@markwellis
markwellis / streaming.pod
Created December 8, 2014 22:34
streaming advent

Callbacks and streaming to reduce memory usage or something

At $work we were looking into some memory issues when we noticed that every now and then we'd spot one of the workers had gone from the normal 120mb memory usage to 1.2gb

We did some investigating and it turned out our 8mb csv feeds were using a ridiculous amount of memory to be generated. I went and made a brew, and came back with a plan.

Quick overview

We have a custom view that outputs csv that's pretty much as follows

@markwellis
markwellis / keybase.md
Created June 2, 2017 08:26
keybase.md

Keybase proof

I hereby claim:

  • I am markwellis on github.
  • I am markellis (https://keybase.io/markellis) on keybase.
  • I have a public key whose fingerprint is E47A EF29 8879 9D1B 738A 57FC 6DCF 2280 6C91 C085

To claim this, I am signing this object: