Skip to content

Instantly share code, notes, and snippets.

@maddijoyce
maddijoyce / kinesis.md
Last active January 25, 2019 03:25
kinesis

About Kinesis

  • AWS Kinesis is a managed data streaming/processing API

  • It's most commonly compared to Kafka, linkedin's open source data processing system, with the obvious difference being that all infrastructure is managed

  • The big benefit of Kinesis is the managed infrastructure. Scaling is made simple through the concept of shards.

  • For each shard, limit of 1MB/second input and 2MB/second output

  • It's also possible to add enhanced fan-out consumers, each with their own 2MB/second output limit

  • Standard data retention is 24 hours but can be increased up to 7 days. Since kinesis is replayable, this data is accessible as needed throughout the retention period.

@maddijoyce
maddijoyce / epicExample.ts
Last active May 29, 2018 04:29
Code Style
/*
SRP: I watch for UTIL_OPERATION_STARTED actions aqnd emit an UTIL_OPERATION_TIMEOUT if UTIL_OPERATION_DONE not seen in time
*/
import { Observable } from 'rxjs';
import { actionGuards, actionCreators } from '../../actions';
import { SharedEpic } from './../_helpers';
const OPERATION_TIMEOUT_WAIT = 15000;
const setConversationMapper : SharedEpic = (action$, store) => {
@maddijoyce
maddijoyce / hookLoader.js
Created June 9, 2016 04:39
Istanbul hook loader to be used programatically
import { hook, Instrumenter } from 'babel-istanbul';
import { clone } from 'lodash';
let instrumenter = null;
const baselineCoverage = {};
export function getCoverageObject() {
global.coverage = global.coverage || {};
return global.coverage;
}