Skip to content

Instantly share code, notes, and snippets.

View ankurcha's full-sized avatar

Ankur Chauhan ankurcha

18 followers · 4 following
View GitHub Profile
@ankurcha
ankurcha / restaurant.plm
Created December 15, 2015 22:14
;;; -*- Mode: Lisp; Package: STELLA; Syntax: COMMON-LISP; Base: 10 -*-
;;; Powerloom file for csci561 Fall2006 project3.
;;; Author: Jon L. Stinzel
;;;
;;; Usage: (load-file "/fullPathToFile/restaurant.plm")
;;; (in-module "restaurant")
;;;
;;;
@ankurcha
ankurcha / AvroToBigQuery.java
Created October 23, 2015 05:29
Helper class to convert Apache Avro schema to BigQuery Table schema
package com.mallo64.dataflow
import com.google.api.client.json.GenericJson;
import com.google.api.services.bigquery.model.TableCell;
import com.google.api.services.bigquery.model.TableFieldSchema;
import com.google.api.services.bigquery.model.TableRow;
import com.google.api.services.bigquery.model.TableSchema;
import com.google.cloud.dataflow.sdk.transforms.DoFn;
import org.apache.avro.Schema;
import org.apache.avro.specific.SpecificRecord;
@ankurcha
ankurcha / drywall.md
Last active January 19, 2016 12:38
[ Project Drywall ] Because software needs more than just colums and beams Raw

This is an aggregation of thoghts, observations, talks, blogs, code reviews and many sleepless nights. Obviously not complete or exhaustive.

Edits/Comments welcome!

Developer Production

  • Measure success of developer experience by time to setup.
  • Same scripts for CI as for developer sandbox.
  • Each app should include the build sripts that are used to compile from scratch in a sandbox.
  • Mentality - Ask the question - Can we opensource/handoff this repo as it stands today?
  • Readme.md must be complete and contain 3 main sections
@ankurcha
ankurcha / example.yml
Created August 11, 2015 09:08
template: Hello, %s!
defaultName: Stranger
server:
applicationConnectors:
- type: http
port: 45080
adminConnectors:
- type: http
port: 45081
@ankurcha
ankurcha / streamToState.scala
Created July 24, 2015 23:35
Function to convert a stream of events into the "state" of the stream. The presence of a Bad event determines if it is errored before play request or after.
// events
trait Event
sealed abstract class BadEvent() extends Event
sealed abstract class GoodEvent() extends Event
sealed case class Ping() extends GoodEvent
sealed case class PlayRequest() extends GoodEvent
sealed case class Buffer() extends BadEvent
// stream states
trait StreamState
@ankurcha
ankurcha / spark-constraints.log
Created June 29, 2015 17:29
ubuntu@i-644498b2-mesos-slave-us-east-1e:~/spark-1.5.0-SNAPSHOT-bin-2.2.0$ bin/spark-shell --verbose
Using properties file: /home/ubuntu/spark-1.5.0-SNAPSHOT-bin-2.2.0/conf/spark-defaults.conf
Adding default property: spark.serializer=org.apache.spark.serializer.KryoSerializer
Adding default property: spark.driver.memory=5g
Adding default property: spark.mesos.constraints=zone:us-east-1a
Adding default property: spark.master=mesos://zk://10.96.239.120:2181,10.96.248.254:2181,10.96.218.65:2181/mesos_qa
Adding default property: spark.executor.uri=http://com.brightcove.rna.repo.dev.s3.amazonaws.com/spark-1.5.0-SNAPSHOT-bin-2.2.0.tgz
Parsed arguments:
master mesos://zk://10.96.239.120:2181,10.96.248.254:2181,10.96.218.65:2181/mesos_qa
deployMode null
@ankurcha
ankurcha / detect.sh
Created June 9, 2015 22:00
#!/bin/bash
# This script emits the task host:port for the DEPENDENCY_URI relative to the
# MARATHON_APP_ID. This is useful when getting the mongodb host:port in the
# current application pod which is located at '../../database/mongo/tasks'.
## MARATHON_HOSTS
## MARATHON_APP_ID
## DEPENDENCY_URI
@ankurcha
ankurcha / spark-console.log
Last active August 29, 2015 14:21
achauhan at pinecone in ~/Downloads/spark-1.3.1-bin-hadoop2.6
$ bin/spark-submit --master local[8] --class com.brightcove.analytics.tacoma.Main ~/Projects/tacoma/target/scala-2.10/tacoma-*.jar
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/05/12 14:46:11 INFO CheckpointReader: Checkpoint files found: file:/tmp/checkpointDir_tacoma/checkpoint-1431467097000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467096000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467095000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467094000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467093000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467092000,file:/tmp/checkpointDir_tacoma/checkpoint-1431467091000,file:/tmp/checkpointDir_tacoma/checkpoint-143146
@ankurcha
ankurcha / keybase.md
Created November 23, 2014 23:41

Keybase proof

I hereby claim:

  • I am ankurcha on github.
  • I am ankurcha (https://keybase.io/ankurcha) on keybase.
  • I have a public key whose fingerprint is 3444 C079 714A 6AF2 6529 3930 1C33 7DEB 6D46 1C4A

To claim this, I am signing this object:

@ankurcha
ankurcha / error1.sh
Last active July 1, 2016 12:36
mesos on hadoop
STARTUP_MSG: java = 1.8.0_25
************************************************************/
14/10/28 06:27:07 INFO mapred.JobTracker: registered UNIX signal handlers for [TERM, HUP, INT]
14/10/28 06:27:08 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: local
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2165)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1764)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1757)