Skip to content

Instantly share code, notes, and snippets.

View mikeatlas's full-sized avatar
💻

Mike Atlas mikeatlas

💻
30 followers · 38 following
View GitHub Profile
@mikeatlas
mikeatlas / pom.xml
Created February 3, 2015 15:53
GeoMesa GDELT Ingest CDH 5.3 Accumulo 1.6 pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>geomesa-gdelt</groupId>
<artifactId>geomesa-gdelt-accumulo1.5</artifactId>
<name>GeoMesa GDELT</name>
<version>1.0-SNAPSHOT</version>
@mikeatlas
mikeatlas / fix csv
Created June 23, 2015 20:57
some simple in/out CSV parsing/fixing
// Fix up our CSV data
var fs = require('fs');
var csv = require('csv');
var allWPIsInputFs = fs.createReadStream('./input-data/all-wpi-lat-long.csv');
var mappedWPIsInputFs = fs.createReadStream('./input-data/mapped-wpi-lat-lons.csv');
var output = fs.createWriteStream('./input-data/all-wpi-lat-long_trim.csv');
var parser = csv.parse({delimiter: ','});
@mikeatlas
mikeatlas / index.html
Created July 6, 2015 19:24
testing
This is a test block.
@mikeatlas
mikeatlas / index.html
Last active August 28, 2015 17:54 — forked from tafsiri/index.html
2D Picking with canvas
<html lang="en">
<head>
<meta charset="utf-8">
<title>2D Picking with canvas</title>
<meta name="description" content="">
<meta name="author" content="Yannick Assogba">
<script src="//rawgit.com/mrdoob/stats.js/master/build/stats.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/dat-gui/0.5/dat.gui.js"></script>
@mikeatlas
mikeatlas / sync_ftp_to_s3.md
Last active January 12, 2018 15:54
syncing ftp to s3 one-time really fast.

Original idea from Transfer files from an FTP server to S3 by "Hack N Cheese".

I moved roughly a terrabyte in less than an hour. Granted, I couldn't take advantage of lftp's --parallel=30 switch due to my ftp source limiting me to one connection at a time, but use-pget-n=N did seem to help out.

  • Get a fast Ubuntu 14.4 EC2 box on Amazon for temporary usage (I went with m1.xlarge) so data tranfers aren't limited by your local bandwidth at least. I also attached a fat 2TB EBS volume and symlinked it to /bigdisk, and made sure the EBS volume was deleted after I terminated this EC2 box. I hope lftp 2.6.4 is available as a stable package by the next time I attempt this.
  • Build lftp 2.6.4+ (Not easy to compile, so read the INSTALL file and plow through all your missing dependencies - you'll also need to re-run `sudo ./configure && su
@mikeatlas
mikeatlas / latency.txt
Last active September 12, 2015 16:56 — forked from jboner/latency.txt
Latency Numbers Every Programmer Should Know
Latency Comparison Numbers
--------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD* 150,000 ns 0.15 ms
@mikeatlas
mikeatlas / keybase.md
Created September 22, 2015 20:15
I am mikeatlas (https://keybase.io/mikeatlas) on keybase

Keybase proof

I hereby claim:

  • I am mikeatlas on github.
  • I am mikeatlas (https://keybase.io/mikeatlas) on keybase.
  • I have a public key whose fingerprint is 083F C7DB EE6C 4D77 D06A 30CF 4FAA AD15 C512 DC33

To claim this, I am signing this object:

@mikeatlas
mikeatlas / parse_iso8601.go
Created October 12, 2015 01:40 — forked from hnakamur/parse_iso8601.go
parse ISO8601 datetime in go http://play.golang.org/p/xe2mWIHZGU
package main
import (
"fmt"
"time"
)
func main() {
t, err := time.Parse(time.RFC3339Nano, "2013-06-05T14:10:43.678Z")
if err != nil {
@mikeatlas
mikeatlas / .golang-example-gitlab-ci.yml
Last active February 2, 2024 08:08
Example Go GitLab CI setup (no root/sudo required)
# See docs/examples
# http://doc.gitlab.com/ce/ci/quick_start/README.html
# http://doc.gitlab.com/ce/ci/yaml/README.html
# GitLab CI template for Go tests. Note this installs
# a new working copy of Go in a non-standard path such
# that sudo/root is not needed for the install stage.
# note that this particular install-environment stage
# is overly verbose in order to debug anything tricky
@mikeatlas
mikeatlas / docker_centos
Created June 30, 2016 21:22 — forked from ravibhure/docker_centos
Docker for CentOS
# Disable selinux as it interferes with functionality of LXC
$ setenforce 0
$ echo 'SELINUX=disabled' > /etc/selinux/config
# Download and setup Fedora EPEL Repository
$ yum -y install http://ftp.riken.jp/Linux/fedora/epel/6/i386/epel-release-6-8.noarch.rpm
# Setup hop5.in repository
$ wget http://www.hop5.in/yum/el6/hop5.repo -O /etc/yum.repos.d/hop5.repo