Skip to content

Instantly share code, notes, and snippets.

View greenlion's full-sized avatar

Justin Swanhart greenlion

View GitHub Profile
@greenlion
greenlion / gist:8831099
Last active August 29, 2015 13:56
Shard-Query demo queries
select count(*)
from ontime_fact
join dim_date on ontime_fact.date_id = dim_date.date_id
where `Year` between 1990 and 1994 and `Quarter` = 1;
select FlightDate, count(*)
from ontime_fact
join dim_date on ontime_fact.date_id = dim_date.date_id
join dim_flight on ontime_fact.flight_id = dim_flight.flight_id
where `Year` = 1998 and `Quarter` = 1
@greenlion
greenlion / gist:9244835
Created February 27, 2014 05:20
A sample data generator for web traffic time series log file
<?php
/* a sample of traffic data from the real website, deduplicated and REST URLs digested */
$traffic = unserialize(file_get_contents('seed.dat')) or die('Could not unserialize seed.dat');
/* 20% chance by default */
function chance($greater_than=20, $bottom=1, $top = 100) {
$toss = mt_rand($bottom, $top);
if($greater_than >= $toss) return true; else return false;
}
@greenlion
greenlion / gist:f876f4cf7f819ff3df5e
Last active August 29, 2015 14:15
FastBit UDF Specification
/*
FastBit UDF for MySQL (c) 2015 Justin Swanhart
Specification 1.0-DRAFT
*/
/*
fb_create(table_dir=>'/path/to/fastbit/data',colspec=>'c1:int,c2:int');
fb_load(table_dir=>'/path/to/fastbit/data', datafile=>'/path/to/file/to/load.txt');
fb_query(table_dir=>'/path/to/fastbit/data',query_string=>'select count(*) where c1 between 1 and 10', temp_dest=>'temporary_table_name');
fb_hits(table_dir=>'/path/to/fastbit/data',query_string=>'select count(*) where c1 between 1 and 10', temp_dest=>'temporary_table_name');
@greenlion
greenlion / gist:31ff082cda58b0e69dba
Created March 3, 2015 00:02
Shard-Query wrapper
delimiter ;;
create database if not exists shard_query;;
use shard_query;;
drop procedure if exists sq_helper;;
drop procedure if exists sq_throw;;
create procedure sq_throw(v_err text)
begin
signal sqlstate '45000' set
class_origin = 'Fastbit_UDF',
subclass_origin = 'Fastbit_UDF',
@greenlion
greenlion / presto.cpp
Created March 8, 2015 05:31
A MySQL UDF to flatten JSON using phpembed-cmake
#include "presto.h"
#include "myphp.h"
#include <iostream>
#include <string>
/* This function hooks PHP output to the error log*/
void php_log_output(const char *str){
std::cerr << str;
}
@greenlion
greenlion / flatten_udf.cpp
Last active August 29, 2015 14:16
UDF using RapidJSON to flatten a JSON document to key/value pairs
/*
* Compile with:
* g++ -std=c++11 -shared -I `mysql_config --include` `mysql_config --cxxflags` flatten_udf.cpp -o flatten_udf.so
*/
#pragma GCC diagnostic ignored "-Wwrite-strings"
#include "presto.h"
#include <iostream>
#include <string>
#include "rapidjson/reader.h"
@greenlion
greenlion / gist:70cc9aa364b7308a3bd3
Created March 9, 2015 08:08
Complex JSON search with MySQL fulltext and flattened JSON data
mysql> select doc from json2 where match(flat) against ('last_name=Vembu')\G
*************************** 1. row ***************************
doc: { "_id" : { "$oid" : "52cdef7c4bab8bd675297d8b" }, "name" : "AdventNet", "permalink" : "abc3", "crunchbase_url" : "http://www.crunchbase.com/company/adventnet", "homepage_url" : "http://adventnet.com", "blog_url" : "", "blog_feed_url" : "", "twitter_username" : "manageengine", "category_code" : "enterprise", "number_of_employees" : 600, "founded_year" : 1996, "deadpooled_year" : 2, "tag_list" : "", "alias_list" : "Zoho ManageEngine ", "email_address" : "pr@adventnet.com", "phone_number" : "925-924-9500", "description" : "Server Management Software", "created_at" : { "$date" : 1180121062000 }, "updated_at" : "Wed Oct 31 18:26:09 UTC 2012", "overview" : "<p>AdventNet is now <a href=\"/company/zoho-manageengine\" title=\"Zoho ManageEngine\" rel=\"nofollow\">Zoho ManageEngine</a>.</p>\n\n<p>Founded in 1996, AdventNet has served a diverse range of enterprise IT, networkin
template<typename K, class D>
class PageManager {
private:
Node* first;
Node* last;
K nodecount = 0;
K max_nodes = 0;
std::hash_set<K, Node*> page_table;
void* flusher = NULL; // The flusher is a callback function which is called when pages
// fall off the tail of the LRU maintained by the PageManager.
@greenlion
greenlion / rewrite.cpp
Last active October 10, 2015 02:04
MariaDB-TroySQL OLAP plugin interface
save_client_capabilities = thd->client_capabilities;
thd->client_capabilities |= CLIENT_MULTI_QUERIES;
save_vio = thd->net.vio;
thd->net.vio = 0;
for(table in tables) {
dispatch_command(COM_QUERY, thd, buf, len);
}
thd->client_capabilities = save_client_capabilties;
thd->net.vio = save_vio;
Nearly everybody is familiar with the story about how Isaac Newton discovered gravity when an apple fell on his head. He hypothesized that the apple was pulled to the Earth by Earth’s gravity, when it fell from the tree, which is a reasonable hypothesis. Literally every educated person born after Newton’s work was published (Whitman, 1999) everyone has agreed with this hypothesis, including Einstein. In Newton’s view, the Earth has a very strong pull on objects, and that pulling force is reduced greatly by the “opposite” pull on Earth by all other objects. This this the very small constant G which pulls almost all the force out of the relationship between two masses in Newton’s equation: F=(Gm_1 m_2)/r^2 .
Unfortunately, Newton got that wrong. The Earth exerts an incredible outward force which affects the topology of space which I call displacement deformation of space (or just displacement). The displacement of the mass of the Earth would throw everything off the surface but for the opposite displace