Skip to content

Instantly share code, notes, and snippets.

View typestruct's full-sized avatar

M. Eric DeFazio typestruct

View GitHub Profile
@typestruct
typestruct / NetflixPrizeTypeStructPrototype.java
Last active August 29, 2015 14:08
typestruct Explaining Usage based on Netflix Prize Training DataSet (Including Prototyping large 100+million record dataset)
package netflix;
import com.typestruct.api.TypeStruct.Bind;
import com.typestruct.api.TypeStruct.Data;
import com.typestruct.api.TypeStruct.Data.Prototype;
import com.typestruct.bind.Field;
import com.typestruct.bind.struct.StructWord;
import com.typestruct.data.store.WordDataStore;
import example.type.DayIntervalType; //this is a "custom" user defined type
@typestruct
typestruct / NetflixPrizeTypeStructLoadFromFiles.java
Created November 7, 2014 13:58
typestruct loading Netflix Prize Training DataSet (100M records, 2Gb) into active memory
package netflix;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import com.typestruct.BaseException;
import com.typestruct.api.TypeStruct.Bind;
import com.typestruct.api.TypeStruct.Data;
@typestruct
typestruct / DayIntervalType.java
Created November 7, 2014 18:56
typestruct - custom type (DayIntervalType) explained
package example.type;
import java.text.ParseException;
import java.util.Arrays;
import java.util.Date;
import java.util.concurrent.TimeUnit;
import com.typestruct.bind.BinBinding;
import com.typestruct.bind.BindingException;
import com.typestruct.bind.MemberBinding;
@typestruct
typestruct / StructDefPrimitivesExample.java
Last active August 29, 2015 14:10
How to create a StructDef containing signed/unsigned/nullsafe primitives, create read, modify and describe packedStruct instances
package com.typestruct.api;
import com.typestruct.api.TypeStruct.Bind;
import com.typestruct.api.TypeStruct.Bind.Modifier;
import com.typestruct.struct.StructDef;
import com.typestruct.struct.StructField;
/**
* Example Illustrating the definition of a multi-word <CODE>StructDef</CODE> containing:
* <UL>
@typestruct
typestruct / StructDefPrimitivesExample_NoComments.java
Created December 4, 2014 13:18
How to build and use StructDef with primitive members
package com.typestruct.api;
import com.typestruct.api.TypeStruct.Bind;
import com.typestruct.api.TypeStruct.Bind.Modifier;
import com.typestruct.struct.StructDef;
public class StructDefPrimitivesExample_NoComments {
static StructDef struct = TypeStruct.def("Struct Primitives",
Bind.byte8("byteField"),
@typestruct
typestruct / DayIntervalType_NoComments.java
Last active August 29, 2015 14:10
Custom DayIntervalType used for NetflixPrizeTrainingDataSet (an interval where each increment is days)
package example.type;
import java.text.ParseException;
import java.util.Date;
import java.util.concurrent.TimeUnit;
import com.typestruct.bind.BaseBinBinding;
import com.typestruct.bind.BinBinding;
import com.typestruct.bind.BindingException;
import com.typestruct.bind.BindingRangeException;
@typestruct
typestruct / NetflixPrizeToStructWordStream.java
Last active August 29, 2015 14:10
uses StructWord and WordOuputStream converts (2gb) worth .txt data from the Netflix Prize Training DataSet to create a streamlined efficient 766 Mb binary
package netflix;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import com.typestruct.BaseException;
import com.typestruct.api.TypeStruct.Bind;
import com.typestruct.bind.Field;
@typestruct
typestruct / UnsignedPrimitives.java
Last active August 29, 2015 14:17
Unsigned primitive fields(uInt32, uShort16, uByte8) in a WordFrame (Usage)
package gist.basics;
import com.typestruct.field.Bind;
import com.typestruct.frame.Frame;
import com.typestruct.frame.word.WordFrame;
/**
* Example illustrating the use of unsigned primitive field bindings
* (uByte8, uShort16, uInt32) within a <CODE>WordFrame</CODE> to store
* and mutate a "compound state" as a 64-bit long.
@typestruct
typestruct / UnsignedPrimitivesBucketized.java
Created March 20, 2015 19:59
Using Type Binding "Bucketizers" wrapping Unsigned Primitive Field values to represent structured data (in a WordFrame)
package gist.basics;
import com.typestruct.field.Bind;
import com.typestruct.field.Bind.Modify;
import com.typestruct.frame.word.WordFrame;
/**
* Example illustrating the use of <B>Bucketizers</B> which
* are Type.Binding <B>Modifier</B>s applied to
* field bindings within a <CODE>WordFrame</CODE>.
@typestruct
typestruct / NetflixPrizeSynthesize.java
Last active August 29, 2015 14:17
Synthesize and use huge (100+ million record) dataset on-heap or off-heap using WordFrame and Synthesize
package gist.basics;
import com.typestruct.field.Bind;
import com.typestruct.frame.Frame.Field;
import com.typestruct.frame.word.WordFrame;
import com.typestruct.synthesize.Synthesize;
import com.typestruct.type.custom.DayInRangeType; //this is a "custom" user defined type
/**
* typestruct provides a simple API for representing, collecting, analyzing, and manipulating data in