Skip to content

Instantly share code, notes, and snippets.

@simonmorley
Created June 12, 2014 09:05
Show Gist options
  • Save simonmorley/8dd81b1456d1d366dcff to your computer and use it in GitHub Desktop.
Save simonmorley/8dd81b1456d1d366dcff to your computer and use it in GitHub Desktop.
Sample JSON
{ "ap_mac" : "00:18:0a:xx:xx:xx", "rssi" : 0, "is_associated" : true, "client_mac" : "xx:xx:xx:xx:xx:xx", "last_seen" : { "$date" : 1396571076936 }, "secret" : "xxx", "created_at" : "1396571075209", "_id" : { "$oid" : "533dfbc3fca8c126209a3ce0" }, "__v" : 0 }
@simonmorley
Copy link
Author

last_seen needs reformatting.

ID should be mapped on import. Currently _id doesn't import.

@mfelsche
Copy link

yes, you have to transform that thing.
created_at should become a number, not a string, otherwise the timestamp semantics won't work.
yes, you have to extract the $oid thing from the id field and the $date thing from the last_seen field.

I would suggest using the java client, issuing bulk insert statements (insert into t (id, col) values (1, 'hfjg'), (2, 'some more'), (...)) via the transport. This is the fastest thing you can do besided using copy to. And you would get correct results. Maybe parallelize this file reading, request issueing stuff, split up your dump, you know...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment