Skip to content

Instantly share code, notes, and snippets.

View rvernica's full-sized avatar

rvernica

13 followers · 0 following
View GitHub Profile
@rvernica
rvernica / BLOB_Example.ipynb
Last active September 4, 2017 03:04
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / perf.py
Last active September 26, 2017 03:53
# AFL% set no fetch; store(apply(build(<x:int64>[i=1:1000; j=1:1000], i + j * 1000), y, 1e6 / double(i + 1), z, string(i)), df); set fetch;
# Query was executed successfully
# AFL% summarize(df);
# {inst,attid} att,count,bytes,chunks,min_count,avg_count,max_count,min_bytes,avg_bytes,max_bytes
# {0,0} 'all',1000000,8041133,4,1000000,1e+06,1000000,48,2.01028e+06,8000072
# In [17]: sys.getsizeof(df)/1024/1024.
# Out[17]: 87.70475769042969 MB
@rvernica
rvernica / arrow-feather-example.cpp
Created September 10, 2017 03:03
/*
g++-4.9 -std=c++11 arrow-feather-example.cpp -larrow
>>> import pandas
>>> pandas.read_feather('example.feather')
*/
#include <iostream>
#include <arrow/api.h>
#include <arrow/io/file.h>
@rvernica
rvernica / Log Output
Last active April 19, 2018 14:56
aio_tools benchmark
> iquery -aq "aio_save(build(<x:int64>[i=1:400:0:100], i), '/tmp/a', 'format=arrow', 'buffer_size=1000')"
{chunk_no,dest_instance_id,source_instance_id} val
> 0
2018-03-25 01:01:41.000158 [0x7fab10474700] [DEBUG]: ALT_SAVE>> Starting SG
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> opening file
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> starting write
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> builder.getTotalSize 1165
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> bytesCount 900
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> nCells 1
@rvernica
rvernica / read-batches.ipynb
Created February 14, 2018 04:27
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.