Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

View rvernica's full-sized avatar

rvernica

13 followers · 0 following
View GitHub Profile
@rvernica
rvernica / read-batches.ipynb
Created February 14, 2018 04:27
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / Log Output
Last active April 19, 2018 14:56
aio_tools benchmark
> iquery -aq "aio_save(build(<x:int64>[i=1:400:0:100], i), '/tmp/a', 'format=arrow', 'buffer_size=1000')"
{chunk_no,dest_instance_id,source_instance_id} val
> 0
2018-03-25 01:01:41.000158 [0x7fab10474700] [DEBUG]: ALT_SAVE>> Starting SG
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> opening file
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> starting write
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> builder.getTotalSize 1165
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> bytesCount 900
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> nCells 1
@rvernica
rvernica / arrow-feather-example.cpp
Created September 10, 2017 03:03
/*
g++-4.9 -std=c++11 arrow-feather-example.cpp -larrow
>>> import pandas
>>> pandas.read_feather('example.feather')
*/
#include <iostream>
#include <arrow/api.h>
#include <arrow/io/file.h>
@rvernica
rvernica / perf.py
Last active September 26, 2017 03:53
# AFL% set no fetch; store(apply(build(<x:int64>[i=1:1000; j=1:1000], i + j * 1000), y, 1e6 / double(i + 1), z, string(i)), df); set fetch;
# Query was executed successfully
# AFL% summarize(df);
# {inst,attid} att,count,bytes,chunks,min_count,avg_count,max_count,min_bytes,avg_bytes,max_bytes
# {0,0} 'all',1000000,8041133,4,1000000,1e+06,1000000,48,2.01028e+06,8000072
# In [17]: sys.getsizeof(df)/1024/1024.
# Out[17]: 87.70475769042969 MB
@rvernica
rvernica / BLOB_Example.ipynb
Last active September 4, 2017 03:04
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / DownloadArray.ipynb
Last active April 2, 2017 01:51
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / SciDB-variant.ipynb
Created March 25, 2017 04:52
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / iquery.out
Last active March 10, 2017 05:40
SciDB-Py to NumPy or Pandas Export
{i,j,k} b,c,dt,dtz,d,f,i8,i16,i32,i64,s,u8,u16,u32,u64
{0,-2,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-2,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-2,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,
@rvernica
rvernica / run.py
Last active December 6, 2016 20:37
SciDB 16.9 issue https://github.com/Paradigm4/SciDB/issues/8
#!/usr/bin/python
# Initialize, start and stop scidb in a cluster.
# BEGIN_COPYRIGHT
#
# Copyright (C) 2008-2016 SciDB, Inc.
# All Rights Reserved.
#
# This file is part of the Paradigm4 Enterprise SciDB distribution kit
@rvernica
rvernica / CMakeLists.txt
Last active December 6, 2016 17:53
SciDB 16.9 issue https://github.com/Paradigm4/SciDB/issues/7
########################################
# BEGIN_COPYRIGHT
#
# Copyright (C) 2008-2016 SciDB, Inc.
# All Rights Reserved.
#
# This file is part of the Paradigm4 Enterprise SciDB distribution kit
# and may only be used with a valid Paradigm4 contract and in accord
# with the terms and conditions specified by that contract.
#