Skip to content

Instantly share code, notes, and snippets.

View read-batches.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / Log Output
Last active Apr 19, 2018
aio_tools benchmark
View Log Output
> iquery -aq "aio_save(build(<x:int64>[i=1:400:0:100], i), '/tmp/a', 'format=arrow', 'buffer_size=1000')"
{chunk_no,dest_instance_id,source_instance_id} val
> 0
2018-03-25 01:01:41.000158 [0x7fab10474700] [DEBUG]: ALT_SAVE>> Starting SG
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> opening file
2018-03-25 01:01:41.000159 [0x7fab10474700] [DEBUG]: ALT_SAVE>> starting write
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> builder.getTotalSize 1165
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> bytesCount 900
2018-03-25 01:01:41.000159 [0x7fab10272700] [DEBUG]: ALT_SAVE>> nCells 1
View arrow-feather-example.cpp
/*
g++-4.9 -std=c++11 arrow-feather-example.cpp -larrow
>>> import pandas
>>> pandas.read_feather('example.feather')
*/
#include <iostream>
#include <arrow/api.h>
#include <arrow/io/file.h>
View perf.py
# AFL% set no fetch; store(apply(build(<x:int64>[i=1:1000; j=1:1000], i + j * 1000), y, 1e6 / double(i + 1), z, string(i)), df); set fetch;
# Query was executed successfully
# AFL% summarize(df);
# {inst,attid} att,count,bytes,chunks,min_count,avg_count,max_count,min_bytes,avg_bytes,max_bytes
# {0,0} 'all',1000000,8041133,4,1000000,1e+06,1000000,48,2.01028e+06,8000072
# In [17]: sys.getsizeof(df)/1024/1024.
# Out[17]: 87.70475769042969 MB
View BLOB_Example.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View DownloadArray.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View SciDB-variant.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@rvernica
rvernica / iquery.out
Last active Mar 10, 2017
SciDB-Py to NumPy or Pandas Export
View iquery.out
{i,j,k} b,c,dt,dtz,d,f,i8,i16,i32,i64,s,u8,u16,u32,u64
{0,-2,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-2,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-2,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,-1,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,0} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,1} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,16,32,64
{0,0,2} true,'a','1970-01-01 00:00:00','1970-01-01 00:00:00 -00:00',0.2,0.1,-8,-16,-32,-64,'abc',8,
View run.py
#!/usr/bin/python
# Initialize, start and stop scidb in a cluster.
# BEGIN_COPYRIGHT
#
# Copyright (C) 2008-2016 SciDB, Inc.
# All Rights Reserved.
#
# This file is part of the Paradigm4 Enterprise SciDB distribution kit
View CMakeLists.txt
########################################
# BEGIN_COPYRIGHT
#
# Copyright (C) 2008-2016 SciDB, Inc.
# All Rights Reserved.
#
# This file is part of the Paradigm4 Enterprise SciDB distribution kit
# and may only be used with a valid Paradigm4 contract and in accord
# with the terms and conditions specified by that contract.
#
You can’t perform that action at this time.