Skip to content

Instantly share code, notes, and snippets.

Avatar

Francesc Alted FrancescAlted

View GitHub Profile
View gist:203be8a44d02566f31dae11a22c179f3
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy as np
In [2]: import numexpr as ne
In [3]: import numba
@FrancescAlted
FrancescAlted / compare-evaluators.py
Created May 9, 2019
Comparison of different evaluation methods in IronArray: iterators and evaluators
View compare-evaluators.py
import iarray as ia
from time import time
import numpy as np
import numexpr as ne
from numba import jit
from itertools import zip_longest as zip
import py2llvm as llvm
from py2llvm import float64, int32, Array
@FrancescAlted
FrancescAlted / simple-key-store.py
Created Jan 14, 2016
A demonstration of a simple key-value store using numpy (.npz) and bcolz (ctable)
View simple-key-store.py
# Benchmark to compare the times for storing numpy arrays in a key-value.
# The main point is to compare numpy serialization vs a bcolz approach.
from __future__ import print_function
import sys
import os
import os.path
import subprocess
import getopt
View optimal-blocksize
{
"metadata": {
"name": "",
"signature": "sha256:82e5b24036955db348e69e8d773df9940f1cc361f8dff79db3f0b5602fd6cf67"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
@FrancescAlted
FrancescAlted / blocking
Created Apr 17, 2014
Study of blocking in numpy/numexpr
View blocking
{
"metadata": {
"name": "",
"signature": "sha256:5a6759998dc5cecdfc269873ed0afe13bceff1f78f0178535b0e07efc6161892"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
@FrancescAlted
FrancescAlted / ngrams
Created Apr 13, 2014
Example of how to find substrings in large datasets new 'contains' in numexpr
View ngrams
{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
@FrancescAlted
FrancescAlted / MSFT-Dataset-Query-Complex.ipynb
Created Feb 26, 2014
A comparison of BLZ with Pandas DataFrames
View MSFT-Dataset-Query-Complex.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@FrancescAlted
FrancescAlted / gist:7926895
Created Dec 12, 2013
Why on hell Python 2.7 (and even 3.3) does not release memory as it should?
View gist:7926895
Using 2.7.6:
$ python2 ~/mem_prof.py
memory deallocated: 0.0 MB
Using 3.4b1:
$ python3 ~/mem_prof.py
memory deallocated: -1040.09765625 MB
@FrancescAlted
FrancescAlted / gist:4595225
Last active Apr 22, 2018
A comparison of variable length storage speed of Blaze vs PyTables
View gist:4595225
"""
Benchmark that compares the storing of objects in both Blaze and PyTables
"""
import os.path
import shutil
from time import time
import blaze
import tables
@FrancescAlted
FrancescAlted / Query2.ipnb
Created Oct 24, 2012
Query2 for PyTables tutorial
View Query2.ipnb
{
"metadata": {
"name": "Queries2"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{