Tom Kooij tomkooij
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <meta charset="utf-8"> | |
| <html> | |
| <head> | |
| <script src='http://d3js.org/d3.v3.min.js'></script> | |
| </head> | |
| <style> | |
| .ditwaseerst_dotbar { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| *** testcase: 1e16 eV 88Mb unsorted | |
| N = 10000 | |
| old code pytables_readwhere: | |
| Time: 4:36:13|############################################################################################################################################|100% | |
| number of coincidences: 5055 | |
| new code: pytbales_readwhere on x. compress on y and particle_id | |
| Time: 5:35:23|############################################################################################################################################|100% | |
| number of coincidences: 5055 | |
| results: |
tomkooij
/ test_cq.py
Created
March 15, 2016 13:04
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding: utf-8 | |
| from sapphire import CorsikaQuery | |
| cq = CorsikaQuery('/data/hisparc/corsika/corsika_overview.h5') | |
| cq.available_parameters('energy') | |
| cq.simulations(energy=16.5) |
tomkooij
/ test_read_where.py
Last active
March 29, 2016 07:55
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding: utf-8 | |
| import tables | |
| print "tables version: ", tables.__version__ | |
| import numexpr | |
| print "numexpr version: ", numexpr.__version__ | |
| FILENAME = '/data/hisparc/corsika/corsika_overview.h5' | |
| data = tables.open_file(FILENAME, 'r') | |
| a = data.root.simulations.read_where('log10(energy) > 14.') | |
| print len(a) |
tomkooij
/ test_NumExpr.py
Last active
March 29, 2016 19:26
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding: utf-8 | |
| import numpy | |
| from hashlib import md5 | |
| from tables.utilsExtension import get_nested_field | |
| import numexpr | |
| from numexpr.necompiler import NumExpr, stringToExpression | |
| import tables | |
| FILENAME = '/data/hisparc/corsika/corsika_overview.h5' |
tomkooij
/ test_recarray.py
Last active
March 30, 2016 13:49
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Simulate a read_where() query on a pytables table | |
| Issue: Using MKL 11.3 the numexpr NumExpr query is wrong | |
| works with without MKL. | |
| works with and without MKL using a single column. | |
| It seems the VML log10 function is not correctly called. | |
| The table has different memory mapping as a single column. | |
| It seems the VML call does not account for this correctly. |
tomkooij
/ test_vml_read_where.py
Last active
March 31, 2016 20:50
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import tables | |
| import numexpr | |
| FILENAME = 'test.h5' | |
| class Test(tables.IsDescription): | |
| a = tables.Float32Col(dflt=1, pos = 2) # single-precision | |
| b = tables.Float32Col(dflt=1, pos = 2) # single-precision | |
| c = tables.Float32Col(dflt=1, pos = 2) # single-precision |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ python -m tables.tests.test_all --heavy | |
| *Warning*: NumPy version is lower than recommended: 1.10.4 < 1.7.1 | |
| -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= | |
| PyTables version: 3.2.2 | |
| HDF5 version: 1.8.16 | |
| NumPy version: 1.10.4 | |
| Numexpr version: 2.5.2.dev0 (using VML/MKL 11.3.1) | |
| Zlib version: 1.2.8 (in Python interpreter) | |
| BZIP2 version: 1.0.5 (10-Dec-2007) | |
| Blosc version: 1.4.4 (2015-05-05) |
tomkooij
/ profile.txt
Created
April 12, 2016 09:14
cProfile (cumulative) of DAT000000 to HDF5 conversion
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| convert to hdf5 | |
| Converting CORSIKA data (DAT000000) to HDF5 format | |
| 100%|##################################################################################################################################################|Time: 0:02:40 | |
| create index | |
| Ensuring the x column for table groundparticles is indexed. | |
| 106369062 function calls (106367147 primitive calls) in 180.480 seconds | |
| Ordered by: cumulative time | |
| ncalls tottime percall cumtime percall filename:lineno(function) |
OlderNewer