Last active
October 31, 2017 07:23
-
-
Save wtbarnes/1a9acc43381a33d016843c666150e8e9 to your computer and use it in GitHub Desktop.
Tutorial for parsing CHIANTI atomic data, building it into an HDF5 file, and accessing it
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sorry for not responding @kdere. I just saw this comment. For some reason GitHub didn't send me a notification or I missed it somehow...
PyTables may be a better way to go. It seems more full featured, but a bit of a steeper learning curve. I've just used h5py because I have experience with it and it is simpler. If some of the functionality I've built can be replaced or improved by PyTables, I'm all for switching over.
The increase in read speeds is one of my main motivations for doing this as well. Better to read the ASCII data once and then all subsequent reads are just from the HDF5 "database". Additionally, this means that you can easily slice into portions of a large file (e.g. some of the Fe .wgfa files) without having to load the whole thing into memory. I've found that ChiantiPy can be a bit sluggish for some of these ions with many, many transitions.
Though the format is HDF5, the format is essentially exactly the same as the ASCII database. There are groups for each element, subgroups for each ion, and then datasets for the individual files (.elvlc, .wgfa, etc.) within those subgroups.
If you have some time, play around with the package and let me know what you think (here or in an issue on the main package page).