ssomnath

N-Dimensional Spectroscopy and Imaging Data (NSID) Format

Preface

Several years ago, when I was a postdoctoral researcher at CNMS, a few fellow researchers and I explored multiple methods for representing measurement data. Long-story short, we decided to go ahead with what is now the Universal Spectroscopy and Imaging Data (USID) model where the data would be written into Heirarchical Data Format (HDF5) files. USID's ability to express all kinds of data, especially niche cases like compressed sensing, spiral scans (neither polar nor cartesian grid), etc., renders itself perhaps unnecessarily complicated for data which do have an N-dimensional form. When data does have a clear N-dimensional form, one could leverage a lot of HDF5's inherent capabilities to easily represent and use N-dimensional datasets. For this reason, I had explored the possibility of using USID only when an N-dimensional fo

ssomnath

import os
from warnings import warn
from functools import partial
import collections
import h5py
import numpy as np
import dask.array as da
from hyperspy.signals import BaseSignal, ComplexSignal
import pyUSID as usid

ssomnath

import os

def search_in_file(file_path, target_string, match_case=False, abs_paths=False):
    if not match_case:
        target_string = target_string.lower()
    _, file_name = os.path.split(file_path)
    with open(file_path) as myFile:
        try:
            for num, line in enumerate(myFile.readlines()):
                if not match_case:

ssomnath

from __future__ import print_function, division, unicode_literals
import os
import numpy as np
import h5py
import json
import sys

if sys.version_info.major == 3:
    unicode = str

ssomnath

import os
from collections import OrderedDict

def count_doc_lines(file_path, chars_per_line=120):
    non_blank_count = 0
    total_lines = 0
    with open(file_path) as file_handle:
        for line in file_handle:
            curr = 1
            if line.strip():