Skip to content

Instantly share code, notes, and snippets.

@techedlaksh
Created March 21, 2017 23:07
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save techedlaksh/9001039bf54ba9d8aec3ad7f5d8bfd08 to your computer and use it in GitHub Desktop.
Save techedlaksh/9001039bf54ba9d8aec3ad7f5d8bfd08 to your computer and use it in GitHub Desktop.
Converting mat files to csv using python, scipy and pandas
import scipy.io
import pandas as pd
mat = scipy.io.loadmat('file.mat')
mat = {k:v for k, v in mat.items() if k[0] != '_'}
data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.iteritems()})
data.to_csv("example.csv")
@ioarun
Copy link

ioarun commented Sep 18, 2019

Executing the above code causes the following error:
Traceback (most recent call last): File "mat2csv.py", line 6, in <module> data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.iteritems()}) AttributeError: 'dict' object has no attribute 'iteritems'
Changed mat.iteritems() to mat.items() and then it works. (as Python 3 renamed iteritems to items)

@Shravankumarhiregoudar
Copy link

Shravankumarhiregoudar commented Jun 12, 2020

import scipy.io
import pandas as pd

mat = scipy.io.loadmat('file.mat')
mat = {k:v for k, v in mat.items() if k[0] != '_'}
data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
data.to_csv("example.csv")

This should work. use items() instead of iteritems()

Executing the above code causes the following error:
Traceback (most recent call last): File "mat2csv.py", line 6, in <module> data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.iteritems()}) AttributeError: 'dict' object has no attribute 'iteritems'
Changed mat.iteritems() to mat.items() and then it works. (as Python 3 renamed iteritems to items)

@tugbatasbasi
Copy link

hello I have 980 mat files they contain ecg records. I want to convert them to a single csv. What code should I write in python?

@hvsw
Copy link

hvsw commented Mar 14, 2021

I'm getting this error:

Exception                                 Traceback (most recent call last)
<ipython-input-87-aeb81113a03a> in <module>
      4 mat = scipy.io.loadmat('S1.mat')
      5 mat = {k:v for k, v in mat.items() if k[0] != '_'}
----> 6 data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
      7 data.to_csv("example.csv")

<ipython-input-87-aeb81113a03a> in <dictcomp>(.0)
      4 mat = scipy.io.loadmat('S1.mat')
      5 mat = {k:v for k, v in mat.items() if k[0] != '_'}
----> 6 data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
      7 data.to_csv("example.csv")

/opt/anaconda3/lib/python3.8/site-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    325                     data = data.copy()
    326             else:
--> 327                 data = sanitize_array(data, index, dtype, copy, raise_cast_failure=True)
    328 
    329                 data = SingleBlockManager.from_array(data, index)

/opt/anaconda3/lib/python3.8/site-packages/pandas/core/construction.py in sanitize_array(data, index, dtype, copy, raise_cast_failure)
    494     elif subarr.ndim > 1:
    495         if isinstance(data, np.ndarray):
--> 496             raise Exception("Data must be 1-dimensional")
    497         else:
    498             subarr = com.asarray_tuplesafe(data, dtype=dtype)

Exception: Data must be 1-dimensional

@Bhavya100
Copy link

hello I have 980 mat files they contain ecg records. I want to convert them to a single csv. What code should I write in python?

open a new csv file and then

  1. convertt one mat file to csv, write in the previously opened csv shift columns as per your mat file
  2. loop

@satvik27git
Copy link

Hello when i am trying to run this code the following error is raised because of line 6:
data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
File "C:\python\lib\site-packages\pandas\core\series.py", line 269, in init
raise ValueError(
ValueError: Cannot construct a Series from an ndarray with compound dtype. Use DataFrame instead.
How do i do this ?

@Laudarisd
Copy link

If you are still not able to solve the problem,
Check this code,
https://www.kaggle.com/vinayak123tyagi/mat-to-csv-code
(might not be perfect one)

This might be helpful for you.

In my opinion, you have to loop to dic items and itter throw them.
To save extracted data in csv format you have to make DataFrame( which currently you are getting as an error)

Thanks.

@MightyMirko
Copy link

MightyMirko commented Nov 24, 2021

Traceback (most recent call last):
  File "C:\Users\mirko\anaconda3\envs\sixSigma\lib\site-packages\IPython\core\interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-14-94978488e9fc>", line 11, in <module>
    mat = scipy.io.loadmat(f)
  File "C:\Users\mirko\anaconda3\envs\sixSigma\lib\site-packages\scipy\io\matlab\mio.py", line 225, in loadmat
    MR, _ = mat_reader_factory(f, **kwargs)
  File "C:\Users\mirko\anaconda3\envs\sixSigma\lib\site-packages\scipy\io\matlab\mio.py", line 74, in mat_reader_factory
    mjv, mnv = get_matfile_version(byte_stream)
  File "C:\Users\mirko\anaconda3\envs\sixSigma\lib\site-packages\scipy\io\matlab\miobase.py", line 212, in get_matfile_version
    mopt_bytes = fileobj.read(4)
  File "C:\Users\mirko\anaconda3\envs\sixSigma\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 204: character maps to <undefined>

loadmat won't work

@jaygupta-2k
Copy link

jaygupta-2k commented Jan 4, 2023

hello I have 980 mat files they contain ecg records. I want to convert them to a single csv. What code should I write in python?

did anyone find a proper solution for this?

@tugbatasbasi
Copy link

tugbatasbasi commented Jan 4, 2023 via email

@jaygupta-2k
Copy link

jaygupta-2k commented Jan 4, 2023 via email

@AmrMoursi1
Copy link

AmrMoursi1 commented Feb 15, 2023

hello I have 980 mat files they contain ecg records. I want to convert them to a single csv. What code should I write in python?

Hi, did you find a proper solution for this?

@AmrMoursi1
Copy link

AmrMoursi1 commented Feb 15, 2023

If yes, could you please post your solution here . thanks in advanced

@AmrMoursi1
Copy link

import scipy.io
import pandas as pd

mat = scipy.io.loadmat('file.mat')
mat = {k:v for k, v in mat.items() if k[0] != '_'}
data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
data.to_csv("example.csv")

This should work. use items() instead of iteritems()

Executing the above code causes the following error:
Traceback (most recent call last): File "mat2csv.py", line 6, in <module> data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.iteritems()}) AttributeError: 'dict' object has no attribute 'iteritems'
Changed mat.iteritems() to mat.items() and then it works. (as Python 3 renamed iteritems to items)

thanks for your explanation, it is working now but this code return only single row of data what should i do to get entire data in format of dataframe , thanks in advanc.

@ioarun
Copy link

ioarun commented May 23, 2023

import scipy.io
import pandas as pd

mat = scipy.io.loadmat('file.mat')
mat = {k:v for k, v in mat.items() if k[0] != '_'}
data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.items()})
data.to_csv("example.csv")

This should work. use items() instead of iteritems()

Executing the above code causes the following error:
Traceback (most recent call last): File "mat2csv.py", line 6, in <module> data = pd.DataFrame({k: pd.Series(v[0]) for k, v in mat.iteritems()}) AttributeError: 'dict' object has no attribute 'iteritems'
Changed mat.iteritems() to mat.items() and then it works. (as Python 3 renamed iteritems to items)

That's what I said.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment