Skip to content

Instantly share code, notes, and snippets.

@seumasmorrison
Created October 9, 2013 11:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save seumasmorrison/6899623 to your computer and use it in GitHub Desktop.
Save seumasmorrison/6899623 to your computer and use it in GitHub Desktop.
Valeport text file to Pandas pickle and Excel xlsx file
# -*- coding: utf-8 -*-
"""
Created on Wed Oct 09 09:58:44 2013
Read text files for MiniCTD exported from Valeport Datalog Express Scroll
Display, writes Excel xlsx and pickle file, creates pickle of main DataFrame
with another DataFrame as metadata attribute
@author: James Morrison
"""
import pandas as pd
from pandas import ExcelWriter
def valeport_txt_to_pickle_excel(text_file):
data_df = pd.read_csv(text_file, skiprows=9, delimiter='\t', index_col = 0,
parse_dates=True, usecols=[0,1,2,3,4,5,6],
names=['Date Time','Pressure (dBar)', 'Temperature (C)',
'Conductivity (MicroSiemens/Centimetre)', 'Salinity (PSU)',
'Density (KG/M3)', 'Sound Velocity (m/sec)'],
keep_date_col=False)
metadata = open(text_file).readlines()[:9]
metadata_dict = {}
for line in metadata:
metadata_dict[line.split(':', 1)[0]] = line.split(':', 1)[1][1:-1]
metadata_df = pd.DataFrame(metadata_dict, index=[0])
data_df.metadata = metadata_df
data_df.to_pickle(text_file[:-4])
writer = ExcelWriter(text_file[:-3] + 'xlsx')
data_df.to_excel(writer, 'Data')
data_df.metadata.to_excel(writer, 'Metadata')
writer.save()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment