Skip to content

Instantly share code, notes, and snippets.

@bitmorse
Last active May 1, 2024 14:12
Show Gist options
  • Save bitmorse/80dedd93290ec8b227d6e7f6539ba717 to your computer and use it in GitHub Desktop.
Save bitmorse/80dedd93290ec8b227d6e7f6539ba717 to your computer and use it in GitHub Desktop.
How to record data from an STWIN.box (STEVAL-STWINBX1) on a Raspberry Pi via USB and convert the resulting .dat files to Pandas df or numpy arrays.
'''
# README FIRST #
###Goal###
On a Raspberry Pi, read .dat files created by STWIN.box from disk and get numpy or pandas objects to work with.
### Requirements ###
- Raspberry Pi
- ST-Link Debugger
- STWIN.box + USB-C Cable
### Guide ###
1. On your computer: Flash [this firmware](https://github.com/STMicroelectronics/fp-sns-datalog2/blob/bb0a1d7560b5c459908b112e091f42f284670259/Projects/STM32U585AI-STWIN.box/Applications/DATALOG2) to your STWIN.box with an ST-Link.
2. On Raspberry Pi: To stream data from the STWIN.box connected to your Raspberry Pi, follow [this guide](https://github.com/STMicroelectronics/fp-sns-datalog2/blob/bb0a1d7560b5c459908b112e091f42f284670259/Utilities/cli_example/README_Linux.md) to setup and run the recording executable.
3a) on bookworm 64bit, install libusb-1.0-0:armhf, libc6:armhf and libstdc++6:armhf as the compiled cli_example binary needs ld-linux-armhf.so.3
3b) On Raspberry Pi: Record some data with the ´cli_example´ tool. The ´cli_example´ executable will create a folder with a bunch of ´.dat´ files containing high resolution raw sensor data. The next steps will show how to convert the data to a dataframe.
4. On Raspberry Pi: To convert the recorded data, first install depencies: sudo apt-get install python3-dev libatlas-base-dev libopenjp2-7 libopenblas-dev
5. On Raspberry Pi: Install the ST HSDataLog libraries:
wget https://github.com/STMicroelectronics/fp-sns-datalog2/raw/bb0a1d7560b5c459908b112e091f42f284670259/Utilities/HSDPython_SDK/st_pnpl/dist/st_pnpl-2.1.0-py3-none-any.whl
pip install st_pnpl-2.1.0-py3-none-any.whl
wget https://github.com/STMicroelectronics/fp-sns-datalog2/raw/bb0a1d7560b5c459908b112e091f42f284670259/Utilities/HSDPython_SDK/st_hsdatalog/dist/noGUI/st_hsdatalog-3.0.1-py3-none-any.whl
pip install st_hsdatalog-3.0.1-py3-none-any.whl
6. On Raspberry Pi: Copy and run this script to see if conversion works. Modify it to your needs. On RP CM4, a 5 second recording (1.8M if gzipped) will take about 3.4s (numpy) or 9s (pandas) to convert all .dat files. For a 5min recording, it takes 20s (numpy).
(venv) sam@aikit:~ $ time python3 convert_dat.py 20240111_08_31_50/ all
2024-01-11 09:31:49,124 - HSDatalogApp.st_pnpl.DTDL.device_template_manager - INFO - dtmi found in locally in base supported models
2024-01-11 09:31:49,124 - HSDatalogApp.st_pnpl.DTDL.device_template_manager - INFO - dtmi: dtmi/appconfig/steval_stwinbx1/FP_SNS_DATALOG2_Datalog2-4.json
stts22h_temp, shape: (800, 1)
iis2mdc_mag, shape: (400, 3)
iis2iclx_acc, shape: (4000, 2)
ism330dhcx_acc, shape: (34000, 3)
imp23absu_mic, shape: (934000, 1) <--- this is an 80khz mic
iis2dlpc_acc, shape: (7000, 3)
ism330dhcx_gyro, shape: (34000, 3)
iis3dwb_acc, shape: (130000, 3) <--- this is an ultra wide band accelerometer
imp34dt05_mic, shape: (231000, 1)
ilps22qs_press, shape: (600, 1)
real 0m3.370s
user 0m3.917s
sys 0m0.257s
'''
import os
import sys
import st_hsdatalog.HSD_utils.converters as converters
from st_hsdatalog.HSD.HSDatalog import HSDatalog
def get_hsd(acq_folder='.'):
hsd_factory = HSDatalog()
hsd= hsd_factory.create_hsd(acq_folder)
return hsd
def dat_to_df(hsd, sensor_name):
'''
Converts a dat file to a pandas dataframe.
@param acq_folder: path to the acquisition folder
@param sensor_name: name of the sensor to convert (see filenames in the acquisition folder)
'''
sensor = HSDatalog.get_sensor(hsd, sensor_name)
#data_time = HSDatalog.get_dataframe(hsd, sensor)
data_time = HSDatalog.get_data_and_timestamps(hsd, sensor)
return sensor, data_time
if __name__ == '__main__':
# get the path to the acquisition folder
acq_folder = sys.argv[1]
sensor_name = sys.argv[2]
hsd = get_hsd(acq_folder)
if sensor_name == 'all':
#loop over all .dat files in folder
for filename in os.listdir(acq_folder):
if filename.endswith('.dat'):
sensor_name = filename.split('.')[0]
sensor, data_time = dat_to_df(hsd, sensor_name)
print("%s, shape: %s"% (sensor_name, data_time[0][0].shape))
else:
sensor, data_time = dat_to_df(hsd, sensor_name)
print(data_time)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment