castle055/gsoc25.md

## gsoc25.md

      
    Raw
  

              gsoc25.md
            
          
    Google Summer of Code 2025 Final Report

GSoC Proposal: GNSS+INS Sensor Fusion

Contributor: Víctor Castillo Agüero

Organization: GNSS-SDR

Mentor: Miguel Ángel Gómez López

Description

Sensor fusion has proven highly effective in improving GNSS precision and accuracy. Traditionally, these advances have relied on hardware receivers, where development is often slow and expensive. The goal of this project was to extend GNSS-SDR, an open-source software-defined GNSS receiver, so it can process external sensor measurements in a flexible and configurable way. This opens the door for future software-defined sensor fusion algorithms, making it easier to design, test, and iterate different receiver architectures.
Instead of connecting to specific hardware devices, this project focused on handling sensor measurements internally. Data is read from binary files with configurable formats and attached to the RF sample stream using GNU Radio’s stream tags. The priority was to synchronize these measurements with the RF sample stream and make them accessible to any DSP block.
At this stage, only pre-recorded sensor measurement files are supported. Real-time input from hardware sensors is left for future development.

Work Done

The first step was to create the domain abstractions needed in order for GNSS-SDR to be able to represent and manipulate sensor measurements from multiple origins and data types. This includes identifying which sensor the measurements belong to (SensorIdentifier) as well as the data types used to represent it in the source file and internally in the GNSS receiver flow graph (SensorDataType).
SensorDataSource reads sensor measurements from a binary file and parses them into a usable internal representation. This is highly configurable in the GNSS-SDR configuration file. The SensorDataSourceConfiguration class implements the logic for parsing all relevant fields as well as validating them.
Sensor measurements are then attached to the RF sample stream using GNU Radio's stream tags. These are lightweight metadata items that travel along the stream and can be retrieved later by any GNU Radio block with minimal coupling with their source. Some block (like Tracking) can rely upon GNU Radio to properly propagate stream tags to their outputs automatically. However, this is not always the case. Three blocks had to be modified to correctly handle sensor data tags. Fortunately, these blocks are either not polymorphic or have few implementations, so these modifications were not too extensive.

Signal Conditioner: Sensor data tags are synchronized using timestamps stored within each tag. This value needs to be updated in the Resampler so that it reflects any interpolation or decimation being done to the RF sample stream.
Sample Counter: The purpose of this block is to synchronize the output of the Observables block with the global receiver timestamp rather than relying in the asynchronous output from each channel. This is the ideal path for sensor measurements to take as they remain synchronized with the global timestamp as well. However, due to implementation details, this block does not propagate tags automatically, so manual propagation logic had to be added in order to handle sensor tags.
Observables: Due to the shorter path through the Sample Counter, sensor measurements arrive to the Observables block much earlier than the corresponding observable data from the channels. A buffer was added to apply this delay correctly, synchronizing the sensor measurement stream with the GNSS stream again for the PVT block.

No modifications were done to any Acquisition, Tracking or Telemetry blocks. These blocks all have access to sensor measurements shall they need them, but they are not responsible for propagating them to the Observables block so no change was required.
Any DSP block that needs access to these tags will make use of the SensorDataAggregator class. This class offers an interface for aggregating measurements from a list of stream tags and accessing them in a variety of ways (get last, get average or iteration for each sensor). The DSP block does not need to know how sensor measurements are stored within each tag, this is an implementation detail.
Additionally, the SensorDataAggregator class can be used by DSP blocks to require certain sensors to be provided if configured to do so. For instance, if a block is configured to use inertial velocity measurements, but they are not provided in the configuration file, and exception will be thrown on start up with an appropriate error message specifying which sensors are missing. The idea being to fail early in case of invalid configuration.
Separating the three concerns of data representation (SensorIdentifier and SensorDataType), data source (SensorDataSource) and data processing (SensorDataAggregator) simplifies integration and future extension.

Pull Requests


#894 [Merged]
#916 [Merged]


How to Use

From the user's perspective, some GNSS-SDR algorithms (such as the Kalman filter in the PVT block) may be configured to work with sensor data (PVT.kf_use_imu_vel = true). Only in such cases, GNSS-SDR will require certain sensors to be provided. If they are not provided, an exception is thrown since the requested algorithm cannot work without them.
In the Kalman filter example, the required measurements are the velocities in the three axis. These can be provided by adding the following SensorData section to the configuration file:
;######### SENSOR DATA CONFIG ############  
SensorData.enabled=true               ; Enable sensor data source

; Configure sensor data file
SensorData.file0.filename=/path/to/sensor/data/file.dat  
SensorData.file0.repeat=true          ; Wrap around once reached EOF
SensorData.file0.chunk_size=24        ; Chunk size [bytes], in this example: 3 values, 8 bytes wide each
SensorData.file0.file_offset=0        ; Start reading file from this offset [bytes]
SensorData.file0.sample_offset=0      ; RF samples to wait before reading any sensor data
SensorData.file0.sample_period=300000 ; RF samples in between sensor samples

; Configure sensors
SensorData.sensor0.data=IMU_VEL_X     ; Sensor identifier
SensorData.sensor0.type=f64           ; In-file data type
SensorData.sensor0.file=0             ; File ID
SensorData.sensor0.offset=0           ; Value offset in chunk

SensorData.sensor1.data=IMU_VEL_Y  
SensorData.sensor1.type=f64
SensorData.sensor1.file=0
SensorData.sensor1.offset=8

SensorData.sensor2.data=IMU_VEL_Z
SensorData.sensor2.type=f64
SensorData.sensor2.file=0
SensorData.sensor2.offset=16
This can be simplified taking into account the following behavior:

All sensors except the first one default their data type and file ID to those of the previous sensor. The first sensor defaults to a file ID equal to 0.
The first sensor's offset defaults to 0. All other sensors default to the result of adding the offset of the previous sensor and its size.

If the measurements all are of the same type and are stored contiguously in file '0', we can write this to the same effect:
...

SensorData.sensor0.data=IMU_VEL_X
SensorData.sensor0.type=f64
SensorData.sensor1.data=IMU_VEL_Y  
SensorData.sensor2.data=IMU_VEL_Z
Sensor ordering is determined by their IDs, which must be unique and contiguous starting from 0. File IDs have these same requirements, but there are no automatic defaults. All fields must be specified for each file.
Finally, we can tell the PVT block to use these values in the Kalman filter instead of velocity estimates from GNSS:
;######### PVT CONFIG ############  
...

PVT.enable_pvt_kf=true
PVT.kf_use_imu_vel=true
Note that the current implementation for integrating IMU velocities into the Kalman filter is just a proof of concept for the underlying sensor data infrastructure. Further refinement to this specific algorithm is needed for proper sensor fusion.
Sensor data types

Sensor measurements are propagated throughout the flowgraph using the polymorphic type pmt::pmt_t. This allows for sensors to work with whatever type best represents their measurements.
Each supported sensor has a specified internal data type. GNSS-SDR will convert sensor measurements from their in-file binary representation to their respective internal type. This means neither the user nor the implementer need to worry about type conversion. The user specifies which type the sensor uses in the sensor data file. Meanwhile, the implementer will always receive the values as converted to their internal types. For instance, the user can specify inertial measurements of velocity to be read from the sensor data file using any scalar type (F32, F64, I32 or I64), but the implementer will always receive them as F32.
Internal types are specified in SensorIdentifier::get_internal_type(). Types are converted using the conversion table at conversion_table. Modification of supported types or assigned internal types implies modification of this table. An exception is thrown if the required conversion is not found.
Using Sensor Measurements

From the implementer's perspective, any GNSS-SDR block implementation after the signal source can access sensor measurements by making use of the class SensorDataAggregator.
This is done inside the GNU Radio block implementation. A SensorDataAggregator is added as a member field to the class. It gets initialized in the constructor as follows:
// `configuration` is a reference to an instance of `ConfigurationInterface`
SensorDataSourceConfiguration sensor_config{configuration};

// Conditionally require certain sensors
std::vector<SensorIdentifier::value_type> required_sensors{};
if (conf_.kf_use_imu_vel)
    {
        required_sensors.emplace_back(SensorIdentifier::IMU_VEL_X);
        required_sensors.emplace_back(SensorIdentifier::IMU_VEL_Y);
        required_sensors.emplace_back(SensorIdentifier::IMU_VEL_Z);
    }

// Initialize aggregator
d_sensor_data_aggregator = std::make_unique<SensorDataAggregator>(sensor_config, required_sensors);
SensorDataAggregator validates the required sensor list against the provided configuration during construction. If a required sensor is not provided, an exception is thrown with an appropriate error message.
Then, in the work() function:
// Get relevant stream tags
std::vector<gr::tag_t> sensor_tags;
this->get_tags_in_range(
  sensor_tags,
  0,
  this->nitems_read(0),
  this->nitems_read(0) + noutput_items,
  d_sensor_data_aggregator->SENSOR_DATA_TAG
);

// Update the aggregator
d_sensor_data_aggregator->update(sensor_tags);

// Get sensor measurements
SensorDataSample<double> vx = d_sensor_data_aggregator.get_last_f64(SensorIdentifier::IMU_VEL_X);
vx.rf_sample_stamp;    // Measurement was taken at the same time as this RF sample
vx.value;              // Measurement value

// It is possible to iterate over all new samples in the last update.
// This will include the last sample from the previous update as well, 
//  so this list will never be empty after the first sensor sample.
for (const auto& vx : sensor_data_aggregator.get_f64(SensorIdentifier::IMU_VEL_X)) {
  vx.rf_sample_stamp;
  vx.value;
}


// Get average value of all sensor measurements in the current update cycle
SensorDataSample<double> vx = d_sensor_data_aggregator.get_average_f64(SensorIdentifier::IMU_VEL_X);

Future Work

The features implemented in this project are not meant as a complete sensor fusion implementation, but rather open the door for future developments. The aim was to provide a framework for working with sensor measurements which can be easily extended to support any and all sensor fusion algorithms. The sensor data infrastructure handles all aspects relative to data parsing, conversion, synchronization and aggregation, as well as configuration and its validation, allowing for any DSP block in the receiver flow graph to access and use it whenever needed.
Future work in the sensor data suite may involve the following:
Adding more sensors

New algorithms may require measurements from sensors other than those already implemented (Barometers, LiDAR, cameras, etc...). New sensors can be added as follows:

Add the new identifier in SensorIdentifier::value_type.
Add the corresponding cases in the following functions:

SensorIdentifier::from_string
SensorIdentifier::to_string
SensorIdentifier::get_internal_type


If required, add new entries to the conversion table in sensor_identifier.cc

Adding more sensor data types

Currently, only 5 data types are supported for sensor measurements (UINT64, F32, F64, I32 and I64, all little endian). It is rare for a sensor to measure unsigned values but the UINT64 type is needed internally in order to handle timestamps, which are measured in samples.
These 5 types should cover most use cases where the sensor measurements are real numbers. However, new types can be added if needed (complex values, images, ...).

Add the new type in SensorDataType::value_type.
Add the corresponding cases in the following functions:

SensorDataType::from_string
SensorDataType::to_string
SensorDataType::get_size
SensorDataType::make_value


If required, add new entries to the conversion table in sensor_identifier.cc
The SensorDataAggregator class holds a map for each data type used as an internal type. If a new type is added and it is intended to be used as the internal representation for a sensor, a new map must be added and properly managed.

Supporting real-time sensor measurements

At this stage, it is only supported to read sensor measurements from pre-recorded binary files. However, the underlying infrastructure regarding sample synchronization and aggregation for processing is independent from where sensor measurements are coming from. It is possible to extend the SensorDataSource class so it can interface with a variety of hardware sensors in real time. However, this involves writing adapters and drivers for each sensor model/brand, which is extensive work that may be done progressively as the need arises.

Acknowledgements

This project would not have been possible without the support of my mentor Miguel, whose guidance and expertise were essential in achieving this project's goals. I would also like to thank the GNSS-SDR organization and community for their support, their insight and code reviews were key factors throughout this project. This was a wonderful opportunity to be a part of an open-source community and experience the process of building free and open source by collaboration.
No results found