This project is part of the GSoC Google Summer of Code 2020 program and sponsored by Google Inc. . The International Neuroinformatics Coordinating Facility (INCF) is supporting the project. The mentors are Dr. John Griffiths (JohnGriffiths) and Dr. Rick Gerkin (rgerkin).
A GitHub repository was made for the project. Link to the repository and the commits:
- https://github.com/GriffithsLab/fooof-unit
- https://github.com/GriffithsLab/fooof-unit/commits/master
The SciUnit framework was developed to help researchers create unit tests for scientific models. Currently, unit tests exist for models of single neurons and small networks thereof. However, unit tests for models concerned with large-scale brain network dynamics, such as meso-scale, mean-field descriptions and corticothalamic circuit models have not been developed yet. During the summer, the aim was to develop tests to validate predictions from corticothalamic neural mass models against relevant features of EEG data. More specifically, by extracting features from the generated neural power spectrum.
An important part of the project consisted in parametrizing the neural power spectrum to obtain accurate results for testing. From the parametrized power spectrum three unit tests were developed:
CentralPeak: Assesses wether a peak is present within a specific frequency rangeBandPowerandPeakPower: Compares the band and peak power of a model against a reference observationCorrelation2Spectrum: Gives the correlation coefficient between two models
All these different tests enable the validation of the model by ensuring it produces brain waves of interest. I also worked on developing a code producing a database for three Single-Node models generated with the Virtual Brain platform (TVB) to present a use case of the package developed.
FOOOF is a tool parametrizing neural power spectra by breaking it into an aperiodic component reflecting 1/f characteristics and a number of periodic components as a set of peaks parametrized as gaussian. The FOOOF tool also gives the option to generate power spectrum with aperiodic and periodic components of our choice for testing purposes.
In each test the neural power spectrum of the model is parametrized with FOOOF as following:
fm = FOOOF()
fm.fit(prediction['freqs'], prediction['powers'], prediction['freq_range'])From the fm object created, FOOOF functions can be used to acquire information of interest such as fm.get_params('peak_params', 'CF'), which gives the central frequency of each detected peaks.
The fooofunit folder contains all the functions and SciUnit classes that are part of the new package created.
The project followed the general steps described in the SciUnit documentation. SciUnit is based on the design, documentation, and implementation of classes in Python. Four different types of classes have been written: capabilities, models, scores and tests. These classes can all be found in their corresponding sub folders in fooofunit.
First the capability class is created which describes the methods a model must implement in order to take the test. In our case, the class is ProducesPowerSpectrum as the only requirement to take the test is that the model generates a neural power spectrum similar to a signal output generated with corticothalamic models.
The model class NeuralPowerSpectra implements the capability. It is instantiated with the frequency, power values and frequency range of the neural power spectrum of the model of interest. Another model class was also made to create a frozen data trace of empirical neural power spectrum data.
As mentioned previously, there are three main test classes. Each one corresponds to a different type of usage of SciUnit. Within each test the neural power spectrum is parametrized with FOOOF. CentralPeak test correspond to a model versus feature unit test. This means that a specific feature is tested for the model. CentralPeak uses the FOOOF function get_params to obtain the central frequency and the height of the detected peaks. This allows use to determine if an important peak is present within a specific range. Therefore, this test is instantiated with the frequency band of interest. For example, if the user wishes to know if the spectrum has a peak in the alpha range, the range [8, 13] corresponds to the band input. There is also an option to pass a minimum peak height value in order to only take the most significant peaks into account. The final result is a boolean score.
The next tests included in Band_by_Band have two different options: BandPower and PeakPower. These are a model versus empirical data unit tests. In this case, two tests compare a model neural power spectrum against an empirical data. In PeakPower, the power of detected peaks is compared whereas in BandPower the power of frequency band ranges are determined. The result is a float score where a smaller value implies a greater similarity between the model and the data.
Finally, Correlation2Spectrum compares two model neural power spectra by computing the correlation coefficient. This corresponds to a model versus model unit test. This test is useful for testing models against a reference model. For this test a SciUnit Correlation Coeffcient score class was created. This score has also been added to the SciUnit score library (Link Pull Request). The result is a correlation coefficient where a correlation of -1.0 shows a perfect negative correlation, while a correlation of 1.0 shows a perfect positive correlation. A correlation of 0.0 shows no linear relationship between the movement of the two variables.
In addition to the classes several functions were written to facilitate the implementation of the code. In utils.py the function loads common brain frequency ranges such as theta, alpha, beta and gamma. Functions in io.py load and visualize MATLAB empirical MEG power spectrum. Furthermore, welch_psd.py has a function which computes the power spectrum of simulated signal using Welch's method. In our case, this function is used to obtain the PSD of TVB simulations.
The following three images represent a time-series, the corresponding power spectrum and the parametrized result with FOOOF:
Three Jupyer Notebooks were written to demonstrate how to use the package for the three different tests and all the functions. Each notebook corresponds to an example of a specific test. They can be found in the examples folder. It also mentions how to load all the necessary libraries and modules.
The final part of the project consisted in presenting a scientifically use case of the package. The model tested are generated with the Virtual Brain, a neuroinformatics platform. TVB is a framework for the simulation of the dynamics of large-scale brain networks with biologically realistic connectivity. TVB has a collection of neuronal dynamics models where parameters can be varied and adapted. For our example, we focused on three different single-node models: Generic2DOscillator, Wilson-Cowan and Jansen-Rit The aim is to create a database with CentralPeak test results for different brain waves ranges by performing parameter sweeps of the three selected models. This enables the user to determine which model parameters can be used to produce peaks of interest.
Here, is an example of a parameter sweep of CentralPeak test within the alpha frequency range for 4 different parameters of a generic dynamic system with two state variables (Generic2DOscillator).
The code to generate the database for any number of parameters is in the db folder (make_database.py). As input a json file is required with information on the parameters of interest as well as which model is simulated. This is an example of a json file for running a parameter sweep on Generic2DOscillator model of TVB for 4 different parameters:
{
"parameters":
{
"c": [0.0, 0.1, 0.2, 0.3, 0.4],
"b": [-1.4, -1.3, -1.2, -1.1, -1.0],
"alpha": [1.0, 1.5, 2.0, 2.5, 3.0],
"g": [0.1, 0.2, 0.3, 0.4, 0.5]
},
"model": "Generic2DOscillator",
"fr_range": "alpha"
}In addition to the json file, if another filename is given as an argument, the database is saved in a .csv file. The file looks as follows:
Installation is best done using pip
The principal non-standard Python dependencies are: sciunit, fooof, tvb-library, tvb-data
Install from source with the following on the command line:
git clone https://github.com/GriffithsLab/fooof-unit
cd fooof-unit
pip install -e .- Run more parameter sweeps to grow the database
- Improve FOOOF peak detection accuracy
- Perform several tests on more complex models
- Build similar test classes for stimulus evoked response simulations
During this project, I developed a package called fooofunit that performs unit tests on models generating a neural power spectrum such as corticothalamic models. A use case of the package is presented for the boolean test checking for the presence of peak within a frequency range. A database for three different Single-Node TVB models can be generated and saved in a .csv file with the results.
This will allow researchers to check the validity of their brain models and select optimal parameter values for their model.
I would like to thank my mentors, Dr. Rick Gerkin and Dr. John Griffiths, for all their guidance and support during this project. They have made this first remote and open source software project an incredible and enriching experience. I look forward to having the opportunity to work with them in the future.