The aim of the project was to conduct a study on what effects does aerosol data have on pv energy generation, and what performance boost could be achieved if we added aerosol data variables to the existing model. Primarily, PVNet and ocf-data-sampler were used to train models and measure performance.
The deliverables of this project included the scripts for gathering and transforming the data, integration of new data with PVNet and ocf-data-sampler and performance analysis
After some discussion we setteled on merra2 dataset openly available on disc.gsfc.nasa.gov, which contains aerosol data, four variables were finalized to be included in the data, behavior of these variables was plotted for the year 2024 and is available in the link below.
The variables were provided in separate netcdf files for each day, this data was transformed and converted to zarr to work with PVNet and ocf-data-sampler, the code for downloading, plotting and transforming the data is in the link below
NWP Loader was added (along with its corresponding test case) to process and load merra2 data into the model
We trained 2 differnt types of models, the first one would serve as a baseline of how the data would behave and what results would we get without the merra2 data, and the second model would show the effects of merra2 data. The baseline model would have GSP data (the value we are trying to predict), along with ECMWF (a set of 12 variables which help predict the GSP value), and the second model would contain merra2 data in addition to GSP and ECMWF, so we could see the improvements. Finally 2 sets of 2 models were trained, both sets containing differntly randomized set of samples to test the performance. The results were stored on wandb and mae analysis results are in the images below
| Model name | What it means |
|---|---|
| baseline_new_ecmwf_20/5 | First baseline model with a unique sample set containing only GSP and ECMWF data |
| merra_new_ecmwf_20/5 | Model trained with GSP + MERRA + ECMWF containing the same sample as the first baseline |
| baseline_12596 | Second baseline model trained with a different sample set containing GSP and ECMWF |
| merra_12596 | Second model trained with GSP + MERRA + ECMWF containing the same sample as the second baseline model (baseline_12596) |
| baseline_new_ecmwf_20/5 MAE % | merra_new_ecmwf_20/5 MAE % | baseline_12596 MAE % | merra_12596 MAE % |
|---|---|---|---|
| 2.308 | 2.246 | 2.311 | 2.257 |
Thats approx. 2.7% improvemnt in "merra_new_ecmwf_20/5"
and approx. 2.3% improvement in "merra_12596"
Following are the issues i raised and the PRs i created while working with PVNet and ocf-data-sampler
- pvnet issue 407 A variable in config file pointed to a yaml which did not exist(fixed)
- pvnet issue 438 This error occured when you tried to run the model (probably caused when you try to use multiple gpus at the same time)
- pvnet issue 443 An error was thrown when trying to log val_horizon_mae_plot (probably because of the same gpu issue as above, a temporary fix was implemented locally, but PR was not raised)
- pvnet issue 463 The names of variable were updated when logging data to wandb but the name in mae_analysis script was not updated(fixed)
- pvnet PR 464 PR raised to fix issue 463 mentioned above, the variable name was changed in the file
- pvnet PR 410 PR raised to fix issue 407, the file names were updated
- ocf-data-sampler issue 286 There was no specific error thrown when gsp/site data was missing
- ocf-data-sampler issue 283 Data sampler, by default installed zarr2 and zarrs created with zarr3 gave an error (fixed by peter, data sampler set to install zarr3 by default)
- ocf-data-sampler issue 282 Data sampler cannot process the data if the timestamps are not rounded to the hour (ongoing fix)
- ocf-data-sampler PR 290 Issue 286 was fixed by adding a pydantic condition
- Solving the time slicing problem so a wider range of data can be processed with data sampler
- Share a complete experiment write-up in OCF's discussions