Skip to content

Instantly share code, notes, and snippets.

@aakash30jan
Last active May 29, 2020 22:45
Show Gist options
  • Save aakash30jan/15cf4051bcb3c6b9fd917301d75cd719 to your computer and use it in GitHub Desktop.
Save aakash30jan/15cf4051bcb3c6b9fd917301d75cd719 to your computer and use it in GitHub Desktop.
Possible TF2.0 or NetCDF4 Issue
"""
numpy 1.17.3
tensorflow 2.0.0
netCDF4 1.5.3
Get these: "pip install -r https://gist.githubusercontent.com/aakash30jan/9ae0cf3dde8a63d28df5275873cb0f10/raw/9e8d7e1f84fc2f67fe842d75e8623756319786bb/requirements.txt"
Get full environment: "pip install -r https://gist.githubusercontent.com/aakash30jan/9ae0cf3dde8a63d28df5275873cb0f10/raw/9e8d7e1f84fc2f67fe842d75e8623756319786bb/frozen-requirements.txt"
"""
#It's weird that the sequence of importing libraries is affecting i/o of nc file.
import numpy as np
import tensorflow as tf ###if put here, saving NC doesn't work
import netCDF4 as nc
#import tensorflow as tf ###if put here, saving NC works properly
Nx, Ny = 160,200
ArrayFieldTest = np.ones((Nx,Ny,2))
x = np.arange(Nx)
y = np.arange(Ny)
def write2NetCDF(ArrayField,x,y,outputfile):
print("Writing field data of shape", ArrayField.shape)
ncfile = nc.Dataset(outputfile,'w',format='NETCDF4_CLASSIC')
ncfile.createDimension('X',x.shape[0])
ncfile.createDimension('Y',y.shape[0])
newx = ncfile.createVariable('X','d',('X'))
newx[:] = x
newy = ncfile.createVariable('Y','d',('Y'))
newy[:] = y
velx = ncfile.createVariable('Component_X','d',('Y','X'))
velx[:] = ArrayField[:,:,0].T
vely = ncfile.createVariable('Component_Y','d',('Y','X'))
vely[:] = ArrayField[:,:,1].T
print("Success till ncfile.close(), something written to: ", outputfile)
ncfile.close() ###### <<<<<<<<<<<<<<<<<<<< Gives error here, and writes garbage to outputfile.nc file
print("Data successfully written to: ", outputfile)
return;
write2NetCDF(ArrayFieldTest,x,y,"outputfile.nc")
"""
Output Summary:
(pyEnvTF2) [aakash.patil@n-in4 test](1169283-->433mn)$ python3 possibleBug.py
Writing field data of shape (160, 200, 2)
Success till ncfile.close(), something written to: outputfile.nc
Traceback (most recent call last):
File "possibleBug.py", line 39, in <module>
write2NetCDF(ArrayFieldTest,x,y,"outputfile.nc")
File "possibleBug.py", line 34, in write2NetCDF
ncfile.close() ###### <<<<<<<<<<<<<<<<<<<< Gives error here, and writes garbage to outputfile.nc file
File "netCDF4/_netCDF4.pyx", line 2485, in netCDF4._netCDF4.Dataset.close
File "netCDF4/_netCDF4.pyx", line 2449, in netCDF4._netCDF4.Dataset._close
File "netCDF4/_netCDF4.pyx", line 1887, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: HDF error
(pyEnvTF2) [aakash.patil@n-in4 test](1169283-->432mn)$
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment