Check presence of Null and NaN values in the train data

null_nan_train.py

null_arr = []
nan_arr = []
for sig in metadata_train['signal_id'].values:
   sig_data = pd.read_parquet('/content/train.parquet',
engine='fastparquet', columns=[str(sig)])
null_arr.append(sig_data.isnull().sum())
nan_arr.append(sig_data.isna().sum())
print(f"Number of Null values in train data: {np.sum(null_arr)}")
print(f"Number of NaN values in train data: {np.sum(nan_arr)}")

VSB Power Line Blog - the presence of null and nan values in train data