How to find nan in dataframe

1

Hello. I need to find the elements of two dataframe columns that are nan and set them to zero:

for k in mat_files:
        my_df[k]['calibratedHsCube'] = my_df[k]['calibratedHsCube'].replace(np.nan, 0)
        my_df[k]['mask'] = my_df[k]['mask'].replace(np.nan, 0)

But I get this: AttributeError: ‘numpy.ndarray’ object has no attribute ‘replace’
Where am I wrong?

2

replace is for Series (pandas.Series.replace — pandas 1.5.0 documentation).

You could use fillna() for DataFrame objects.

3

I noticed that when I print the dataframe column there are still some nan values:

    for k in mat_files:
        my_df[k][['calibratedHsCube', 'mask']] = my_df[k][['calibratedHsCube', 'mask']].fillna(value=0)
        print(my_df[k]['calibratedHsCube'])

Did I make a mistake maybe?

4

I am not sure what mat_files is and consequently how exactly the dataframe is being indexed; looks like k corresponds to row index.
Apparently, you aren’t replacing nans in a full column or the full dataframe at once. Check your indexing technique to look for potential sources of error.

5

mat_files is:

 mat_files = glob(f'{Folder}/*.mat')