[python] Calculate the mean and standard deviation of ndarray containing nan

Python Tips

I’ll leave some personal notes on useful notations when using python.

github

The file in jupyter notebook format on github is here.

google colaboratory

To run it in google colaboratory here

Author’s environment

sw_vers
ProductName: Mac OS X

ProductName: Mac OS X
ProductVersion: 10.14.6
BuildVersion: 18G103

Python -V

Python 3.8.5

%matplotlib inline
%config InlineBackend.figure_format = 'svg'

import time
import json

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

import japanize_matplotlib

Take the mean and standard deviation of an array with NaN

When calculating the mean of an array with NaN, using np.mean will return np.nan. This is inconvenient, so np.nanmean and np.nanstd are useful to calculate the mean and standard deviation without np.nan.

a = np.array([i for i in range(5)])
a = np.append(a, np.nan)
a = np.append(a, np.nan)
a = np.append(a, np.nan)
a

array([ 0., 1., 2., 3., 4., nan, nan, nan])

b = np.array([i for i in range(5)])

np.nanmean(a) == np.nanmean(b)
np.nanmean(a)

2.0

We see that the result is the same.

np.nanstd(a) == np.nanstd(b)
np.nanstd(a)

1.4142135623730951

We can see that the result is the same for the standard deviation as well.

Maximum and minimum

We can calculate max and min in the same way.

array([ 0., 1., 2., 3., 4., nan, nan, nan])

np.nanmax(a)

4.0

np.nanmin(a)
0.0

0.0

Very useful!