I would like to get the mean and standard deviation from a tensor with shape (H,W) along the dimension 1 (so the output would be a pair of tensors with shape (H) or (H,1)). My problem is: I need to filter each row and only use the values selected by a given mask and the mask selects a different number of values from each row.
My actual approach is to generate another tensor with NaNs where I don’t care about the value and use torch.nanmean to get the mean ignoring those values but I don’t find an analogue function to get the std.
Example of behaviour.
>>> input = torch.randn(2, 4) >>> input tensor([[-1,4,3,8], [7,5,-2,-9]]) >>> mask = input >= 0 >>> input_nans = torch.where(mask, input, torch.nan) >>> mean = torch.nanmean(input_nans, dim=1) >>> mean tensor([5, 6])
This should be feasible (a mean from the values greater than 0) but there is no nanstd function to get the standard deviation ignoring nans. It’s important to note that the first value comes from the mean over 3 values while the second comes from the mean over 2 values (that is the reason I cannot simply filter to get a tensor only with the desired values, the rows length would not match).