I am not sure, can you provide an example if possible please?
my question is what is happening for out1 = m(embedding1) and out1 = m(embedding1).
I dont get any error in case out1 = m(embedding1), although it does not have the format that suggested by pytorch doc.
I think in my example the InstanceNorm is actually doing something.
if I print (out1==embedding1).all(),(out2==embedding2).all() the results is false, false.
But I am not sure what is happening actually.
Also, it is not clear to me what it means when you say:
even if I use nn.InstanceNorm1d(in_channels, track_running_stats=False) it still works okay for both cases…
I see, yes that is correct, thanks for the clarification (I think you meant m(embedding2).mean(-1), m(embedding2).std(-1, unbiased=False) though).
so do you know when we track_running_stats=False what is computed in the m(embedding1)?