Hi i am implementing a Version of DCGAN that works on 1d spectral data. And i have run into a problem where as i need each feature map in the output of the network to be normalized in the range min=0 max=1. I have tried a number of different solutions to this problem, but what i really want to do is just normalize the output of the last layer(Tanh) to be in that range.
What i want is something like this:
min_val = X.min(-1)[0].min(-1)[0]
max_val = X.max(-1)[0].max(-1)[0]
X = (X-min_val[:,None, None])/(max_val[:,None,None]-min_val[:,None,None])
Is it possible to make a function that does this to the output of Tanh, and simultaneously tracks gradients, and is this necessary/desirable?
to illustrate what i want:
