Hi there! 
I wanted to integrate the pre- and postprocessing inside my pytorch model (following this thread How to add preprocess into the model - #3 by MarcSteven) such that I have exactly the same processing steps during training and serving time.
For this purpose I apply a list of torch functions before and after applying the ‘actual’ processing steps of the model in the forward method:
def forward(self, x: torch.Tensor) -> torch.Tensor:
# apply preprocessing functions
for function in self.preprocessing_funcs:
x = function(x)
for layer_idx, layer in enumerate(self.layers):
x = layer(x)
if layer_idx == len(self.layers) - 1:
# apply last layer activation function after last layer
x = self.last_activation_fn(x)
else:
# apply hidden layer activation function after input and each hidden layer
x = self.hidden_activation_fn(x)
# apply postprocessing functions
for function in self.postprocessing_funcs:
x = function(x)
return x
Of course I set all parameters used in these functions to requires_grad = False such that they are not trained. So as a result of this ‘in model processing’ I would need to perform the training on completely raw input and output data and this feels kind of wrong to me…
Therefore my question is: Does the training work that way? Or am I missing something related to the torch autograd logic?