I have a model that has a GRU in it. I have traced the model but when I load it and call the forward method it works only if the length of the sequence I use is the same as the dummy input used when tracing. I get this warning when tracing:
TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
My dummy sequence has a length of 25 with features of size 256 (6400). I chose this at random since the model should work for variable sequence lengths. However, when I load the model and pass a sequence with a different length to the dummy input I get this error:
RuntimeError: shape ‘[-1, 30, 256]’ is invalid for input of size 6400
Is this normal (i.e. what the warning was about) and if so are there any workarounds?
I suppose one workaround is to have a max size and pad the tensor when I use it but this is not optimal.