I have been using Module.register_forward_hook and Module.register_forward_pre_hook. I looked at the shape of the data I got from them and there is something I can’t figure out.
Say the shape of some data going into a module is: (2, 3, 256, 256), if I print the shape of the input data that I got from the hook I get: (1, 2, 3, 256, 256). The input data always comes in the format tuple(<Tensor with data>). I haven’t found a case where the tuple contain more than one entry.
Interestingly, this is not the case for output data. That I found to be in the format <Tensor with data>.
What is the purpose of the tuple?