Hi,
I know my title is kind of weird. Here is the problem.
In my model, when implement the forward step. I use torch.Tensor.view() method.
Reasons: the torchsde does not allow the k-D input tensors (k > 2).
def forward(self,x):
out = self.fe(x)
bs = x.shape[0]
# print(f"Shape after Feature Extraction Layer: {out.shape}")
out = out.view(bs,-1)
# print(f"After the feature extraction step, shape is: {out.shape}")
# print(f"Device of out {out.device}")
# print(f"Shape before the SDE Intergral: {out.shape}")
out = sdeint(self.rm,out,self.intergrated_time, options=self.option,method="euler", atol=5e-2,rtol=5e-2, dt=0.1, dt_min=0.05)[-1]
out = out.view(bs,self.input_conv_channel, self.input_conv_size, self.input_conv_size)
out = self.fcc(out)
return out
I have view the tensor as flattened shape, after that i apply the sdeint to the data.
But for some reason, the accuracy is really bad on any size dataset (cifar10 and svhn).
I also tried a same method (sdeint) for bunch of linear layers, which I don’t need to use view method to forward. And the accuracy is really good.
I have read and see the document say that
" PyTorch allows a tensor to be a View of an existing tensor. View tensor shares the same underlying data with its base tensor. Supporting View avoids explicit data copy, thus allows us to do fast and memory efficient reshaping, slicing and element-wise operations.".
But I am in doubt that the view method affected my training process. Doesn’t it ?.
Hope to received the answer.
Many thanks
Best regards,
Tien Dung