I have a problem that I cannot justify and I need an expert opinion as it might be architecture-related.
So I define a fresh Xception-based model, which is the following (Xception block is also provided):
I then run:
model = SamplingSegmentation(3, 1, 6).cuda()
out = model(torch.ones((5, 3, 64, 64)).cuda())
And the error is:
Does anyone know why this is happening? Printing the the input to the SeparableConvolution tells me it is already in cuda.
It seems you are using plain Python lists to store submodules in your model.
While this might work logically in your forward method, the submodules won’t be properly registered, so that e.g. model.parameters() won’t show these parameters.
Also, .to() or .cuda() calls won’t have any effect on the submodules, as is here the case.
Instead you should use nn.ModuleList to add your modules, which will properly register them.
PS: It’s always better to post code snippets and wrap them in three backticks ```, so that this code will be searchable in the discussion board and we could copy it in case we would like to debug it 
Thank you, your advice saved my day, I just enclosed the modules list inside a nn.ModuleList and the problem was fixed. I will keep your other note in mind the next time I post something.
