I’m working on a Variation AutoEncoder example and I have a simple question that is mainly about PyTorch.
Let’s suppose that we use binary cross entropy (torch.nn.BCELoss()) as the reconstruction loss; that is, we apply BCELoss on the reconstructed images (output of decoder) and target images (original images).
However, original images are the input of the encoder, and, thus, they are naturally defined as torch.autograd.Variable. On the other hand, the reconstructed images are torch.Tensor. Applying BCELoss on them makes PyCharm complain about the types of inputs (BCELoss is expected to take two tensors, not one tensor and one Variable).
I’m just wondering whether this might cause any problems. Should I give BCELoss its targets (i.e., original images) as non-Variable tensors, or is it ok to keep it as is?
Hi, thanks for the quick response. That’s fine, but what do you mean by "Variable is not a thing anymore"? Shouldn’t input tensors, for instance, be defined as Variables for auto-differentiation etc?
Variable and Tensor have been merged. So they are now exactly the same thing ! (since version 0.4 )
You can just give regular Tensors to your forward pass. And if you need them to require gradients, you can either create them with torch.rand(10, 10, requires_grad=True) or set it with your_tensor.requires_grad_().
That’s interesting (and terrifying that I didn’t notice this major change before).
So I can just get my input tensors from my data loader and just apply .requires_grad() on them? For example, in a training session, should we just do the following?
for i, x in enumerate(data_loader):
out = model(x.requires_grad())
And in the case where we need to upload data on GPU, we just apply .cuda() after .requires_grad()?
If you don’t plan on computing the gradients for x, you don’t need to do anything
The parameters inside the model already require gradients and that will be enough for the autograd to know when to start recording the graph.
)