How to make a dataLoader with reqiures_grad=True from a dasaset and not revert reqiures_grad to false?

projects
1

I am trying to create a Dataloader using the built-in DataLoader class from a dataset created using the built-in Dataset class, the problem is that the tensor in the DataLoader reverts to reqiures_grad=False. I think the DataLoader class makes a copy or something, what might be the best way of going around this?.

_data=torch.tensor(newData).float();
lebels=torch.tensor(y_train).float();
data=torch.utils.data.TensorDataset(_data,lebels)
print("data shape: ", _data.shape);
print("Lebels shape : ", lebels.shape);
data=torch.utils.data.TensorDataset(data.requires_grad()
, lebels.requires_grad());

_data[0][0].requires_grad #outputs true

trainDataLoader = DataLoader(dataset=_data, batch_size=15, shuffle=True)
it = iter(trainDataLoader)
first = next(it)
first[0][0].requires_grad #outputs False

2

I cannot reproduce the issue using:

_data = torch.randn(100, 1)
lebels = torch.randn(100, 1)
data = torch.utils.data.TensorDataset(_data,lebels)
print("data shape: ", _data.shape);
print("Lebels shape : ", lebels.shape);
data = torch.utils.data.TensorDataset(_data.requires_grad_(), lebels.requires_grad_())

print(_data[0][0].requires_grad)
> True

x, y = data[0]
print(x.requires_grad)
> True
print(y.requires_grad)
> True

trainDataLoader = torch.utils.data.DataLoader(dataset=data, batch_size=15, shuffle=True)
it = iter(trainDataLoader)
first = next(it)
first[0][0].requires_grad
> True

In your current code snippet you are trying to “call” the requires_grad attribute, which is a bool value and which would raise an error:

data=torch.utils.data.TensorDataset(data.requires_grad(), lebels.requires_grad())

so I changed it to the inplace ops: tensor.requires_grad_().
Also, you are passing the _data tensor (not the data Dataset) to the DataLoader, which you also might want to change.

1 Like