Hello! I am trying to transfer a model to GPU as shown below:
net = TripletNet(resnet101())
net = torch.nn.DataParallel(net).to(torch.device('cuda:0'))
During training, when I run
print(next(net.parameters()).is_cuda)
I get False. Why does it happen? How can I transfer the model to GPU?
Note: when I try to transfer the data to the GPU, it works:
data1 = data1.cuda()
print(data1.is_cuda)
The above two lines of code yields True
How is TripletNet
defined and are you seeing any device mismatch error during training?
Below are the class definitions that compose TripletNet:
model_urls = {
'resnet18': 'https://download.pytorch.org/models/resnet18-5c106cde.pth'
}
def resnet18(model_urls, pretrained=True):
"""
Construct a ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = torchvision.models.resnet.ResNet(
torchvision.models.resnet.BasicBlock, [2, 2, 2, 2])
if pretrained:
model.load_state_dict(torch.utils.model_zoo.load_url(
model_urls['resnet18'], model_dir='../resnet18'))
return EmbeddingNet(model)
class TripletNet(nn.Module):
"""Triplet Network."""
def __init__(self, embeddingnet):
"""Triplet Network Builder."""
super(TripletNet, self).__init__()
self.embeddingnet = embeddingnet
def forward(self, a, p, n):
"""Forward pass."""
# anchor
embedded_a = self.embeddingnet(a)
# positive examples
embedded_p = self.embeddingnet(p)
# negative examples
embedded_n = self.embeddingnet(n)
return embedded_a, embedded_p, embedded_n
class EmbeddingNet(nn.Module):
"""EmbeddingNet using ResNet-101."""
def __init__(self, resnet):
"""Initialize EmbeddingNet model."""
super(EmbeddingNet, self).__init__()
# Everything except the last linear layer
self.features = nn.Sequential(*list(resnet.children())[:-1])
num_ftrs = resnet.fc.in_features
self.fc1 = nn.Linear(num_ftrs, 4096)
def forward(self, x):
"""Forward pass of EmbeddingNet."""
out = self.features(x)
out = out.view(out.size(0), -1)
out = self.fc1(out)
return out
Yes, I get the following device mismatch error during training:
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
Hope I have given you the right information, I am relatively new in this.
Thanks for the code.
I cannot reproduce this issue and get the expected True
output:
model_urls = {
'resnet18': 'https://download.pytorch.org/models/resnet18-5c106cde.pth'
}
def resnet18(model_urls, pretrained=True):
"""
Construct a ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
model = torchvision.models.resnet.ResNet(
torchvision.models.resnet.BasicBlock, [2, 2, 2, 2])
if pretrained:
model.load_state_dict(torch.utils.model_zoo.load_url(
model_urls['resnet18'], model_dir='../resnet18'))
return EmbeddingNet(model)
class TripletNet(nn.Module):
"""Triplet Network."""
def __init__(self, embeddingnet):
"""Triplet Network Builder."""
super(TripletNet, self).__init__()
self.embeddingnet = embeddingnet
def forward(self, a, p, n):
"""Forward pass."""
# anchor
embedded_a = self.embeddingnet(a)
# positive examples
embedded_p = self.embeddingnet(p)
# negative examples
embedded_n = self.embeddingnet(n)
return embedded_a, embedded_p, embedded_n
class EmbeddingNet(nn.Module):
"""EmbeddingNet using ResNet-101."""
def __init__(self, resnet):
"""Initialize EmbeddingNet model."""
super(EmbeddingNet, self).__init__()
# Everything except the last linear layer
self.features = nn.Sequential(*list(resnet.children())[:-1])
num_ftrs = resnet.fc.in_features
self.fc1 = nn.Linear(num_ftrs, 4096)
def forward(self, x):
"""Forward pass of EmbeddingNet."""
out = self.features(x)
out = out.view(out.size(0), -1)
out = self.fc1(out)
return out
net = TripletNet(resnet18(model_urls, pretrained=False))
net = torch.nn.DataParallel(net).to(torch.device('cuda:0'))
print(next(net.parameters()).is_cuda)
> True
Could you rerun this code and check, your output, please?
I checked again, the problem was in another part of the code where the network was re-initialized. The problem had nothing to do with what I sent you. Thank you very much for your help!