When I do print(type(x))
of a tensor in forward()
, the type is <class 'torch.Tensor'>
, shouldn’t it be .cuda.Tensor
?
cuda.is_available()
is True, next(model.parameters()).is_cuda
is also True, and I can tell my GPU is being used. So I’m sure it’s running on GPU, but I’m confused by the type.
Here’s the model
class Net(nn.Module):
def __init__(self, vocab_sz, emb_sz, input_sz, out_sz):
super(Net, self).__init__()
self.embed = nn.Embedding(vocab_sz, emb_sz)
self.embed_pos = nn.Parameter(torch.randn(input_sz, 32))
self.conv1 = nn.Conv2d(1, 32, (3, emb_sz + 32,))
self.conv2 = nn.Conv2d(32, 64, (2, 1,))
self.conv3 = nn.Conv2d(64, 64, (2, 1,))
self.fc = nn.Linear(64, out_sz)
def forward(self, x):
bs = x.size(0)
emb_pos = self.embed_pos.unsqueeze(0).repeat(bs, 1, 1)
emb = torch.cat((self.embed(x), emb_pos,), dim=2)
x = emb.unsqueeze(1)
# print(x.size())
x = F.relu(self.conv1(x))
print(type(x))
# print(x.size())
x = F.relu(self.conv2(x))
# print(x.size())
x = F.relu(self.conv3(x))
# print(x.size())
x = x.view(bs, -1)
y = self.fc(x)
return F.log_softmax(y, dim=1)
Also: I’m also worried that the line self.embed_pos.unsqueeze(0).repeat(bs, 1, 1)
is wasteful, but unsure if it is, and if it is what one could do about it (what I’ve tried so far has resulted in weird errors).