I am using
Net = torchvision.models.resnet18(pretrained=True)
net = Net().to(dev)
just simply giving input(CIFAR10) and see how is the training loss. But I have gotten this error:
result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: ‘x’
how can I solve this?
Could you try with result = self.forward(input, **kwargs) instead?
You mean using Net = torchvision.models.resnet18(pretrained=True, **kwargs) instead of Net = torchvision.models.resnet18(pretrained=True)?
given the snippets you provided it is unclear to me what you did when you tried to forward the net.
pretrained=True/False flag wouldn’t be the cause of the complaints.
Did you called result = net(X) for calling forward pass?
It looks like you are missing the input to be fed to the net’s forward.
This is the code:
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision
import torchvision.transforms as transforms
import torch.optim as optim
dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
batch_size = 5
learning_rate = 0.001
num_epoch = 10
num_classes = 10
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
shuffle=True, num_workers=1)
testset = torchvision.datasets.CIFAR10(root='./data', train=False,
download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
shuffle=False, num_workers=1)
classes = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')
Net = torchvision.models.resnet18(pretrained=True)
net = Net().to(dev)
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=learning_rate, momentum=0.9)
for epoch in range(10):
for i, data in enumerate(trainloader, 0):
inputs, labels = data
inputs, labels = inputs.to(dev), labels.to(dev)
outputs = net(inputs)
loss = criterion(outputs, labels)
outputs = net(inputs)
optimizer.zero_grad()
loss.backward()
optimizer.step()
correct = 0
total = 0
for data in testloader:
inputs, labels = data
inputs, labels = inputs.to(dev), labels.to(dev)
outputs = net(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum()
print('Accuracy of the network on the 10000 test images: %d %%' % (
100 * correct / total))
I just called torchvision.models.resnet18 and gave input to net. I should do something else?
Change net = Net().to(dev) to net = Net.to(dev).
Currently you are calling the model (i.e. its forward method).
Thank you. It is working now. But it is showing another error:
RuntimeError: Given input size: (512x1x1). Calculated output size: (512x-5x-5). Output size is too small
This means CIFAR10 cannot be used for resnet18? how can I use CIFAR10, with a pretrained Resnet?
CIFAR10 images are quite small for the original input size of resnet, which is 224x224.
You could resize the images to this size before feeding them to the model using torchvision.transforms.Resize.
Thank you very much.
Can we get two outputs from pretrained ResNet? I mean adding two parallel linear layers at the end and getting outputs from them? I added a linear layer at the last layer of ResNet,
Net = torchvision.models.resnet18(pretrained=True)
num_ftrs = Net.fc.in_features
Net.fc = nn.Sequential( nn.Linear(num_ftrs, 10))
but how can I get two outputs from two parallel linear layer at the end? By using getattr https://discuss.pytorch.org/t/add-multiple-fc-layers-in-parallel/19008/13?
clf_outputs = {}
for i in range(self.num_fcs):
clf_outputs["fc%d" %i] = getattr(self, "fc%d" %i)(f)
how can this should be changed in pretrained case?
This might be a solution, but I would rather recommend to derive your custom ResNet module using the torchvision implementation from here.
Here is a small example using two custom linear layers:
import torchvision.models as models
import torch.utils.model_zoo as model_zoo
class MyResNet(models.ResNet):
def __init__(self, block, layers, num_classes=1000, pretrained=False):
super(MyResNet, self).__init__(block, layers, num_classes)
if pretrained:
self.load_state_dict(model_zoo.load_url(models.resnet.model_urls['resnet18']))
num_ftrs = self.fc.in_features
self.fc1 = nn.Linear(num_ftrs, 10)
self.fc2 = nn.Linear(num_ftrs, 10)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
x = self.avgpool(x)
x = x.view(x.size(0), -1)
a = self.fc1(x)
b = self.fc2(x)
return a, b
model = MyResNet(models.resnet.BasicBlock, [2, 2, 2, 2], pretrained=True)
x = torch.randn(2, 3, 224, 224)
output1, output2 = model(x)
Note that the current master branch from torchvision changed the definition slightly, e.g. two new arguments were introduced:
zero_init_residual=False, norm_layer=None
If you are installing torchvision from source, you should add them to the code. Otherwise it should work using my example.
I used MyResNet, it is giving me this error:
ret = torch.addmm(torch.jit._unwrap_optional(bias), input, weight.t())
RuntimeError: size mismatch, m1: [50 x 4096], m2: [512 x 10] at /dev/shm/pytorch_build_2018-12-10_20-29-49/avx2/python-3.6
how can I solve it?
Do you get this error from my code example?
Could you post a code snippet yielding this error?
Based on the error message the matrices are misshaped for the matrix multiplication.
Yes, I used your code example. When the net is calling, that error appears:
for i, data in enumerate(trainloader, 0):
inputs, labels = data
inputs, labels = inputs.to(dev), labels.to(dev)
(outputs1, outputs2) = net(inputs)
where the inputs are CIFAR10 and the net is
import torchvision.models as models
import torch.utils.model_zoo as model_zoo
class MyResNet(models.ResNet):
def __init__(self, block, layers, num_classes=1000, pretrained=False): #num_classes=1000
super(MyResNet, self).__init__(block, layers, num_classes)
if pretrained:
self.load_state_dict(model_zoo.load_url(models.resnet.model_urls['resnet18']))
num_ftrs = self.fc.in_features
self.fc1 = nn.Linear(num_ftrs, 10)
self.fc2 = nn.Linear(num_ftrs, 10)
def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)
x = self.avgpool(x)
x = x.view(x.size(0), -1)
a = self.fc1(x)
b = self.fc2(x)
return a, b
Net = MyResNet(models.resnet.BasicBlock, [2, 2, 2, 2], pretrained=True)
net = Net.to(dev)
Did you resize the CIFAR10 images to 224x224 as discussed before?
No, sorry 
The input can be larger? If I want to concatenate two images (448x224) is acceptable for the network?
ResNet can accept input larger than 224x224 but you’ll need to change the in_features for your FC layer after the pooling layer accordingly.
No you don’t have to do this. This should be an adaptive average pooling, which should pool always to the same spatial size (1x1 here) for all input sizes.
@Niki have you tried with 224x224 or 448x448 images? Theoretically the code provided by @ptrblck should work and this way we could make sure, the network is running fine and the problem lies within the image size. I don’t know if non-square images are a problem here but from my understanding they shouldn’t.