CNN with target vector and output of a vector

vision
1

Hello, I’m building a CNN and a bit unusually the data’s targets are not scalars, but vectors. So each image is described by 8 different floats. I built a model that I think should work, but I’m getting an error when I run the model, specifically in the backwards step of it all. Since the targets and outputs are vectors I pass in a gradient (I just made them all the same since I don’t really know what would be the best structure for it).

Here’s my model. The code below is for a test sample set before I run it on the full one

class ConvTest(nn.Module):
    def __init__(self):
        super(ConvTest, self).__init__()
        self.conv1 = nn.Conv2d(3,64,kernel_size = 11,padding = 5,stride = 1) #(64,210,280)
        self.conv1_drop = nn.Dropout2d()
        self.conv1_bn = nn.BatchNorm2d(64)
        
        self.conv2 = nn.Conv2d(64,128, kernel_size = 7, padding =  3,stride =1) #(128,210,280)
        self.conv2_drop = nn.Dropout2d()
        self.conv2_bn = nn.BatchNorm2d(128)
        
        self.conv3 = nn.Conv2d(128,192,kernel_size=3, padding = 1)
        self.conv3_drop = nn.Dropout2d()
        self.conv3_bn = nn.BatchNorm2d(192)
        
        self.conv4 = nn.Conv2d(192,256, kernel_size=3, padding = 1)
        self.conv4_drop = nn.Dropout2d()
        self.conv4_bn = nn.BatchNorm2d(256)
        
        self.fc1 = nn.Linear(256*2*3,4096)

        self.fc2 = nn.Linear(4096,4096)
        
        self.fc3 = nn.Linear(4096,1000)
        
        self.fc4 = nn.Linear(1000,NUM_LABELS)
        
    def forward(self,x):
        #print("step 1")
        #print(x)
        x = F.max_pool2d(F.relu(self.conv1_bn(self.conv1_drop(self.conv1(x)))),3)
        #print("step 2")
        #print(x)
        x = F.max_pool2d(F.relu(self.conv2_bn(self.conv2_drop(self.conv2(x)))),3)
        #print("step 3")
        #print(x)
        x = F.max_pool2d(F.relu(self.conv3_bn(self.conv3_drop(self.conv3(x)))),3)
        #print("step 4")
        #print(x)
        x = F.max_pool2d(F.relu(self.conv4_bn(self.conv4_drop(self.conv4(x)))),3)
        #print("step 5")
        #print(x)
        x = x.view(x.size(0),-1)
        #print("step 6")
        #print(x)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        output = F.relu(self.fc4(x))
        #print("step 7")
        #print(output)
        return output,x

Here is the code for running it

cnn = ConvTest()
print(cnn)
optimizer = torch.optim.Adam(cnn.parameters(), lr = LR)
loss_func = nn.MSELoss()


for epoch in range(EPOCH):
    for step, val in enumerate(loader):
        if step < 100:
            b_x = Variable(val['image'])
            b_y = Variable(val['target'])
            output = cnn(b_x)[0]

            loss = loss_func(output,b_y)
            optimizer.zero_grad()
            
            #because not a scalar
            gradients = torch.FloatTensor([0.125,0.125,0.125,0.125,0.125,0.125,0.125,0.125])
            loss.backward(gradients)
            
            optimizer.step()
        else:
            break
            
        if step % 5 == 0 :
            print(loss)

Would anyone have an idea of what this error means?

RuntimeError: Need gradOutput of dimension 1 and gradOutput.size[0] == 1 but got gradOutput to be of shape: [8] at /Users/soumith/minicondabuild3/conda-bld/pytorch_1518372155635/work/torch/lib/THNN/generic/MSECriterion.c:52
2

For anyone else out there I realized that the loss I was getting was 1 dimensional instead of the expected 8 dimensional. That was the issue

1 Like
3

How did you solve it? What loss function did you end using?

Isn’t the least as function supposed to return a scalar? (you could sum all the 8 values of loss on each of the vector component wise)?