 # How to get continuous value from output layer

I have a code (from here) to classify the MINST digits. The code is working fine. Here they used `CrossEntropyLoss` and `Adam` optimizer.

The model code is given below

``````class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Sequential(
nn.Conv2d(
in_channels=1,
out_channels=16,
kernel_size=5,
stride=1,
),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2),
)
self.conv2 = nn.Sequential(
nn.Conv2d(16, 32, 5, 1, 2),
nn.ReLU(),
nn.MaxPool2d(2),
)
# fully connected layer, output 10 classes
self.out = nn.Linear(32 * 7 * 7, 10)
# self.softmax = torch.nn.Softmax(dim=1)
def forward(self, x):
x = self.conv1(x)
x = self.conv2(x)
# flatten the output of conv2 to (batch_size, 32 * 7 * 7)
x = x.view(x.size(0), -1)
output = self.out(x)
# output = self.softmax(output)
return output, x    # return x for visualization

``````

Now, I wanted to get continuous value from the output layer. Say, I want the output as alike i.e, 1.0, 0.9, 8.6, 7.0, etc. If the value of the output layer is 1.0 and the label is 1 that means the prediction is perfect. Otherwise, not perfect. More simply, I want to think the MNIST digits as a regression problem.

So, I changed the loss function to `MSELoss` and optimizer to `SGD` (the rest of the code remains as the same as the website). But now, I am getting an error

``````/home/Opps_0/.local/lib/python3.8/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size()) that is different to the input size (torch.Size([100, 10])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
return F.mse_loss(input, target, reduction=self.reduction)
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/Opps_0/Desktop/MNIST/src/train.py", line 60, in <module>
File "/home/Opps_0/Desktop/MNIST/src/train.py", line 45, in train
loss = criterion(output, b_y)
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 528, in forward
return F.mse_loss(input, target, reduction=self.reduction)
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/nn/functional.py", line 2925, in mse_loss
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/functional.py", line 74, in broadcast_tensors
RuntimeError: The size of tensor a (10) must match the size of tensor b (100) at non-singleton dimension 1
``````

Could you tell me what I have to change to get the continuous value for the output layer?

I’m not sure how the target `b_y` is created, but it seems that it has a shape of `[unknown, 100]`, which doesn’t fit the model output of `[batch_size, 10]`. Could you explain how the target is computed and what its dimensions/shapes represent?

``````def data_loaders():
train_data = datasets.MNIST(
root = 'data',
train = True,
transform = transforms.ToTensor(),
)
test_data = datasets.MNIST(
root = 'data',
train = False,
transform = transforms.ToTensor()
)

batch_size=100,
shuffle=True,
num_workers=1),

batch_size=100,
shuffle=True,
num_workers=1),
}
``````

Later, at the time of training

``````def train(NB_EPOCS, model, loaders):
model.train()

# Train the model
for epoch in range(NB_EPOCS):
for i, (images, labels) in enumerate(loaders['train']):
b_x = Variable(images)   # batch x
b_y = Variable(labels)   # batch y
print(np.shape(b_x), np.shape(b_y))
output = model(b_x)
loss = criterion(output, b_y)
``````

The shape of the `b_x` and `b_y` is

``````torch.Size([100, 1, 28, 28]) torch.Size()
``````

Thanks for the update! I cannot reproduce the issue using your code and the training works fine:

``````model = CNN()

# Train the model
criterion = nn.CrossEntropyLoss()

for i, (images, labels) in enumerate(loaders['train']):
b_x = images   # batch x
b_y = labels  # batch y
print(np.shape(b_x), np.shape(b_y))
output = model(b_x)
loss = criterion(output, b_y)
``````

@ptrblck are you using `MSELoos` or `CrossEntropy`? CrossEntropy is working perfectly but getting error at the time of `MSELoos`

As seen in my code snippet I’m using `nn.CrossEntropyLoss` since it’s a classification use case.
If you want to use `nn.MSELoss` for specific reasons, you would have to provide the target in the same shape as the model output, i.e. `[batch_size, 10]` e.g. by one-hot encoding it.

You can see the `dtype` of the `b_y` is `Int64` and I am getting an error

``````Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/Opps_0/Desktop/MNIST/src/train.py", line 69, in <module>
File "/home/Opps_0/Desktop/MNIST/src/train.py", line 60, in train
loss.backward()
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/tensor.py", line 245, in backward
File "/home/Opps_0/.local/lib/python3.8/site-packages/torch/autograd/__init__.py", line 145, in backward
Variable._execution_engine.run_backward(
RuntimeError: Found dtype Long but expected Float
``````

Code

``````def train(NB_EPOCS, model, loaders):
model.train()
# Train the model
for epoch in range(NB_EPOCS):
for i, (images, labels) in enumerate(loaders['train']):
# gives batch data, normalize x when iterate train_loader
b_x = Variable(images)   # batch x
b_y = Variable(labels)   # batch y
print(np.shape(b_x), b_x.dtype, np.shape(b_y), b_y.dtype)
ohe_target = torch.nn.functional.one_hot(b_y, num_classes=10)
output = model(b_x)
loss = criterion(output, ohe_target)
``````
``````torch.Size([100, 1, 28, 28]) torch.float32 torch.Size() torch.int64
``````

If I change the dtype of the b_y to `Float` by the following code

``````b_y=b_y.float()
``````

Then I am getting another error,

``````RuntimeError: one_hot is only applicable to index tensor.
``````

What I have to do to solve the errors? (Main link of the question)

I guess you are transforming `b_y` to `float` before applying `one_hot`, not after?
This works:

``````# Train the model
criterion = nn.MSELoss()

for i, (images, labels) in enumerate(loaders['train']):
b_x = images   # batch x
b_y = labels  # batch y
ohe_target = torch.nn.functional.one_hot(b_y, num_classes=10).float()
output = model(b_x)
loss = criterion(output, ohe_target)
``````
1 Like