Mat1 and mat2 shapes cannot be multiplied (784x10 and 784x10)

1

Hi People,

I am using the following code to create a simple neural network but it gives me the following error: mat1 and mat2 shapes cannot be multiplied (784x10 and 784x10)

Not sure why this is happening as matrix sizes are same i.e. 784*10

#import libraries
%matplotlib inline
import torch
import torchvision
import torchvision.transforms as transforms

#load data
transform = transforms.Compose(

[transforms.ToTensor(),

 transforms.Normalize((0.5), (0.5))])

batch_size = 10

trainset = torchvision.datasets.MNIST(root=’./data’, train=True,

                                    download=True, transform=transform)

trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,

                                      shuffle=True, num_workers=2)

testset = torchvision.datasets.MNIST(root=’./data’, train=False,

                                   download=True, transform=transform)

testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,

                                     shuffle=False, num_workers=2)

#wrap iterator around the batch
dataiter = iter(trainloader)

x_train, y_train = dataiter.next()

dataiter = iter(testloader)

x_valid, y_valid = dataiter.next()

#initialize weights and bias
import math

weights = torch.randn(784, 10) / math.sqrt(784)

weights.requires_grad_()

bias = torch.zeros(10, requires_grad=True)

#Define loss function
def log_softmax(x):

return x - x.exp().sum(-1).log().unsqueeze(-1)

def model(xb):

return log_softmax(xb @ weights + bias)

#create prediction

bs = 64 # batch size

xb = x_train[0:bs] # a mini-batch from x

xb=xb.reshape(784,10)

preds = model(xb) # predictions

preds[0], preds.shape

print(preds[0], preds.shape)

2

@ptrblck can you plz help this code…thanx

3

You could have to make sure that the inner matrix sizes are equal:

x = torch.randn(784, 10)
y = torch.randn(10, 784)
out = torch.matmul(x, y)
print(out.shape)
> torch.Size([784, 784])
1 Like
4

Hi Thank you for your reply.

I have made following changes but still get this error : “The size of tensor a (28) must match the size of tensor b (10) at non-singleton dimension 3”

weights = (torch.randn(28, 28) / math.sqrt(28))
weights.requires_grad_()
bias = torch.zeros((10), requires_grad=True)
print(weights.shape)
print(bias.shape)
def log_softmax(x):

return (x - x.exp().sum(-1).log().unsqueeze(-1))

def model(xb):

return log_softmax(xb @ weights + bias)

bs = 1 # batch size

xb = x_train[0:bs] # a mini-batch from x

preds = model(xb) # predictions

preds[0], preds.shape

print(preds[0], preds.shape)

5

If i change bias to 28, the error goes away but prediction result is a matrix of 28*28 instead of 10 probability values for y