How to train a network with one single input in pytorch?

Site Feedback
1

I want to try a toy case in pytorch. I want to use a simple full-connected neural network to fit a sin curve (the dataset is 100 points in the sin curve).
the architecture of the neural network is below:

20180411_diagram_1-4-6-4-1
20180411_diagram_1-4-6-4-11250×1530 103 KB

However, errors are reported like this:

Traceback (most recent call last):
File “main.py”, line 93, in
train(epoch)
File “main.py”, line 63, in train
output = model(data)
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 325, in call
result = self.forward(*input, **kwargs)
File “/home/hdeping/complexNetwork/00_CCodes/20_pytorchFitContinuousTarget/01_v1/mymodel.py”, line 13, in forward
x = F.relu(self.fc1(x))
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 325, in call
result = self.forward(*input, **kwargs)
File “/usr/lib/python3.6/site-packages/torch/nn/modules/linear.py”, line 55, in forward
return F.linear(input, self.weight, self.bias)
File “/usr/lib/python3.6/site-packages/torch/nn/functional.py”, line 837, in linear
output = input.matmul(weight.t())
File “/usr/lib/python3.6/site-packages/torch/autograd/variable.py”, line 386, in matmul
return torch.matmul(self, other)
File “/usr/lib/python3.6/site-packages/torch/functional.py”, line 168, in matmul
return torch.mm(tensor1.unsqueeze(0), tensor2).squeeze_(0)
RuntimeError: Expected object of type Variable[torch.cuda.DoubleTensor] but found type Variable[torch.cuda.FloatTensor] for argument #1 ‘mat2’

I am sure that the type of the input data is torch.cuda.DoubleTensor.
What is wrong? I am doubting that something is wrong with the single input.

2

Could you cast it to a FloatTensor with x = x.float()?
The modules are initialized with FloatTensors by default, so that’s probably throwing an error:

lin = nn.Linear(1, 10)
print(type(lin.weight.data))
3

Thanks for your reply.
Yeah！That is it :joy:
.float should be added in the first layer!
thanks so much!

but , something new happens.

Traceback (most recent call last):
File “main.py”, line 93, in
train(epoch)
File “main.py”, line 63, in train
output = model(data)
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 325, in call
result = self.forward(*input, **kwargs)
File “/home/hdeping/complexNetwork/00_CCodes/20_pytorchFitContinuousTarget/02_v2/mymodel.py”, line 13, in forward
x = F.relu(self.fc1(x.float))
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 325, in call
result = self.forward(*input, **kwargs)
File “/usr/lib/python3.6/site-packages/torch/nn/modules/linear.py”, line 55, in forward
return F.linear(input, self.weight, self.bias)
File “/usr/lib/python3.6/site-packages/torch/nn/functional.py”, line 833, in linear
if input.dim() == 2 and bias is not None:
AttributeError: ‘function’ object has no attribute 'dim’

what is wrong?

4

.float() is a function, so you have to use parentheses.

5

Thanks! What a stupid mistake I make.

However, one more error was reported.

File “/home/hdeping/complexNetwork/00_CCodes/20_pytorchFitContinuousTarget/02_v2/mymodel.py”, line 13, in forward
x = F.relu(self.fc1(x.float()))
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 325, in call
result = self.forward(*input, kwargs)
File “/usr/lib/python3.6/site-packages/torch/nn/modules/linear.py”, line 55, in forward
return F.linear(input, self.weight, self.bias)
File “/usr/lib/python3.6/site-packages/torch/nn/functional.py”, line 837, in linear
output = input.matmul(weight.t())
File “/usr/lib/python3.6/site-packages/torch/autograd/variable.py”, line 386, in matmul
return torch.matmul(self, other)
File “/usr/lib/python3.6/site-packages/torch/functional.py”, line 168, in matmul
** return torch.mm(tensor1.unsqueeze(0), tensor2).squeeze_(0)
RuntimeError: size mismatch at /build/python-pytorch/src/pytorch-0.3.0-py3-cuda/torch/lib/THC/generic/THCTensorMathBlas.cu:243

6

It seems that a Linear layer throws a size mismatch error.
Try to print the Tensor shape before you apply the layer and check, what went wrong.

Also, could you post the model code, in case there are some more mistakes? :wink:

7

Thanks again.
Of course, my model code is below:

2018-04-11-220837_476x457_scrot

And, here is my training code:

def train(epoch):
model.train()
for batch_idx, (data, target) in enumerate(train_loader):
#print(type(data),type(target))
print(data.shape,target.shape)
if args.cuda:
data = data.cuda()
target = target.cuda()
print(type(data),type(target))
data, target = Variable(data), Variable(target)
optimizer.zero_grad()
output = model(data)
loss = F.cross_entropy(output, target)
loss.backward()
optimizer.step()

the training batch-size was set to 64, I have no idea where the size
mismatch arose.

8

The size mismatch error comes from self.fc1, so I assume your x has an invalid shape.
x should have the dimensions [batch_size, *, 1], in the simplest case just [batch_size, 1].
Have a look at this example:

batch_size = 10
x = Variable(torch.randn(batch_size, 1))

fc1 = nn.Linear(1, 10)
out = fc1(x)

Also, in your first post, you mentioned, that you would like to fit a sin curve.
Based on the loss you are using (CrossEntropyLoss), if seems you would like to perform a classification?
Maybe you should change the loss function to MSELoss and remove the log_softmax in your model.
As an activation function you could try F.tanh.

9

Thanks！ The problem of size mismatch was solved and I use MSELoss as
my loss function, my training code was below:

41 data, target = Variable(data), Variable(target)
42 optimizer.zero_grad()
43 output = model(data)
44 loss = nn.MSELoss()
45 diff = loss(output, target)
46 diff.backward()
47 optimizer.step()

New errors are reported:

File “”, line 74, in
File “”, line 51, in train
File “/usr/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 366, in getattr
** type(self).name, name))**
AttributeError: ‘MSELoss’ object has no attribute 'data’

10

Some code is missing, could you post the full stack trace and your code?
Also, you can insert code by encapsulating it with three backticks (```):

three `

Your code

three `

11

my training code is

39 target = target.cuda()
40
41 data, target = Variable(data), Variable(target)
42 optimizer.zero_grad()
43 output = model(data)
44 print(“loss here”)
45 diff = loss(output, target)
46 diff.backward()
47 optimizer.step()
48 #if batch_idx % args.log_interval == 0:
49 print(‘Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}’.format(
50 epoch, batch_idx * len(data), len(train_loader.dataset),
51 100. * batch_idx / len(train_loader), loss.data[0]))

I change “loss.data” into “diff.data” , it works finally!
Thanks so much!
Have a good day!