Thanks @ptrblck Sir for ur answer. Actually Sir why the above code is not running with model.train() function as I need to train the model as well with the custom dataset.
The model is not working, since the shape of an activation is wrong and will thus raise a shape mismatch error in a conv layer.
As previously described, you would need to isolate this layer (by e.g. using the posted shapes) and fix the shape mismatch.
hi i’m new to pytorch but i’ve been getting the same ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1,256,1,1])
I’ve already set the batch size to be 5 at the terminal command:
CUDA_VISIBLE_DEVICES=0,1,2 python main2D_c.py --epochs 100 --lr 0.0025 --batch_size 4 --pattention --use_multiple_fusing_fpa --use_dropout
set batch size at the dataloader and include drop last:
dataset_train_loader = torch.utils.data.DataLoader(dataset_train, batch_size=opts.batch_size, shuffle=True,
pin_memory=False, num_workers=2,drop_last=True)
dataset_val_loader = torch.utils.data.DataLoader(dataset_val, batch_size=opts.batch_size, shuffle=False,
pin_memory=False, num_workers=2,drop_last=False)
and still got the error in the training loop:
for epoch in range(start_epoch, opts.epochs):
train_loss=[]
for i, (inputs, targets) in enumerate(dataset_train_loader):
cur_iter = epoch * len(dataset_train_loader) + i
lr = opts.lr * (1 - float(cur_iter) / max_iter) ** 0.9
optimizer.param_groups[0]['lr'] = lr
inputs, targets= inputs.to(device), targets.to(device)
#error from this line
outputs = model(inputs)
loss = criterion(outputs, targets)
train_loss.append(loss.item())
losses.update(loss.item(), opts.batch_size)
loss.backward()
optimizer.step()
optimizer.zero_grad()
print('epoch: {0} iter: {1}/{2} lr: {3:.6f} loss: {4:.4f}({5:.4f}) exp: {6}'\
.format(epoch + 1, i + 1, len(dataset_train_loader), lr, losses.val, losses.ema, exp))
any help would be greatly appreciated.
The error shows that the problematic batch contains a single sample, which is causing the error.
Since you are already using drop_last=True in the training DataLoader, I guess you might see this issue during the validation, if you don’t use model.eval(), or the reshaping of some intermediate activations in the forward method of your model is wrong and changes the batch size to 1.
Hi i have the same Error even if i call model.eval(). So i think my InsatnceNorm2d() Layers are missed while calling .eval(). How can i prevent this?
raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])```One simple way I used to overcome this issue when you do not have control directly over the batch size or do not want to change BN in your model, and do not have batch size=1 is slightly changing the proportion of training and validation sets, in a loop until succesful, usually will work in the second or third iteration.
import time
model_path = “/content/drive/My Drive/Generating-Webpages-from-Screenshots-master/models/”
batch_count = len(dataloader)
start = time.time()
for epoch in range(epochs):
#s = 0
for i , (images , captions , lengths) in enumerate(dataloader):
images = Variable(images.cuda())
captions = Variable(captions.cuda())
#lenghts is a list of caption length in descending order
#The collate_fn function does padding to the captions that are short in length
#so we need to pad our targets too so as to compute the loss
targets = nn.utils.rnn.pack_padded_sequence(input = captions, lengths = lengths, batch_first = True)[0]
#Clearing out buffers
E.zero_grad()
D.zero_grad()
features = E(images)
output = D(features , captions , lengths)
loss = criterion(output , targets)
loss.backward()
optimizer.step()
#s = s + 1
if epoch % log_step == 0 and i == 0:
print("Epoch : {} || Loss : {} || Perplexity : {}".format(epoch , loss.item()
, np.exp(loss.item())))
#Uncomment this to use checkpointing
#if (epoch + 1) % save_after_epochs == 0 and i == 0:
#print("Saving Models")
#torch.save(E.state_dict , os.path.join(model_path , 'encoder-{}'.format(epoch + 1)))
#torch.save(D.state_dict , os.path.join(model_path , 'decoder-{}'.format(epoch + 1)))
print(“Done Training!”)
print(“Time : {}”.format(time.time() - start))
RuntimeError Traceback (most recent call last)
in ()
22 D.zero_grad()
23
—> 24 features = E(images)
25 output = D(features , captions , lengths)
26 loss = criterion(output , targets)
6 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
394 _pair(0), self.dilation, self.groups)
395 return F.conv2d(input, weight, bias, self.stride,
→ 396 self.padding, self.dilation, self.groups)
397
398 def forward(self, input: Tensor) → Tensor:
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
Based on the error message:
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
it seems you haven’t moved the model to the GPU via model.to('cuda') or model.cuda().
Also, Variables are deprecated since PyTorch 0.4 so you can use tensors now.
You can also post code snippets by wrapping them into three backticks ```, which makes debugging easier 
The solution worked, you are the best moderator I have seen on PyTorch
Thanks, @ptrblck hat of to you
1 Like
I am getting the same issue. I am using BN1d and for the test phase I am using a batch size of 1, but I am using model.eval() before evaluating, so shouldn’t it not have an issue? I am wondering if I am not calling model.eval() correctly because adding dropout to the model seems to hurt the validation accuracy of the network, which could be becasue dropout isnt supposed to be turned on during validation so its possible that model.eval() isnt even actually working for me.
class MLP(nn.Module):
def __init__(self, num_features, num_outputs, hidden_layers, drop_out_rate, BN = True):
super(MultipleRegressionCustom4, self).__init__()
self.use_BN = BN
self.layers = nn.ModuleList()
last_layer = num_features
for layer in hidden_layers:
self.layers.append(nn.Linear(last_layer,layer))
if self.use_BN==True:
self.layers.append(nn.BatchNorm1d(num_features=layer))
self.layers.append(nn.ReLU())
self.layers.append(nn.Dropout(drop_out_rate))
last_layer = layer
self.layers.append(nn.Linear(last_layer, num_outputs))
print(self.layers)
def forward(self, inputs):
x = inputs
for i in range(0,len(self.layers)-1):
x=self.layers[i](x)
x = self.layers[-1](x)
return (x)
this is my model definition
and my test step is like this:
def test(self):
print('test step')
y_pred_list = []
y_test = []
with torch.no_grad():
self.model.eval()
for X_batch, y_batch in self.test_loader:
X_batch = X_batch.to(self.device)
y_test_pred = self.model(X_batch)
Any ideas?
Calling model.eval() works for me using your code:
model = MLP(1, 1, [1], 0.5)
x = torch.randn(1, 1)
out = model(x)
> ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 1])
model.eval()
out = model(x)
print(out)
> tensor([[0.0654]], grad_fn=<AddmmBackward>)
thats odd because when I do my test function I get the value error!
ok so the error seems to be in the training portion, I am training with a batch size of 16 but for some reason a mini batch with size of 1 seems to come at the end, possibly because I have a multiple of 16 +1 samples…
If you are using a DataLoader, you could use drop_last=True, which would drop the last batch in case it’s smaller than the specified batch size.
Thanks for pointing this out!
Amazing!
This works!!!
Have you resolved this problem?
Hi Im facing the same problem even though I put droplast=True on a dataloader and also multilple batch
size by number of GPU.
droplast is not a valid argument, so make sure you are using drop_last=True in the DataLoader creation.
If that’s already the case, print the shape of the input data by and compare it to the data you would receive by just iterating the DataLoader.
1 Like