Using tensors of different size to calculate loss

1

Hi,

I am searching ways to compute the loss using slices. The code below is an example that works, but it has a fixed length (1:30). What I need requires the use of variable length, one for each element of my dataset.

To explain the context, I have a convnet, but one dimension of the cases in my dataset has variable size. I am not sure what I have to do, but I am using pad to equalize the length of my data. I think that would be easier to slice the y_pred and the y after the conv to calculate the loss.

Any ideas how I could do this? Are there better ways to do this?

I can’t transform the data.

x = Variable(torch.randn(160,21,60,1))
y = Variable(torch.LongTensor(160,60,1).random_(0, 4), requires_grad=False)

model = Model()

criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(),lr=0.001,momentum=0.9)

for t in range(500):

    y_pred = model(x)
    
    loss = criterion(y_pred.narrow(2,1,30),y.narrow(1,1,30))

    print(t, loss.data[0])

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
1 Like
2

I don’t get what you mean. Do you want

size2 = y_pred.size(2)
loss = criterion(y_pred ,y[:,:size2,:])

it shall work, you can put input of various length to criterion.

3

@chenyuntc Thank you for aswering me.

Each example in my dataset has a different width. I am not sure if I could use this in a convnet, so I made all example with equal width using 0s. So all examples have the width equals the wider.

I am trying to understand how I could do this. Sorry if the answer to my problem is very obvious, is that I’m still studying all this stuff.

If I understand your example, I should use a minibatch=1, right?

4

Is it an NLP problem? Input shall have the same width in a batch. You can use pack_padded_sequence to pad your input to the same size or handle it with your own code(seems you have done it with using 0s padding).

Maybe you want reshape y and y_pred to calculate loss as below

loss = criterion(y_pred.view(-1,4), y.view(-1,1))

Sorry If I get wrong, I am not familiar with such problem.