# What's wrong with me in the process of forward？

We want to realize a net that sent the vgg features into the lstm, then generate a tensor (2,3) and a fc layer for classification. For the generated (2,3) tensor, we want some elements to be 0. So we use the tensor arr to element-wise multiplication with m1. And then the m1 continue to be sent to the next .
What’s wrong with me in the process of forward？
How to realize the element-wise multiplication in the forward?
Another question ,I want to make my own loss function that need to use the element of m1,m2,m3. How can i get the element from the tensor to make my loss function.

``````class ST_LSTM(nn.Module):
def __init__(self,features,num_classes):
super(ST_LSTM,self).__init__()
self.features = features
for p in self.features.parameters():

self.classifier2 = nn.Sequential(
nn.Linear(1024, 1024),
nn.ReLU(True),
nn.Dropout(),
nn.Linear(1024, 30),
)
self.mm = nn.Sequential(
nn.Linear(1024, 512),
nn.ReLU(True),
nn.Dropout(),
nn.Linear(512, 6),
)
self.lstm = nn.LSTM(7*7*512,1024,batch_first=True)

# stn is the Spatial Transformer Networks from the pytorch Tutorial
def stn(self, x, theta):
# x = self.features(x)
# xs = self.localization(x)
# xs = xs.view(-1, 10 * 3 * 3)
# theta = self.fc_loc(xs)
# theta = theta.view(-1, 2, 3)
grid = F.affine_grid(theta, x.size())
x = F.grid_sample(x, grid)
return x

def forward(self, x):
y =self.features(x)
# z1 = self.stn(x)
z1= y.view(-1,1,7*7*512)
r1, (h1, c1) = self.lstm(z1)
s1 = self.classifier2(r1[:,-1,:])
m1 = self.mm(r1[:,-1,:])

m1 = m1.view(-1,2,3)
arr = torch.cuda.FloatTensor([1, 0, 1, 0, 1, 1])
# arr = np.array([1,0,1,0,1,1])
# arr = torch.from_numpy(arr)
# 16 is batchsize we want to make a tensor(batchsize,2,3) to element-wise mul with m1
arr = arr.repeat(16,1)
arr = arr.view(16,2,3)
m1 = torch.mul(m1,arr)
z2 = self.stn(y,m1)
z2 = z2.view(-1,1,7*7*512)
r2, (h2, c2) = self.lstm(z2,(h1,c1))
s2 = self.classifier2(r2[:,-1,:])
m2 = self.mm(r2[:,-1,:])

m2 = m2.view(-1,2,3)
z3 = self.stn(y,m2)
z3 = z3.view(-1,1,7*7*512)
r3, (h3, c3) = self.lstm(z3,(h2,c2))
s3 = self.classifier2(r3[:,-1,:])
return s1,s2,s3
``````

I add the Variable. And then notice that the number of the last batch is not batchsize

``````arr = Variable(arr,requires_grad=False)

``````

another question who can help me.
custom my loss need to use the element from the m1,m2,m3.

How did you solve your first error?
Did you drop the last batch (which was too small)?

As far as I understand you need `mX` for your loss function. Would it be possible to just return them in the `forward` method and calculate the loss?

I solved the first error with add the Variable out of the arr,because the arr we built is a tensor ,but in the process of forward the tensor we should change into the Variable.

yes,we would return the mx in the forward function,how to pick out the element from the tensor (2,3)and count the loss.
now, i plan to use a mask tensor like[[1,0,0],[0,0,0]] and mul with the mk in the custom loss function and then use a sun() to get the element. Is this method effective？

This is one way to achieve this.
You could also use masked_select to get the value, which is probably faster.

Thanks a lot. This is my first topic in the pytorch discuss. I am new to PyTorch.
Thanks.
I also create a new topic in https://discuss.pytorch.org/t/too-many-parameters-cause-single-gpu-memory-out-how-to-solve-it/18973
Could you please have a look?