I have a binary classification, there are image and variable in the data set, I have an idea for compare image and variable together.
Every time when I pass conv-layer, I want to multiply a weight scalar to all feature map, where weight scalar is computed from a fc-layer.
For example, suppose batch size is 8, there are two tensor x1 and x2, where x1’s size is (8,3,224,224) and x2’s size is (8,16).
import torch from torch.nn import Module, Sequential from torch.nn import Conv2d, BatchNorm2d, ReLU, MaxPool2d, Softmax, Linear import numpy batch_size = 8 x1 = torch.rand(batch_size*3*224*224).view(batch_size,3,224,224) x2 = torch.rand(batch_size*16).view(batch_size,16)
I define conv-layer and fc-layer and compute the output from image and variable.
conv_01 = Conv2d(in_channels=3, out_channels= 9, kernel_size=3, stride=1, padding=1)
linear_02 = Linear(16, 1)
c1 = conv_01(x1) ## torch.Size([8, 9, 224, 224])
c2 = linear_02(x2) ## torch.Size([8, 1])
The problem is write the suitable code like blow.
I want to do like blow
c1[0,:,:,:] = c1[0,:,:,:] * c2[0,0] # 1st data in the mini-batch
c1[1,:,:,:] = c1[1,:,:,:] * c2[0,1] # 2nd data in the mini-batch
c1[2,:,:,:] = c1[2,:,:,:] * c2[0,2]
c1[3,:,:,:] = c1[3,:,:,:] * c2[0,3]
c1[4,:,:,:] = c1[4,:,:,:] * c2[0,4]
c1[5,:,:,:] = c1[5,:,:,:] * c2[0,5]
c1[6,:,:,:] = c1[6,:,:,:] * c2[0,6]
c1[7,:,:,:] = c1[7,:,:,:] * c2[0,7]
output is a (8, 9, 224, 224)
and do more layer like this operation
I already take a look in the Multiply feature map by a learnable scalar - #2 by fmassa.
But this only support when batch size is 1, but in my case, batch size is more than 1.
How to write a suitable code to the forward function in my case?
Thanks a lot.