Hi, my question is how to train the encoder with other FC classifier?
In my case, I have to extract two feature vectors from two images in each batch, then let these two vectors do the element-wise subtraction to generate a new feature vector.
finally, input this new feature vector to other FC layer to classify it.
is there any wrong in my code, because the testing result is not good in my expectations…
In my code, I set optimizer below:
opt_2 = optim.Adam([{'params': encoder.parameters()}, {'params': fc_2.parameters()}], lr=_args.init_lr, weight_decay=1e-6)
In training stage, I write:
vector_1 = encoder(image_1)
vector_2 = encoder(image_2)
outputs_2 = fc_2((vector_1 - vector_2))
# backward
_opt_2.zero_grad()
loss_2 = F.cross_entropy(outputs_2, labels)
loss_2.backward()
_opt_2.step()