I have an app that uses facenet-pytorch to generate embedded vectors for images of faces. The model I’m using is InceptionResnetV1 trained on vggface2. The output layer of this model is a 512d tensor.
I’m trying to write a TFGSM attack that will allow me to send an Image to the resnet model and get back an embeddeing vector that is simillar to embedded vector of someone specific.
For example, I took 10 images of Eylon Musk , passed them to the model and got 10 embedded vectors. I created one average embedded vector from all of those vectors and this new vector is an embedded vector that represents Eylon Musk. I have an image of Barak Obama` face. I want to run an TFGSM attack that will change some pixels in the image , so that when I’ll forward this image in the resnet model I will get a vector that is simillar to the vector of Eylon Mask(L2 distance will be less than some threshold…).
I got 2 questions :
I’m trying to understand what LOSS function should I use. I checked CosineSimilarity and CosineEmbeddingLoss . I’m not sure why I need the y parameter in COsineMbeddingLoss since I got 2 vectors… And what is the main difference between them ?
My attack code :
def TFGSM(image:torch.Tensor, model, target_vector,epsilon):
loss = nn.CosineSimilarity()
#loss = nn.CosineEmbeddingLoss()
loss = loss(model(image), target_vector)
#loss = loss(model(image), target_vector,torch.Tensor([[1]*512]))
model.zero_grad()
loss.backward()
data_grad = image.grad.data
sign_data_grad = data_grad.sign()
image_with_noise = image + epsilon*sign_data_grad
image_with_noise = torch.clamp(image_with_noise, 0, 1)
return image_with_noise
It doesnt matter If I’m using the CosineEmbeddinngLoss or the CosineSimilarity I’m getting the following exception in both case :
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
Tried adding retain_graph=True but it didnt help.