I want to optimize a few output of my model's layers, how should i do?

1

I have a ResNet18 network and have obtained a small set of specific parameters at each layer. Using a hook function, I have obtained the output of each layer. I want to optimize the model’s input to achieve specific values for their corresponding outputs. For example, in the ‘layer1.0.conv1’ layer (Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)), I obtained the parameter [x][y][m][n] and the output of this layer. How should I optimize the input to achieve a specific value for the output corresponding to this parameter [x][y][m][n]? Please note that I want to optimize the input, not the parameter. Also, in a convolutional layer, one parameter may correspond to multiple outputs.

2

Does making your inputs requires_grad=True work? You may also need to update your update your optimizer to include the inputs e.g. something like optim.SGD(model.parameters() + [inputs])

3

YES,I already use requires_grad=True . THe problem is how to find the specfic target in output corresponding to the parameter [x][y][m][n]?

4

Could you do something like this?

import torch
import torch.nn as nn
import torch.nn.functional as F

a = torch.rand(16, 10, requires_grad=True)
seq = nn.Sequential(nn.Linear(10, 10), nn.Linear(10, 10))

intermediate = [None]

def fn(m, input, output):
    intermediate[0] = output
seq[0].register_forward_hook(fn)
seq[0].weight.requires_grad_(False)
seq[0].bias.requires_grad_(False)

out = seq(a)

target = torch.rand_like(intermediate[0])
loss = F.l1_loss(intermediate[0], target)
loss.backward()

print(a.grad)
5

it seems you dont understand my question.

import torch
import torch.nn as nn
import torch.nn.functional as F

a = torch.rand(3,16, 10, requires_grad=True)
net = nn.Sequential(
nn.Conv2d(3, 16, 3, padding=1),
nn.ReLU(),
nn.Conv2d(16, 32, 3, padding=1),
nn.ReLU(),
nn.Conv2d(32, 10, 3, padding=1),
nn.ReLU()
)
intermediate = [None]
weights=[None]
def fn(m, input, output):
intermediate[0] = output
weights[0]=m.weight

# print('weight',m.weight.shape)

net[2].register_forward_hook(fn)
out = net(a)
print(intermediate[0].shape)

weight=weights[0][0][0][0][0]

What is the index of the output corresponding to this parameter?

If I cannot obtain the index, I cannot optimize my input using the output of this layer.

target_output=intermediate[0].[?][?][?]

optimizer=torch.optim.SGD(a,lr=0.1)

loss=(target_output-10)**2
loss.backward()
optimizer.step()

I am grateful for you help.

6

If I understand correctly, by “correspond” you mean “if one perturbs the parameter but some amount, which indices of the output would change”. If that is the case, there are at least two ways:

You could derive it potentially by looking at what convolution does, e.g. given weight[c_out][c_in][i][j]. I perhaps the following view of output is affected: output[:, c_out, i, h - k_h - i + 1, w - k_w - i + 1] where h, w are the height and width of your input, k_h, k_w are the dimensions of the kernel, etc.

A super dumb way to do this is just to set the particular index of your parameter to inf or nan and observe which outputs become inf or nan. You could also use this dumb method as a test for the programmatic approach.