Trainable input z-vector to generator

[sudocoder]

How do I get the input noise vector to a generator to train, while freezing the generator weights? I’ve been trying to set requires_grad=True for the input, freezing the model weights, and training. However, the input as I print it out does not change over the coarse of training (nor does the model), so I’m clearly missing something.

Do I have to use the input as a parameter into the optimizer when I initialize the optimizer (like in neural style transfer), and if so, how do I do that with MSELoss? Or is there a simpler method?

# Get the noise vector
z = util.get_input_noise() # vector of size 100 with values [0,1)
z = z.detach().requires_grad_()

# Freeze the model
for param in model.parameters():
    param.requires_grad = False

model.eval() # Can I use model in eval mode for this or does this do something weird with backprop?

# Training loop (simplified -- assume certain vars already initialized)
for i in range(n_iters):
    probs = model.forward(z)
    loss = torch.zeros(1, requires_grad=True).to(device)
    loss = loss_fn(probs, target)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

Thanks so much!

Related posts (did not work for me ):

How can i train input (not weight)?

In GAN, i want to find latent vector z corresponding to the real image. One way to do this is to train z to minimize the error between sampled image and real image. However, when i ran the code below, Error message appeared: “ValueError: can’t optimize a non-leaf Variable”. targets # target images of shape (batch_size, 3, 64, 64) z = Variable(torch.randn(batch_size, 100), requires_grad=True).cuda() optim = torch.optim.Adam([z], lr=0.01) # This causes an error. (if i del…

Trainable parameter as input

I can’t understand the difference: prior = Variable(torch.zeros(24), requires_grad=True) prior_param = [nn.Parameter(prior.data)] h_test = torch.autograd.grad(outputs=prior.mean(), inputs=prior_param) #does not work h_test = torch.autograd.grad(outputs=prior.mean(), inputs=prior) #works I want to make a trainable parameter as an input to the computation graph. The error is RuntimeError: One of the differentiated Variables appears to not have been used in the graph

Neural Style transfer: Neural Transfer Using PyTorch — PyTorch Tutorials 2.1.1+cu121 documentation

[JuanFMontesinos (Juan Montesinos)]

I’d say you have to parse the input to the optimizer. Are you doing so?

[sudocoder]

Yes, that’s what I was doing incorrectly - the optimizer needs to take in the input.