Optimization of inputs

[alwynmathew]

1. Add input tensor to the optimizer:

input.requires_grad = True
optimizer = optim.Adam([input], learning_rate)

Note that Variables don’t exist anymore in the newest versions on PyTorch.

In case, if you want to add input tensor along with model parameters to optimizer:

params_to_train = [input]
params_to_train += list(model.parameters())
optimizer = optim.Adam(params_to_train, learning_rate)

2. Yes, adding the input to the optimizer will make input trainable. Make sure that grad of input tensor is turned on.

This may help you:

Add one tensor from dataloader to optimiser

I want to optimize my init_tensor instead of optimizing the model. My data loader train_dataloader gives the initial value of init_tensor. model.eval() for inputs in train_dataloader: init_tensor = inputs["init_tensor"] optimizer = optim.SGD(init_tensor, learning_rate) output = model(init_tensor) loss = loss_fn(output) model.zero_grad() loss.backward() optimizer.step() Defining the optimizer inside the training loop doesn’t look right. Will this hinder opti…