Hi all,
I have a use case where I need to optimize over dual variables of the size of my dataset. Namely, I would like to load (input, duals, targets) at each batch, and then optimize both over duals and my model’s parameters. My loss function takes (model(input), duals, targets) as input. At the end of the optimization, I would like to update duals in my dataset.
How can I do this?
Thanks!
I don’t understand your use case clearly, especially the dual variables of the size of the dataset.
If the dual variable you mentioned is learnable, you could set requires_grad=True and add them to the optimizer.
Does this method fit your use case?
I’m optimizing the dual of a convex optimization problem, that yields an unconstrained optimization problem of the above form. My dual variable is of size n_data x m since originally I had n_data x m constraints in my primal constrained convex problem and each row of my dual variable corresponds to a specific datapoint. I would prefer batching these variables with the datapoints to allow for easy shuffling.