Network parameter setting using Varitional Optimization

(Arash Javanmard) #1

Hi there,

i am trying to optimise a CNN using methods of variational optimisation David Barbr blog on VO., here you draw samples from a gaussian-dist and assign them to the network’s weights.

till now i did byjust looping through the net parameters:

            for param in list(net.parameters()):
                 param_length =
                sample_ = sample[idx: idx + param_length]
                sample_ = sample_.view(

                idx += param_length

I would like to know if there is some better way to assgin weights to a network manually. btw. i also have read this question Assign variable to model parameter.


It the variational optimisation something like a weight initialization scheme?
If so, you could create a function with your code and use apply on your model.

def weights_init(m):
    if isinstance(m, nn.Conv2d):
        "Your code here e.g.:"


(Arash Javanmard) #3

Well actualy you assign new params at each iteration, not just once at initialization, but nevertheless your approach is also interesting, since i can use it to assign at each iter. :slight_smile: