Network parameter setting using Varitional Optimization


(Arash Javanmard) #1

Hi there,

i am trying to optimise a CNN using methods of variational optimisation David Barbr blog on VO., here you draw samples from a gaussian-dist and assign them to the network’s weights.

till now i did byjust looping through the net parameters:

            for param in list(net.parameters()):
                 param_length = np.prod(list(param.size())).astype(np.int)
                sample_ = sample[idx: idx + param_length]
                sample_ = sample_.view(param.data.size())

                param.data = sample_.data
                idx += param_length

I would like to know if there is some better way to assgin weights to a network manually. btw. i also have read this question Assign variable to model parameter.


#2

It the variational optimisation something like a weight initialization scheme?
If so, you could create a function with your code and use apply on your model.

def weights_init(m):
    if isinstance(m, nn.Conv2d):
        "Your code here e.g.:"
        xavier(m.weight.data)
        xavier(m.bias.data)

model.apply(weights_init)

(Arash Javanmard) #3

Well actualy you assign new params at each iteration, not just once at initialization, but nevertheless your approach is also interesting, since i can use it to assign at each iter. :slight_smile: