RNN all_weights

chssch · August 25, 2017, 7:16am

Hi,

i have problems to understand a piece of code in Pytorch:

pytorch/pytorch/blob/6648677acf1b653a2c80ee790dd85dc29664f6b0/torch/nn/modules/rnn.py#L104


    fn.x_descs = cudnn.descriptor(any_param.new(1, self.input_size), 1)
    fn.rnn_desc = rnn.init_rnn_descriptor(fn, handle)


    # Allocate buffer to hold the weights
    num_weights = rnn.get_num_weights(handle, fn.rnn_desc, fn.x_descs[0], fn.datatype)
    fn.weight_buf = any_param.new(num_weights).zero_()
    fn.w_desc = rnn.init_weight_descriptor(fn, fn.weight_buf)


    # Slice off views into weight_buf
    params = rnn.get_parameters(fn, handle, fn.weight_buf)
    all_weights = [[p.data for p in l] for l in self.all_weights]


    # Copy weights and update their storage
    rnn._copyParams(all_weights, params)
    for orig_layer_param, new_layer_param in zip(all_weights, params):
        for orig_param, new_param in zip(orig_layer_param, new_layer_param):
            orig_param.set_(new_param.view_as(orig_param))


    self._data_ptrs = list(p.data.data_ptr() for p in self.parameters())


def _apply(self, fn):

Where is self.all_weights initialized or get its values here? I can only find self._all_weights with an underscore.

Thanks you for any pointers,

Christoph

chssch · August 25, 2017, 7:18am

Ah after sending this I directly found the property:

github.com

pytorch/pytorch/blob/master/torch/nn/modules/rnn.py#L203


        self.input_size,
        self.hidden_size,
        num_layers=self.num_layers,
        batch_first=self.batch_first,
        dropout=self.dropout,
        train=self.training,
        bidirectional=self.bidirectional,
        batch_sizes=batch_sizes,
        dropout_state=self.dropout_state,
        flat_weight=flat_weight
    )
    output, hidden = func(input, self.all_weights, hx)
    if is_packed:
        output = PackedSequence(output, batch_sizes)
    return output, hidden


def __repr__(self):
    s = '{name}({input_size}, {hidden_size}'
    if self.num_layers != 1:
        s += ', num_layers={num_layers}'
    if self.bias is not True:

Thanks,

#close