How do I create a layer in a model with separate weights?

vision
1

How do I create a layer with separate weights? I’m trying to have code that can “interpolate” points in a 2d plane, but interpolated points of the outputs just “stack” on the same input points, and I thought separating the weights would help reduce the stacking that I observe.

2

Can you elaborate on what did you mean by this?
It will be great if you can post some code examples on what have you tried.

3

I’m currently attempting to reimplement at first, a 2d version of Pu-net, but I’ve given up on that and went straight to a “cut-down” Pointnet Version. From the PU-net paper:

Hence, we
further add another convolution (with separate weights) for
each feature set. Since we train the network to learn the r
different convolutions for the r feature sets, these new features can include more diverse information, thus reducing
their correlations.

In the implementation, they seemed to be not adding a relu activation functions, so I tried that, but that didn’t seem to have the desired effect.

4

In my understanding (from Fig.1 of the PU-Net paper), they are applying r different convolutional pipelines (each pipeline has 2 conv layers with its own weights) to get r different (non-linear) affine projections of point-features. i.e., N x C goes to r * (N x C2). I do not understand their reason for adding C_2 layer. However, its adding more capacity (parameters) to the model.

Implementation could be:


def __init__(...):
    self.expansion_set = nn.ModuleList()
    for i in range(r):
        self.expansion_set.append(nn.Sequential(nn.Conv1d(C, C1, kernel_size=1), 
                                           nn.ReLU(),
                                           nn.Conv1d(C1, C2, kernel_size=1)))

    ...

def forward(...):
    ...
    r_sets = []
    for i in range(r):
        r_set.append(self.expansion_set[i](features))