How to add new feature for a model？(point cloud)

vision
#1

I am handle 3d point cloud.The data has type as: xyz intensity.I just use the xyz information in a pointnet model.It works.But I wonder how to apply ‘intensity’ in pointnet. The definition of the network ispointnet

class PointNetDenseCls(nn.Module):
    def __init__(self, num_points = 10000, k = 8):
        super(PointNetDenseCls, self).__init__()
        self.num_points = num_points
        self.k = k
        self.feat = PointNetfeat(num_points, global_feat=False)
        self.conv1 = torch.nn.Conv1d(1088, 512, 1)
        self.conv2 = torch.nn.Conv1d(512, 256, 1)
        self.conv3 = torch.nn.Conv1d(256, 128, 1)
        self.conv4 = torch.nn.Conv1d(128, self.k, 1)
        self.bn1 = nn.BatchNorm1d(512)
        self.bn2 = nn.BatchNorm1d(256)
        self.bn3 = nn.BatchNorm1d(128)
def forward(self, x):
        batchsize = x.size()[0]
        x, trans = self.feat(x)
        x = F.relu(self.bn1(self.conv1(x)))
        x = F.relu(self.bn2(self.conv2(x)))
        x = F.relu(self.bn3(self.conv3(x)))
        x = self.conv4(x)
        x = x.transpose(2,1).contiguous()
        x = F.log_softmax(x.view(-1,self.k), dim=-1)
        x = x.view(batchsize, self.num_points, self.k)
        return x, trans

I dont want to change the extracted features.May I add a new parameter in forward and combine the result x with intensity?just like that:

def forward(self, x,intensity):
        batchsize = x.size()[0]
        x, trans = self.feat(x)
        x = F.relu(self.bn1(self.conv1(x)))
        x = F.relu(self.bn2(self.conv2(x)))
        x = F.relu(self.bn3(self.conv3(x)))
        x = self.conv4(x)
        x = x.transpose(2,1).contiguous()
        x = F.log_softmax(x.view(-1,self.k), dim=-1)
        x = somennlayer(x,intensity)
        x = x.view(batchsize, self.num_points, self.k)
        return x, trans

Is there any layer in pytorch to structure the relationship between the two parameters?

#2

You could create a second branch for the intensity input and concat the x features with your intensity features.
Then you could pass these concatenated features into a classifier, e.g. linear layers with non-linearities in-between.

#3

Thank you! I will try this idea right away. I will upload the implementation code as soon as possible.

#4

I add the new feature in my network. It works. Thanks again.

    def forward(self, x,intensity):
        batchsize = x.size()[0]
        x, trans = self.feat(x)
        x = F.relu(self.bn1(self.conv1(x)))
        x = F.relu(self.bn2(self.conv2(x)))
        x = F.relu(self.bn3(self.conv3(x)))
        x = self.conv4(x)
        x = x.transpose(2,1).contiguous()
        x = F.log_softmax(x.view(-1,self.k), dim=-1)
        x = x.view(batchsize, self.num_points, self.k)
        x = x.transpose(2,1).contiguous()
        intensity = intensity.view(batchsize,1,self.num_points)
        x = torch.cat((x,intensity),1)
        x = self.bn4(self.conv5(x))
        x = x.transpose(2,1).contiguous()
        x = F.log_softmax(x.view(-1,self.k), dim=-1)
        x = x.view(batchsize, self.num_points, self.k)
1 Like