How to translate this pytorch code to c++(libtorch)?

C++
1

I’m unable to figure out how to overide these functions in c++. What’s the right way to translate this?

class DenseLayer(nn.Linear):
    def __init__(self, in_dim: int, out_dim: int, activation: str = "relu", *args, **kwargs) -> None:
        self.activation = activation
        super().__init__(in_dim, out_dim, *args, **kwargs)

    def reset_parameters(self) -> None:
        torch.nn.init.xavier_uniform_(
            self.weight, gain=torch.nn.init.calculate_gain(self.activation))
        if self.bias is not None:
            torch.nn.init.zeros_(self.bias)
1 Like
2

If you’re just looking for a function to init an inputted linear layer with custom weight and bias initialization methods, you can do so like this with a function:

torch::nn::Linear reset_parameters(torch::nn::Linear layer) {

    torch::NoGradGuard noGrad;
    torch::nn::init::xavier_uniform_(layer->weight, torch::nn::init::calculate_gain(torch::enumtype::kReLU));
    torch::nn::init::zeros_(layer->bias);

    return layer;

}

You can see documentation for libtorch’s torch::nn::init functions here:Torch::nn::Init Documentation

I use torch::NoGradGuard noGrad to initialize the layer’s weights and biases, the C++ equivalent to Pytorch’s with torch.no_grad():, as according to this documentation: Autograd mechanics — PyTorch 1.11.0 documentation, The implementations in torch.nn.init also rely on no-grad mode when initializing the parameters as to avoid autograd tracking when updating the initialized parameters in-place. You can remove that line if you know there’s no need for it.

Hopefully, this helps.

1 Like