How to convert the tensorflow code to pytorch?

1

Hi everyone,

I am trying to convert the following code in TensorFlow to PyTorch. But I couldn’t get the point of how to write the placeholders and tf.variable_scope() in PyTorch. Is there any equivalent function for those?

class Dense(Layer):
“”“Dense layer.”""
def init(self, input_dim, output_dim, placeholders, dropout=0., sparse_inputs=False,
act=nn.ReLU(), bias=False, featureless=False, **kwargs):
super(Dense, self).init(**kwargs)

    if dropout:
        self.dropout = placeholders['dropout']
    else:
        self.dropout = 0.

    self.act = act
    self.sparse_inputs = sparse_inputs
    self.featureless = featureless
    self.bias = bias

    # helper variable for sparse dropout
    self.num_features_nonzero = placeholders['num_features_nonzero']

    with tf.variable_scope(self.name + '_vars'):
        self.vars['weights'] = glorot([input_dim, output_dim],
                                      name='weights')
        if self.bias:
            self.vars['bias'] = zeros([output_dim], name='bias')

    if self.logging:
        self._log_vars()

def _call(self, inputs):

    x = inputs

    # # dropout
    if self.sparse_inputs:
        x = SparseDropout(x, 1-self.dropout, self.num_features_nonzero)
    else:
        x = nn.Dropout(x, 1-self.dropout)

    # transform
    output = dot(x, self.vars['weights'], sparse=self.sparse_inputs)
    
    # bias
    if self.bias:
        output += self.vars['bias']

    return self.act(output)
2

In Pytorch, you do not deal with maintaining different scopes as in TF.
Here, the network layers (torch.nn.*) are included with their own parameters internally.
I strongly suggest you to go through Pytorch tutorials:

3

Oh I see, thanks :slight_smile: also, there is no placeholders in PyTorch, right?

4

You are right.
There is no placeholders in pytorch, as it is not building static-graph like TF.