How to find PyTorch source for specific functionality?

1

What is the recommend method for searching the PyTorch source code?

For example I’m attempting to find the source for Dropout. I begin with the doc :
https://pytorch.org/docs/stable/_modules/torch/nn/modules/dropout.html

Can see dropout is implemented as :

class Dropout(_DropoutNd):
@weak_script_method
def forward(self, input):
return F.dropout(input, self.p, self.training, self.inplace)

I wish to view how dropout is implemented but I’m unable to find the source for :
F.dropout(input, self.p, self.training, self.inplace)

From viewing https://pytorch.org/docs/stable/_modules/torch/nn/functional.html#dropout2d it appears to be implemented as _VF.feature_dropout(input, p, training)) but where is _VF.feature_dropout(input, p, training)) implemented in source ?

I’ve searched for _VF.feature_dropout(input, p, training)) the results of which are : https://github.com/pytorch/pytorch/blob/a47749cb28bb630bc99601a64e24ad453d9583a9/torch/nn/functional.py but this does not show how dropout is implemented.

How to find the source for dropout and in general are there recommendations for navigating the PyTorch source https://github.com/pytorch/pytorch ?

4 Likes
2

Hi Adrian,
This will not be a good answer to your question, but I was looking for something similar and had basically the same set of questions.
I found this dropout.py
It is a couple years old, but has the code. It is sitting under torch/nn/_functions
Its not there in the current repository. I wonder if it has something to do with the refactoring they did that moved state to ‘the graph’. Just a guess. Post if you find the current location!

1 Like
3

Have you checked what is _VF?
Very strange:

import torch
import sys
import types


class VFModule(types.ModuleType):
    def __init__(self, name):
        super(VFModule, self).__init__(name)
        self.vf = torch._C._VariableFunctions

    def __getattr__(self, attr):
        return getattr(self.vf, attr)

sys.modules[__name__] = VFModule(__name__)

My guess is _VF will navigate you to a specified module.
Also notice there is a module called dropout.cpp:https://github.com/pytorch/pytorch/blob/d1623f4cc9334ade7376b3685bb91d3071e3b418/torch/csrc/api/src/nn/modules/dropout.cpp

And most of PyTorch’s computation internals are written in C++.

4 Likes