# Compare correct gradients of PyTorch with own Implementation

Hi there!

I am trying to build some PyTorch functions, like convolution 2D, from scratch using numpy.
To achieve this, I need to check if my current implementations are correct, before I move on to the next implementation. I got the forward pass correct, however, for the backward pass it is not so simple to compare the output of PyTorch with mine.

Basically, what I am trying to compare are the gradients of the loss w.r.t to the weighst of the convolution (without bias for now). Let me show you how I wanted to achieve this:

``````# create some random image for the input for the convolution and one for calculating the loss
input_numpy = np.random.uniform(size=(3, 3, 64, 64))
input_tensor = torch.from_numpy(input_numpy).float()
y_true = torch.randn(size=(3, 3, 30, 30))

# init the pytorch model and transfer the weights to 'my' model
torch_conv2d = nn.Conv2d(3, 3, 6, 2, 0, 1, bias=False)
my_conv2d = Conv2D(3, 3, 6, 2, 0, 1, bias=False)
conv2d.weights.data = torch_conv2d.weight.data

# calculate the forward pass for both models, those are equal (I checked that already)
torch_pred = torch_conv2d(input_tensor)
my_pred = conv2d.forward(input_numpy)
``````

Now, to make my backpass function work, I need the gradient of the loss w.r.t to the predictions.
If I understand it correctly, the loss function (as an example)

``````criterion = nn.MSELoss()
``````

Does not return the gradients. I can only get the gradients of the weights by calling

``````loss = criterion(torch_pred, y_true)
loss.backward()
``````

``````from torch.autograd import grad as autograd

``````

Am I correct, that the first item of grad (grad), are the gradients of the MSE Loss w.r.t to the prediction, and thus is the required input for my backward pass?

This line of code:

explicitly detaches the outpute tensor and recreates a new deprecated `Variable`.
I would assume you want to check the weight and data gradients as seen here:

``````conv = nn.Conv2d(3, 3, 3)
x = torch.randn(1, 3, 24, 24, requires_grad=True)
out = conv(x)