Strange 'Inverse' gradient flow

Hi there,

I am a newbie in Pytorch and I have got some problem in my current project.

I am working on depth map generation from 2D images and the source code is as follows:
https://github.com/shuuchen/depth_eigen/blob/master/test.ipynb

In the first plot:


I plot the gradient flow of each layer with reference to Check gradient flow in network

However, the gradient seems to flow ‘inversely’: the gradients of late layers are much smaller than early layers, which is opposite to Check gradient flow in network

The second plot seems to be normal.

Anyone can help explain ? Thanks !