Unfold second order derivative

autograd
1

Hi,

I’m working on a custom layer which uses the unfold operator.
I would like to compute the second order derivatives (for some hessian vector products…). However I have the following error:

derivative for _thnn_im2col_backward is not implemented

Can anyone tell me how to solve this? Or are there any workaround/hacks to solve this?

thanks for your help

2

Hi,

The problem is that the second derivative of the unfold operation is not implemented.
You would need a function that computes this second derivative or the first derivative to be implemented in a differentiable way with the autograd engine.

3

Hello,

Thanks a lot!

You would need a function that computes this second derivative

How does it work exactly ? Should I write a backward for the backward ? I didn’t find many implementations of such layers.

the first derivative to be implemented in a differentiable way with the autograd engine.

Do you mean that as long as the backward is implemented in a differentiable way, it will automatically handle double diff ?

thank you

4

Hi,

When implemented in python, a forward/backward pair is defined by a Function. You can see here in the doc how in details.
By default, if higher order derivatives are required, the autograd is applied to the backward function that is defined.
You can manually defined second derivatives by using 2 Functions, where the backward of the first one is simply calling the forward of the second one.
You also have a decorator @oncedifferentiable to notify the engine that a backward is implemented in a non differentiable manner. If you ask for second derivatives for such a function, the error you see is raised.

For the cpp implementations, most of them are not implemented in a differentiable manner hence the error you see.

5

Awesome, it really helps ! Thank you

Bruno