Fmod or remainder: RuntimeError: the derivative for 'other' is not implemented

1

Hello everyone

I am using fmod or remainder to compute a sawtooth waveform, where the divisor argument ‘other’ is computed by some layers (in this case it is for defining the sawtooth frequency), thus it requires gradient.
When performing backpropagation it gives the following error for fmod or remainder:

RuntimeError: the derivative for ‘other’ is not implemented

Is there any workaround please ?

The input argument is a torch.arange(size) which is then shaped as a saw thanks to the modulo operation.

Thanks !

2

Hi,

I’m afraid we don’t have a gradient formula for this. What would be d(fmod(x, y)) / dy ?

3

Hi,

x = torch.arange has a constant derivative/slope, however through the modulo, it becomes non defined at the modulo jump, which makes it non differentiable in each of these points. So as it is, indeed there is an issue for gradients I guess.

I alternatively tried to generate sawtooth as wavetables, by using interpolate to rescale a given sawtooth at a reference frequency to any other frequency, feeding the ratio of the wavetable reference frequency and the predicted f0.

However, it seems as well that interpolate does not accept a gradient variable in the scale factor. Is it correct ?

In this case it seems as well that backpropagation is not possible through the interpolate scale factor.

4

The scale factor is a multiplier for the size. So it is not differentiable either indeed I’m afraid.

5

I am not very sure regarding the use case but you can try this function. It seems to work as a very good approximation and everything seems differentiable. I have just made some back of the envelope calculations regarding how gradient might work in such a situation. Hope you find it useful.

fmod( x, a ) ≈ ( a / π ) arctan( tan( π ( x / a - 0.5 ) ) ) + a / 2

link to a small demonstration of the function: Differentiable approximate modulus | Desmos

image
image1366×460 66.9 KB

2 Likes
6

Not sure if it is helpful but here is my implementation of the sawtooth function using pytorch only.

def sawtooth_wave(t, period = 0.5, amplitude = 1.0, phase = 0.0, move = 0.0):
    x = (t + phase) / period
    return amplitude * (x - th.floor(x)) + move

More details and examples can be seen here: Sawtooth wave function purely in pytorch · GitHub