First I tried training a CNN with the function f(x)=x.

This worked fine.

Then my new function is f(x)=x, x<0.5

f(x)=0, x>0.5

This did not work, network could not figure it out.

Is that a weakness of neural network that it cannot model non-differentiable function, or what.

Thanks Matt