Probably the OP is referring to this diagram in the paper
If that the case then as the legend says the network is fitting a superposition of two sine waves with frequencies 4 and 14 and not just one. Additionally, at epoch 50 they claim the network has learned the lower freq component and only at epoch 22452 fits completely the superposition of sine functions