Validation loss is much higher than training loss on first epoch and after

I tried to implement the FinerPCN model in PyTorch (the original code is in TF) for point cloud completion, but modified it to accept tensor size [B, N, 6] instead of [B, N, 3].
The problem is that I’m getting a validation error of 833.99 on the first epoch and a training loss of 0.6.
After that training loss is decreasing, but validation loss jumps higher / lower.
What could be the cause of that?

Here’s my model: