Designing a network pytorch

Hi I want to design a network that reconstruct a frame using the previous frame. I used conv layers with simple mse loss but it doesnt work. would you mind suggest me an architecture to help me?