Output of Model is nan every time

That seems odd. I can’t find anything in your case that would cause your error (expect that there would be something weird with the data). Have you tried to overfit one a single example before training on your entire dataset?

Also I think you could make your network a little cleaner (or I’m not sure why you use act):

class my_network(nn.Module):
    def __init__(self, num_classes):
        super(my_network, self).__init__()

        self.layer1 = nn.Linear(1 * 60 * 80, 50 * 30 * 40)
        self.layer2 = nn.Linear(50 * 30 * 40, 70 * 10 * 15)
        self.layer3 = nn.Linear(70 * 10 * 15, 90 * 5 * 8)
        self.layer4 = nn.Linear(90 * 5 * 8, 80)
        self.layer5 = nn.Linear(80, num_classes)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        x = F.relu(self.layer1(x))
        x = F.relu(self.layer2(x))
        x = F.relu(self.layer3(x))
        x = F.relu(self.layer4(x))
        out = self.layer5(x)
        return out

model = my_network(num_classes=52)

I was also thinking that since you’re training on images it would probably be beneficial from a performance viewpoint to use convolutional neural networks rather than a fully connected (but perhaps something you can do after you’ve identified the error).