Torch has not attribute load_state_dict?

Losspost · July 26, 2018, 7:29pm

Hi. I am trying to load a model with:

import torch

import pyautogui as mouse
import cv2 


from ScreenRecorder import Record,IniRecord,Frame


def start(model):
    sc_ini = Frame()
    monitor = sc_ini.get()
    sc = IniRecord(monitor,1.6)
    while True:
        frame = sc.getFrame()
        cv2.imshow('frame',frame)
        output_xy,output_click = model.forward(frame)
        #print(output_xy,output_click)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            print("break")            
            break
        

model = torch.load_state_dict(torch.load('Model/model_save'))

But it says


  File "D:/Nextcloud/Python/Gamebot/Bot.py", line 31, in <module>
    model = torch.load_state_dict(torch.load('Model/model_save'))

AttributeError: module 'torch' has no attribute 'load_state_dict'

ptrblck · July 26, 2018, 7:31pm

You have to call it on your model:

model.load_state_dict(torch.load(...))

Losspost · July 26, 2018, 7:32pm

Is there another way to load it? because else i have to import the whole model (network) in my file.
And it still complains

.    output_xy,output_click = model(frame)

TypeError: 'NoneType' object is not callable

ptrblck · July 26, 2018, 8:17pm

Did you try to assign the model? (model = model.load_state_dict())
Just skip it and load the state_dict directly:

model = MyModel()
model.load_state_dict(torch.load(...))

Losspost · July 26, 2018, 8:27pm

If i test the model. Do i have to use the batch size as input as well or can i put single images in as well?

ptrblck · July 26, 2018, 8:30pm

You can use single images, but have to make sure the tensor contains the batch dimension:

image = ... # shape = [channels, height, width]
image = image.unsqueeze(0) # shape = [1, channels, height, width]
output = model(image)

Losspost · July 26, 2018, 8:38pm

Hmm is it just not enough training or why does it always ouputs 0 and never tries 1 ?

ptrblck · July 26, 2018, 8:39pm

I don’t know what kind of model and use case you are working on, but it might be worth starting a new thread if you encounter any errors to keep this topic clean.

saba · November 7, 2019, 1:48am

Hi ptrblck

I saved my trained Nets on GPU and now wants to use them on CPU.

I read your comments but still have same problem as (AttributeError: ‘list’ object has no attribute ‘load_state_dict’
My code is:

            checkpoint = torch.load(Path1,map_location=torch.device('cpu'))

model.load_state_dict(torch.load(Path1,map_location=torch.device(‘cpu’))[‘model_state_dict’])

            model.load_state_dict(torch.load(Path1)['model_state_dict'])
            optimizer.load_state_dict(torch.load(Path1,map_location=torch.device('cpu'))['optimizer_state_dict'])

ptrblck · November 7, 2019, 1:57am

How did you create your model instance?
Based on the error message it seems it’s a list instead of a subclass of nn.Module.

saba · November 7, 2019, 2:04am

Hi , Many thnaks fo ryour reply. muy model is
import torch
import torchvision
import torchvision.transforms as transforms
from torch.utils.data import TensorDataset, DataLoader
import torch.optim as optim
import torch.nn as nn
from torch.utils.data.dataset import random_split
from torch.nn import functional as F
import matplotlib.pyplot as plt
from torch.autograd import Variable

class ConvNet(nn.Module):
def init(self,numf1,numf2,fz1,fz2,nn2,nn3):
super(ConvNet, self).init()
self.numf1=numf1
self.numf2=numf2
self.fz1=fz1
self.fz2=fz2
self.nn2=nn2
self.nn3=nn3
self.layer1 = nn.Sequential(nn.Conv3d(1, self.numf1, kernel_size=self.fz1, stride=1, padding=2),nn.ReLU(),nn.MaxPool3d(kernel_size=2, stride=2))
self.layer2 = nn.Sequential(nn.Conv3d(32,self.numf2, kernel_size=self.fz2, stride=1, padding=2),nn.ReLU(),nn.MaxPool3d(kernel_size=2, stride=2))
self.fc1 = nn.Linear(3072, self.nn2) ##3027
self.drop_out = nn.Dropout(0.3)
print(“Dropout”)
self.fc2 = nn.Linear( self.nn2, self.nn3) # FULLY CONNECTED LAYERS
self.fc3 = nn.Linear( self.nn3, 1) # FULLY CONNECTED LAYERS

    self.sigmoid = nn.Sigmoid()
    
def forward(self, x):

print(“here: {}”.format(x.shape))

   # print(type(x))

x=np.expand_dims(x,axis=0)

print(“xthen”,x.shape)

x=x.astype(int)

x=torch.from_numpy(x)

    x=x.unsqueeze(1).float()

print(type(x))

    #print(x.shape)
    out = self.layer1(x)
    
   # print(out.shape)
  #  print(out)
    out = self.layer2(out)

print(out.shape)

    out = out.view(out.size(0), -1)

print(out.shape)

    out = self.fc1(out)
    out = self.fc2(out)
    out = self.fc3(out)
    out = self.sigmoid(out)

print(“outsize”,out.shape)

    return out

saba · November 7, 2019, 2:06am

and I saved it in this way:

Path=root_dir1+’/Fold_’+str(FoldNum)+‘NumDarw=’+str(NumDraw)+‘Iteration’+str(Iteration)+".pth"
checkpoint = {‘model_state_dict’: model.state_dict(),‘optimizer_state_dict’: optimizer.state_dict()}
# print(checkpoint)
torch.save(checkpoint,Path) ## save for each 10 iteration

saba · November 7, 2019, 2:07am

one time I run my code when I saved the trained Nets on CPU. but now I am loading the trained Nets from GPU to use on CPU and get this error

ptrblck · November 7, 2019, 2:08am

Thanks for the code. Could you post the code you are using to restore the model?

saba · November 7, 2019, 4:33am

        model=[]
        optimizer=[]
        TargetWholev2=[]
        for Iteration1 in range(9):
            Path1=root_dir2+'/Fold_'+str(FoldNum)+'NumDarw='+str(NumDraw)+'Iteration'+str(Iteration1+1)+'.pth'

            
            checkpoint = torch.load(Path1,map_location=torch.device('cpu'))

            model.load_state_dict(torch.load(Path1,map_location=torch.device('cpu'))['model_state_dict'])


            optimizer.load_state_dict(torch.load(Path1,map_location=torch.device('cpu'))['optimizer_state_dict'])

ptrblck · November 7, 2019, 6:19am

You are currently initializing model and optimizer as an empty Python list, which creates this error.
Initialize both as you have done in your training script, i.e.:

model = ConvNet(...)
optimizer = optim.SGD(model.parameters(), ...)

# Now load the state_dicts
model.load_state_dict(...)
optimizer.load_state_dict(...)

saba · November 7, 2019, 6:38am

I used this but give me this error:(Error(s) in loading state_dict for ConvNet:
size mismatch for layer1.0.weight: copying a param with shape torch.Size([32, 1, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([2, 1, 3, 3, 3]).
size mismatch for layer1.0.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([2]).)

model = ConvNet(2,64,3,3,300,20)
optimizer =torch.optim.Adam(model.parameters(), lr=LR)
TargetWholev2=[]
for Iteration1 in range(9):
Path1=root_dir2+’/Fold_’+str(FoldNum)+‘NumDarw=’+str(NumDraw)+‘Iteration’+str(Iteration1+1)+’.pth’

checkpoint = torch.load(Path1)

            checkpoint = torch.load(Path1,map_location=torch.device('cpu'))

            model.load_state_dict(torch.load(Path1,map_location=torch.device('cpu'))['model_state_dict'])


            optimizer.load_state_dict(torch.load(Path1,map_location=torch.device('cpu'))['optimizer_state_dict'])

ptrblck · November 7, 2019, 6:39am

The error points to different shapes of your parameters, which means that you’ve initialized the model in a different way.
Could you check, how you’ve initialized the ConvNet before saving the state_dict and use the same arguments?

saba · November 7, 2019, 6:42am

yes you are right 2 is 32 i missed the 3
I really appreciate your time

saba · November 7, 2019, 6:46am

excuse me, the time of training, despite the CNN was shallow was 4 days for just one draw. How I can speed up the GPU? The number of workers in data loader are important for speed?