RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #2 'weight' in call to _thnn_conv2d_forward

I am trying to implement a small CNN with PyTorch .I have changed the model.named_parameters() with other numbers, nothing else is changed. Could someone please help me with this issue? with the following logic, I updated the new weights.

count=0

for name,param in model.named_parameters():
    print(type(param.data))
    print("before:",param.data)
    #sizes is the sizes of the weights
    #un_params is my new wieghts list of all layers
    if(len(sizes[count])==1):
        param.data=torch.from_numpy(un_params[count][0])
    else:
        param.data=torch.as_tensor(un_params[count])
    print(type(param.data))
    print("after :",param.data)
    count+=1

Numpy uses float64 as the default data type. If you load or create an array without changing this dtype, torch.from_numpy() will respect the dtype and will also return a DoubleTensor.
You should solve this issue by using:

param = torch.from_numpy(un_params[count][0]).float()

PS: the usage of the .data attribute is discouraged as it might yield unexpected side effects.
Instead you should wrap the assignment in a with torch.no_grad() block and directly assign new nn.Parameters.

3 Likes

thank you, .float() solved the problem! I’ll look into how to wrap the assignment using 'with torch.no_grad() ’ but could you please elaborate it if possible? My goal is to only change the weight and bias values of all layers keeping the other attributes of the layers(like drop-out functionality etc) the same as before. Will using the 'with torch.no_grad() block ’ help in attaining my goal?

If you want to manually update the parameters, this dummy code snippet should work:

with torch.no_grad():
    for param in model.parameters():
        param -= learning_rate * param.grad

I’m not sure how your use case works and how you are calculating the un_params.
Could you explain it a bit?

my use case is to get to the global solution in the solution space, as gradient descent can get stuck in local optimum sometimes.I am applying GA(genetic algorithm) on the 1D array (i.e model.named_prameters() after flattening everything to a single array). After GA is done I unflatten this 1D array back and load them to model.named_parameters(). I am using the below code snippets to unflatten the 1d array and to load the unflattened arrays to model_parameters().

def unflatten(matrix,shapes):
final_matrix=[]
#print(“len matrix:”,len(matrix))
index=0

for shape in shapes:
    prod=1
    for num in shape:
        prod=prod*num
    final_matrix.append(np.reshape(matrix[index:index+prod],shape))
    #print("final length:",len(final_matrix))
    # print (prod,final_matrix,np.reshape(matrix[index:index+prod],shape))
    index+=prod
return final_matrix

#shapes are list of shapes of weights and biases in every layer
un_params = unflatten(1d_array, shapes)

#for loading the new weights
def cnn_after_unflatten(un_params,sizes,model):
count=0

for name,param in model.named_parameters():
    #print(param.data.dtype)
    #print("before:",param.data)
    #sizes is the sizes of the weights
    #un_params is my new wieghts list of all layers
    if(len(sizes[count])==1):
        param.data=torch.from_numpy(un_params[count][0]).to(torch.float32)
    else:
        param.data=torch.from_numpy(un_params[count]).to(torch.float32)
    #print(param.data.dtype)
    #print("after :",param.data)
    count+=1