# RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same **URL:** \ **Category:** Uncategorized\ **Created:** [July 26, 2018, 7:36pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782 "2018-07-26T19:36:19Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![rjsdebug](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rjsdebug/32/50585_2.png) [@rjsdebug](https://discuss.pytorch.org/u/rjsdebug)\ **Post date:** [July 26, 2018, 7:36pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/1 "2018-07-26T19:36:19Z")
I am getting the above error (mentioned in title). I was wondering what this issue even means. My guess is that I am using the cpu and gpu together, which is causing errors. However, how can the problem only lie in the weight data? Does this mean I have to convert all of my tensors to tensor.cuda()? ---
**Author:** ![justusschock](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/justusschock/32/4637_2.png) [@justusschock](https://discuss.pytorch.org/u/justusschock)\ **Post date:** [July 26, 2018, 7:47pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/2 "2018-07-26T19:47:49Z")
It seems that your input tensors are already on GPU, but your model isn’t. Did push your model to GPU as well? If you did not, this might be causing the error. If you did so, you should probably show the code since we can only have random guesses without the code. ---
**Author:** ![rjsdebug](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rjsdebug/32/50585_2.png) [@rjsdebug](https://discuss.pytorch.org/u/rjsdebug)\ **Post date:** [July 26, 2018, 7:52pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/3 "2018-07-26T19:52:22Z")
editing… i need to format this correctl sorry ---
**Author:** ![rjsdebug](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rjsdebug/32/50585_2.png) [@rjsdebug](https://discuss.pytorch.org/u/rjsdebug)\ **Post date:** [July 26, 2018, 7:55pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/4 "2018-07-26T19:55:14Z")
Here is the code. I feel like the error might be caused when I say layers+=[RowLSTM()], since this has not been pushed to GPU yet? Sorry for the length but the error could be anywhere so here it all is. ```auto import torch.nn.init as init import torch __all__ = [ 'VGG', 'vgg11', 'vgg11_bn', 'vgg13', 'vgg13_bn', 'vgg16', 'vgg16_bn', 'vgg19_bn', 'vgg19', ] class VGG(nn.Module): ''' VGG model ''' def __init__ (self, features): # features represents the layers array super(VGG, self). __init__ () self.features = features self.classifier = nn.Sequential( nn.Dropout(), nn.Linear(512,512), nn.ReLU(True), nn.Dropout(), nn.Linear(512, 512), nn.ReLU(True), nn.Linear(512, 10), ) # Initialize weights for m in self.modules(): if isinstance(m, nn.Conv2d): n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels m.weight.data.normal_(0, math.sqrt(2. / n)) m.bias.data.zero_() def forward(self, x): # x is the image, we run x through the layers print(x.size()) x = self.features(x) # runs through all features, where each feature is a function x = x.view(x.size(0), -1) # after running through features, does sequential steps to finally classify x = self.classifier(x) # print(x) return x def make_layers(cfg, batch_norm=False): # print("Making layers!") layers = [] in_channels = 3 for v in cfg: if v == 'M': layers += [nn.MaxPool2d(kernel_size=2, stride=2)] else: conv2d = nn.Conv2d(in_channels, v, kernel_size=3, padding=1) if batch_norm: layers += [conv2d, nn.BatchNorm2d(v), nn.ReLU(inplace=True)] else: layers += [conv2d, nn.ReLU(inplace=True)] in_channels = v layers+=[RLSTM()] return nn.Sequential(*layers) class RLSTM(nn.Module): def __init__ (self): super(RLSTM,self). __init__ () def forward(self, image): print("going in rowlstm") global current global _layer global isgates size = image.size() b = size[0] indvs = list(image.split(1,0)) # split up the batch into individual images #print(indvs[0].size()) tensor_array = [] for i in range(b): current = 0 _layer = [] isgates = [] tensor_array.append(self.RowLSTM(indvs[i])) seq=tuple(tensor_array) trans = torch.cat(seq,0) return trans.cuda() # trying to make floattensor error go away def RowLSTM(self, image): global current global _layer global isgates # input-to-state (K_is * x_i) : 3x1 convolution. generate 4h x n x n tensor. 4hxnxn tensor contains all i -> s info # the input to state convolution should only be computed one time if current==0: n = image.size()[2] ch=image.size()[1] input_to_state = torch.nn.Conv2d(ch,4*ch,kernel_size=(1,3),padding=(0,1)) isgates = self.splitIS(input_to_state(image)) # convolve, then split into gates (4 per row) cell=RowLSTMCell(0,torch.randn(ch,n,1),torch.randn(ch,n,1),torch.randn(ch,n,1),torch.randn(ch,n,1),torch.randn(ch,n,1),torch.randn(ch,n,1)) # now have dummy, learnable variables for first row _layer.append(cell) else: Cell_prev = _layer[current-1] # access previous row hidPrev = Cell_prev.getHiddenState() ch = image.size()[1] # print("about to apply conv1d") state_to_state = torch.nn.Conv2d(ch,4*ch,kernel_size=(1,3),padding=(0,1)) # error is here: hidPrev is an array - not a valid number of input channel # print("applied conv1d") prevHid=Cell_prev.getHiddenState() ssgates = self.splitSS(state_to_state(prevHid.unsqueeze(0))) #need to unsqueeze (Ex: currently 16x5, need to make 1x16x5) gates = self.addGates(isgates,ssgates,current) # split gates ig, og, fg, gg = gates[0], gates[1], gates[2], gates[3] # into four, ADD SIGMOID! cell = RowLSTMCell(Cell_prev,ig,og,fg,gg,0,0) cell.compute() _layer.append(cell) # attempting to eliminate requirement of getting size #print(current) try: current+=1 y=(isgates[0][0][1][current]) return self.RowLSTM(image) except Exception as error: concats=[] for cell in _layer: tensor=torch.unsqueeze(cell.h,0) concats.append(tensor) seq=tuple(concats) tensor=torch.cat(seq,3) return tensor def splitIS(tensor): #always going to be splitting into 4 pieces, so no need to add extra parameters inputStateGates={} size=tensor.size() # 1 x 4h x n x n out_ft=size[1] # get 4h for the nxnx4h tensor num=size[2] # get n for the nxn image hh=out_ft/4 # we want to split the tensor into 4, for the gates tensor = torch.squeeze(tensor) # 4h x n x n # First, split by row: Creates n tensors of 4h x n x 1 rows = list(tensor.split(1,2)) for i in range(num): # Each row is a tensor of 4h x n x 1, split it into 4 of h x n x 1 row=rows[i] inputStateGates[i]=list(row.split(hh,0)) return inputStateGates def splitSS(tensor): # 1 x 4h x n x 1, create 4 of 1 x h x n x 1 size=tensor.size() out_ft=size[1] # get 4h for the 1x4hxn tensor num=size[2] # get n for the 1xhxn row hh=out_ft/4 # we want to split the tensor into 4, for the gates tensor = tensor.squeeze(0) # 4h x n x 1 splitted=list(tensor.split(hh,0)) return splitted def addGates(i2s,s2s,key): """ these dictionaries are of form {key : [[i], [o], [f], [g]]} we want to add pairwise elemeents """ # i2s is of form key: [[i], [o], [f], [g]] where each gate is hxn # s2s is of form [[h,n],[h,n],[h,n], [h,n]] gateSum = [] for i in range(4): # always of length 4, representing the gates gateSum.append(torch.sigmoid(i2s[key][i] + s2s[i])) return gateSum cfg = { 'A': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 'B': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 'D': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'], 'E': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'], } ``` ---
**Author:** ![Noosh\_Nabi](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/e0b2c6/32.png) [@Noosh\_Nabi](https://discuss.pytorch.org/u/Noosh_Nabi)\ **Post date:** [December 20, 2018, 11:15pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/5 "2018-12-20T23:15:31Z")
I have faced the similar error as well, and will be thankful for any help and advice. RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same ---
**Author:** ![justusschock](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/justusschock/32/4637_2.png) [@justusschock](https://discuss.pytorch.org/u/justusschock)\ **Post date:** [December 21, 2018, 8:08am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/6 "2018-12-21T08:08:10Z")
can you post a minimal example to reproduce it? Typically something hasn’t been pushed to the correct device (usually the GPU). ---
**Author:** ![Hung\_Nguyen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hung_nguyen/32/8273_2.png) [@Hung\_Nguyen](https://discuss.pytorch.org/u/Hung_Nguyen)\ **Post date:** [December 21, 2018, 9:25am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/7 "2018-12-21T09:25:23Z")
You need to convert your input to cuda before feeding it to the model. Let’s say: ```auto model = VGG16() model.cuda() for inp in dataset: x = inp.cuda() y = model(x) ... ``` ---
**Author:** ![sun](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/49beb7/32.png) [@sun](https://discuss.pytorch.org/u/sun)\ **Post date:** [December 23, 2018, 8:58am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/8 "2018-12-23T08:58:23Z")
need to move your model to GPU: `model.cuda()` ---
**Author:** ![zhjikoshlizhzc](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/ba9def/32.png) [@zhjikoshlizhzc](https://discuss.pytorch.org/u/zhjikoshlizhzc)\ **Post date:** [March 12, 2019, 7:10am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/9 "2019-03-12T07:10:19Z")
Thanks very much! my problem was fixed because of your idea! ---
**Author:** ![shwinshaker](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shwinshaker/32/50475_2.png) [@shwinshaker](https://discuss.pytorch.org/u/shwinshaker)\ **Post date:** [January 9, 2020, 3:25am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/10 "2020-01-09T03:25:24Z")
I got the same problem. while later found that `nn.ModuleList` should be used to properly implement list of modules in pytorch, otherwise, it can’t be recognized. Hope this helps ---
**Author:** ![AlittleBrave](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/a88e4f/32.png) [@AlittleBrave](https://discuss.pytorch.org/u/AlittleBrave)\ **Post date:** [April 2, 2020, 4:03am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/11 "2020-04-02T04:03:09Z")
Thanks. Fixed my problem. ---
**Author:** ![zongoalbert](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/7feea3/32.png) [@zongoalbert](https://discuss.pytorch.org/u/zongoalbert)\ **Post date:** [June 9, 2020, 3:16pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/12 "2020-06-09T15:16:17Z")
i have the some issue please need your help ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [June 10, 2020, 7:59am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/13 "2020-06-10T07:59:22Z")
The error is raised if your data or model are not pushed to the device, while one of them is. You should make sure that both are on the GPU using: ```python model.to('cuda') data = data.to('cuda') ``` as explained in this topic. ---
**Author:** ![devin](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/devin/32/32107_2.png) [@devin](https://discuss.pytorch.org/u/devin)\ **Post date:** [December 8, 2020, 12:22pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/15 "2020-12-08T12:22:52Z")
Thanks a lot! Worked. ---
**Author:** ![jacob\_williams](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jacob_williams/32/32467_2.png) [@jacob\_williams](https://discuss.pytorch.org/u/jacob_williams)\ **Post date:** [December 17, 2020, 8:33pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/16 "2020-12-17T20:33:50Z")
Thanks for this suggestion, this was my issue. ---
**Author:** ![Shyam\_Gupta196](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shyam_gupta196/32/29724_2.png) [@Shyam\_Gupta196](https://discuss.pytorch.org/u/Shyam_Gupta196)\ **Post date:** [May 13, 2021, 4:56pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/17 "2021-05-13T16:56:06Z")
```auto Model.cuda() #not a good approach ``` is not what i will tell you to do … you may use ```auto device = torch.device('cuda')if torch.cuda.is_available() else torch.device('cpu') then push the model to device & not cuda model = model.to(device) ## this will be a correct approach ### otherwise it will not be reproducable ### ``` ---
**Author:** ![Sajedeh\_Molavi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sajedeh_molavi/32/31769_2.png) [@Sajedeh\_Molavi](https://discuss.pytorch.org/u/Sajedeh_Molavi)\ **Post date:** [September 22, 2021, 6:39pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/18 "2021-09-22T18:39:27Z")
Thank you so much. Great. ---
**Author:** ![senitent\_signal](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/senitent_signal/32/45937_2.png) [@senitent\_signal](https://discuss.pytorch.org/u/senitent_signal)\ **Post date:** [August 4, 2022, 11:53am UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/20 "2022-08-04T11:53:42Z")
One of element of n-network is not registered on GPU ```auto CNN_net( (conv1): Conv2d(1, 16, kernel_size=(5, 5), stride=(1, 1)) (maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) (conv2): Conv2d(16, 24, kernel_size=(5, 5), stride=(1, 1)) (maxpool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) (fc1): Linear(in_features=384, out_features=100, bias=True) (fc2): Linear(in_features=100, out_features=10, bias=True) (prelu): PReLU(num_parameters=1) ) ``` Example 2 ```auto CNN_net( (conv1): Conv2d(1, 16, kernel_size=(5, 5), stride=(1, 1)) (maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) (conv2): Conv2d(16, 24, kernel_size=(5, 5), stride=(1, 1)) (maxpool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) (fc1): Linear(in_features=384, out_features=100, bias=True) (fc2): Linear(in_features=100, out_features=10, bias=True) ) ``` Only solution is to include elements of n-network in init function else torch wont register it while transfering to gpu. ---
**Author:** ![mna](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/edb3f5/32.png) [@mna](https://discuss.pytorch.org/u/mna)\ **Post date:** [May 9, 2024, 1:44pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/21 "2024-05-09T13:44:23Z")
i have an error : /usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py in \_conv\_forward(self, input, weight, bias) 454 weight, bias, self.stride, 455 \_pair(0), self.dilation, self.groups) → 456 return F.conv2d(input, weight, bias, self.stride, 457 self.padding, self.dilation, self.groups) 458 RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same and it runs the first epoch well, but after that i have error above. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [May 9, 2024, 1:51pm UTC](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782/22 "2024-05-09T13:51:03Z")
Try to narrow down which layer fails exactly and make sure its parameters are in the GPU since the error message claims they are still on the CPU. [Next page](https://discuss.pytorch.org/t/runtimeerror-input-type-torch-cuda-floattensor-and-weight-type-torch-floattensor-should-be-the-same/21782.md?page=2)