Expected tensor for argument #1 'indices' to have scalar type Long; but got CPUFloatTensor instead (while checking arguments for embedding)

1

What does the embedding expect for input? I got this error code when trying to embed the sentences.

x = torch.randint(0, 10, (2, 5))
embedding = nn.Embedding(5, 3)
seq_len = len(x)
embedded = embedding(x)
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python2.7/site-packages/torch/nn/modules/module.py”, line 477, in call
result = self.forward(*input, **kwargs)
File “/usr/local/lib/python2.7/site-packages/torch/nn/modules/sparse.py”, line 110, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File “/usr/local/lib/python2.7/site-packages/torch/nn/functional.py”, line 1110, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 ‘indices’ to have scalar type Long; but got CPUFloatTensor instead (while checking arguments for embedding)

3 Likes
2

The first argument to nn.Embedding should be the num_embeddings, i.e. the size of your dictionary.
In your code sample it looks like you are using a dictionary of 10 words, so try to create your embedding as:

embedding = nn.Embedding(10, 3)

and run your code again.

The error message seems to be a bit strange, as x should be a LongTensor.
Which PyTorch version are you using?

3

No, it’s not correct. I figured it out in this way:

x = torch.tensor([3])
x
tensor([3])
x.size()
(1,)
embembedding = nn.Embedding(5, 3)
embedded = embedding(x)
embedded
tensor([[-0.3584, -0.4176, 0.1584]], grad_fn=)

4

x = torch.tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 0]])
x
tensor([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 0]])
embedded = embedding(x)
embedded
tensor([[[ 0.2623, -0.6216, 1.8112],
[ 0.5538, -1.0141, -1.6522],
[-0.3584, -0.4176, 0.1584],
[ 1.3007, -1.4878, 0.7318],
[ 1.0062, 0.8589, -1.6106]],

    [[-0.4756,  1.3294,  0.3395],
     [ 0.4187,  0.9915, -0.5702],
     [ 0.3695,  0.8041, -0.6507],
     [-0.0904,  0.4995, -0.6367],
     [ 0.4574, -1.1187, -0.2644]]], grad_fn=<EmbeddingBackward>)
5 
>>> x = torch.LongTensor(2, 5).random_(0, 10)
>>> x
tensor([[5, 3, 9, 3, 8],
        [8, 1, 8, 6, 9]])
>>> embedded = embedding(x)
>>> embedded
tensor([[[ 1.0062,  0.8589, -1.6106],
         [-0.3584, -0.4176,  0.1584],
         [-0.0904,  0.4995, -0.6367],
         [-0.3584, -0.4176,  0.1584],
         [ 0.3695,  0.8041, -0.6507]],

        [[ 0.3695,  0.8041, -0.6507],
         [ 0.2623, -0.6216,  1.8112],
         [ 0.3695,  0.8041, -0.6507],
         [-0.4756,  1.3294,  0.3395],
         [-0.0904,  0.4995, -0.6367]]], grad_fn=<EmbeddingBackward>)
6

I’m getting this error. What exactly did you learn?

1 Like
7

make sure your embeddings are having integer and not float like values in them; that’s what the error means actually (like label encoding the values(if going for entity embeddings))

3 Likes
8

Loading glove vector embeddings have vector of float values.

9

That’s not what i meant, embedding layer is just a lookup table, that table index (dtype for your entity embeddings) needs to be int type or long etc…

1 Like
10

Yes thanks. I understood that.

11

Hi Ptrblck,

I am trying to apply conditional DCGAN. The deined discriminator and generator are as follow, But I got an error when using Discriminator. ( RuntimeError: index out of range: Tried to access index 14 out of table with 9 rows. at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:418

)

class Discriminator(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator, self).__init__()
        self.ngpu = ngpu
        self.label_embedding = nn.Embedding(10, 10)
            # input is (nc) x 64 x 64
        self.l1= nn.Sequential(nn.Conv2d(1, ndf, 4, 2, 1, bias=False),nn.LeakyReLU(0.2, inplace=True))
             # state size. (ndf) x 32 x 32
        self.l2=nn.Sequential(nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),nn.BatchNorm2d(ndf * 2),nn.LeakyReLU(0.2, inplace=True))
        self.drop_out2 = nn.Dropout(0.5)

            # state size. (ndf*2) x 16 x 16
        self.l3= nn.Sequential(nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True))
        self.drop_out3 = nn.Dropout(0.5)

            # state size. (ndf*4) x 8 x 8
        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, 1, 4, 2, 1, bias=False),nn.Sigmoid()

    def forward(self, x,Real_volume):
        
         c = self.label_embedding(Real_volume)
         x = torch.cat([x, c], 1)
         out = self.l1(x)
#         print("outsize1",out.shape)
         out=self.l2(out)
         out=self.drop_out2(out)
#         print("outsize2",out.shape)
         out=self.l3(out)
         out=self.drop_out3(out)

#         print("outsize3",out.shape)
         out=self.l4(out)
#         print("outsize4",out.shape)
      
         return out

class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.label_embedding = nn.Embedding(10, 10)
        
        self.l1= nn.Sequential(nn.ConvTranspose2d( nz+10, ngf * 8, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 8),
        nn.ReLU(True))
        # state size. (ngf*8) x 4 x 4
        self.l2=nn.Sequential(nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True))
        # state size. (ngf*4) x 8 x 8
        self.l3=nn.Sequential(nn.ConvTranspose2d( ngf * 4, ngf * 2, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True))
        # state size. (ngf*2) x 16 x 16
        self.l4=nn.Sequential(nn.ConvTranspose2d( ngf * 2, ngf, 3, 2, 1, bias=False),
        nn.BatchNorm2d(ngf),
        nn.ReLU(True))
        # state size. (ngf) x 32 x 32
        self.l5=nn.Sequential(nn.ConvTranspose2d( ngf, 1, 3, 2, 3, bias=False),nn.Sigmoid())
#            nn.Tanh()
            # state size. (nc) x 64 x 64


    def forward(self, x,Real_volume):
        c = self.label_embedding(Real_volume)
        x = torch.cat([x,c], 1)
        output = self.l1(x)
        output = self.l2(output)
        output = self.l3(output)
        output = self.l4(output)
        output = self.l5(output)
        return output

I used this command:
        output = netD(real_cpu,Real_volume).view(-1)
Do you think is there any thing wrong with my code and discriminator and generator implementation? I think sth is wrong with structure. What do you think? the z dimension is 100, and the training dataset are 21*21 patches with 64 batch size.
12

Which line of code is raising this error?
Could you print the input shape as well as the min and max values of the input tensor to this particular layer once you’ve isolated it?

13

The discriminator and generator are as follow:

### Discriminator-----
class Discriminator(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator, self).__init__()
        self.ngpu = ngpu
        self.label_embedding = nn.Embedding(10,10)
            # input is (nc) x 64 x 64
        self.l1= nn.Sequential(nn.Conv2d(1, ndf, 4, 2, 1, bias=False),nn.LeakyReLU(0.2, inplace=True))
             # state size. (ndf) x 32 x 32
        self.l2=nn.Sequential(nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),nn.BatchNorm2d(ndf * 2),nn.LeakyReLU(0.2, inplace=True))
        self.drop_out2 = nn.Dropout(0.5)

            # state size. (ndf*2) x 16 x 16
        self.l3= nn.Sequential(nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True))
        self.drop_out3 = nn.Dropout(0.5)

            # state size. (ndf*4) x 8 x 8
        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, 1, 4, 2, 1, bias=False),nn.Sigmoid())

#        self.l4= nn.Sequential(nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 8),nn.LeakyReLU(0.2, inplace=True),nn.Sigmoid())
#            # state size. (ndf*8) x 4 x 4
#        self.l5= nn.Sequential(nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False),nn.Sigmoid())
     

    def forward(self, x,Real_volume):
        
         c = self.label_embedding(Real_volume)
         print(c.shape)
         print(x.shape)
         x = torch.cat([x, c], 1)
         out = self.l1(x)
#         print("outsize1",out.shape)
         out=self.l2(out)
         out=self.drop_out2(out)
#         print("outsize2",out.shape)
         out=self.l3(out)
         out=self.drop_out3(out)

         out=self.l4(out)
      
         return out

## -----------Generator ------------
class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.label_embedding = nn.Embedding(10,10)
        
        self.l1= nn.Sequential(nn.ConvTranspose2d( nz+10, ngf * 8, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 8),
        nn.ReLU(True))
        # state size. (ngf*8) x 4 x 4
        self.l2=nn.Sequential(nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True))
        # state size. (ngf*4) x 8 x 8
        self.l3=nn.Sequential(nn.ConvTranspose2d( ngf * 4, ngf * 2, 3, 1, 0, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True))
        # state size. (ngf*2) x 16 x 16
        self.l4=nn.Sequential(nn.ConvTranspose2d( ngf * 2, ngf, 3, 2, 1, bias=False),
        nn.BatchNorm2d(ngf),
        nn.ReLU(True))
        # state size. (ngf) x 32 x 32
        self.l5=nn.Sequential(nn.ConvTranspose2d( ngf, 1, 3, 2, 3, bias=False),nn.Sigmoid())
#            nn.Tanh()
            # state size. (nc) x 64 x 64


    def forward(self, x,Real_volume):
        c = self.label_embedding(Real_volume)
        print(c.shape)
        x = torch.cat([x,c], 1)
        output = self.l1(x)
        output = self.l2(output)
        output = self.l3(output)
        output = self.l4(output)
        output = self.l5(output)
        return output
        
  The error is when I use discriminator ( output = netD(real_cpu,Real_volume.squeeze(1)).view(-1)) and the error referred to the (c = self.label_embedding(Real_volume))

The real_cpu  is the training set (64x1x21x21) and the Readl_volume are the 64x1 volume (float numbers) from 2 to 89, the nz=100.
As an suggestion I set it to the
        self.label_embedding = nn.Embedding(100,10)
I set to this it give not any error but I am not sure if it works correctly or not!?
14

would you please help me that with defining the layer for conditional DCGAN? I am really confused with them.

I appreciate your help.

15

Readl_volume should contain long values, in the range [0, embedding_dim-1], while it seems you are trying to pass float values and are not using the full range.
If you don’t need to provide 100 different indices, you wouldn’t need to use nz=100.

However, after this layer the torch.cat([x, c], 1) would most likely fail, as x and c would have a different number of dimensions.

I’m not sure what your current use case is, but you won’t be able to concatenate two tensors with different shapes in more then a single dimension.

16

Yes, you are right. the size is different . I am trying to apply conditional GAN. The volumes are the condition and 100 is the size of the noise for generator. I need to cat the condition and noise in generator . and cat the condition with the training datasets in the discriminator. The size of noise for cat is 100 and the size of training dataset is 21x21. I don’t know how to embed to have these size for the conditions (volume) and cat with the noise for generator and training data (21x21) for discriminator.

17

The concatenation won’t work out of the box, if you want to keep the dimensions.
You could of course flatten the condition tensor to [batch_size, -1] and concatenate in dim1, however you would have to check, if it fits your use case or not.

18

when building vocabulary , changing to torch.FloatTensor to torch.LongTensor solved model not accepting data in my case. nn.Embedding expects inputs LongTensor only

1 Like
19

Input data is looking this:
tensor([[-1.0000, -1.0000, -1.0000, …, -0.1132, -1.0000, -1.0000],
[-1.0000, -1.0000, -1.0000, …, -0.1253, -1.0000, -1.0000],
[-1.0000, -1.0000, -1.0000, …, -0.1316, -1.0000, -1.0000],
…,
[-1.0000, -1.0000, -1.0000, …, -0.1269, -1.0000, -1.0000],
[-1.0000, -1.0000, -1.0000, …, -0.1322, -1.0000, -1.0000],
[-1.0000, -1.0000, -1.0000, …, -0.1047, -1.0000, -1.0000]],
device=‘cuda:0’)

data = self.embeddingd(data)
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 1051, in _call_impl
return forward_call(*input, **kwargs)
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/sparse.py”, line 160, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File “/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py”, line 2043, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 ‘indices’ to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)

I am having this error too. Please help me on this.

20

As the error message explains, nn.Embedding expects indices as its input (as LongTensor or IntTensor), while you are passing FloatTensors to it.
Note that an embedding is working as a “lookup table” and outputs dense feature tensors.
Based on your input you also won’t be able to simply transform the input to a LongTensor, since it contains negative indices (besides the obvious rounding, which I don’t know if it would fit your use case).

1 Like