Broadcast a tensor

nlp
1

Hello everyone, I have a question about how I can broadcast my tensor. For example, I a tensor X = [1,0,0,1,0] and an embedding as follow:

Y = [[0.1992, 0.5196, 0.4954],
          [0.1224, 0.0704, 0.6504],
          [0.5171, 0.8013, 0.1156],
          [0.8703, 0.2654, 0.6088],
          [0.1722, 0.2609, 0.0734]]

And then, I want to convert X from [1,0,0,1,0] to

X = [[0.1992, 0.5196, 0.4954],
          [0.8703, 0.2654, 0.6088]]

This is the first time I post a topic, so if there are any problem, please remind me. Thanks

2

I did not get your question. I think your understanding of embedding is wrong. For the desired output , The values of X should be

X = [0,3]

The embedding layer is basically a dictionary lookup. So, new_X[i] = Y[X[i]].

3

@lengocduc195khtn I guess you might want to know about the process of forward propagation of neural networks.

it depends. For example, whether it is a convolutional layer or a linear layer. Take convolution as an example.

image
image716×364 15.3 KB

below is a pytorch example

>>> import torch
>>> m = torch.nn.Conv2d(1, 1, 2)
>>> input = torch.randn(1, 3, 3)
>>> input
tensor([[[ 0.4180, -0.0296, -1.3577],
         [ 0.0083, -0.0097,  0.7227],
         [ 0.8191, -0.1416,  0.4224]]])
>>> output = m(input)
>>> output
tensor([[[0.0241, 0.1335],
         [0.0582, 0.2718]]], grad_fn=<SqueezeBackward1>)