Convert tensor of size 768 to 128

I want to make a projection to the tensor of shape [197, 1, 768] to [197,1,128] using
nn.Conv()

You could use a kernel size and stride of 6, as that’s the factor between the input and output temporal size:

x = torch.randn(197, 1, 768)
conv = nn.Conv1d(in_channels=1, out_channels=1, kernel_size=6, stride=6)
out = conv(x)
print(out.shape)
> torch.Size([197, 1, 128])
2 Likes

Is there a solution that is independent of batch size (second dimension)?

The second dimension of the input is the channel dimension while the first dimension is the batch dimension. nn.Conv1d is independent w.r.t. the batch size.

1 Like