# Dynamic layers combination **URL:** \ **Category:** Uncategorized\ **Created:** [March 9, 2021, 9:23am UTC](https://discuss.pytorch.org/t/dynamic-layers-combination/114176 "2021-03-09T09:23:10Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Aiman\_Mutasem-bellh](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aiman_mutasem-bellh/32/14771_2.png) [@Aiman\_Mutasem-bellh](https://discuss.pytorch.org/u/Aiman_Mutasem-bellh)\ **Post date:** [March 9, 2021, 9:23am UTC](https://discuss.pytorch.org/t/dynamic-layers-combination/114176/1 "2021-03-09T09:23:10Z")
Hi all 🙂 I’m trying to **combine encoder layers output dynamically** as below figure : ![WeChat Image_20210309171047](https://discuss.pytorch.org/uploads/default/original/3X/7/e/7e5826aa579e7b4533ffefe0f99bd29878404ae2.jpeg) I have followed an answer [here](https://discuss.pytorch.org/t/concatenate-layer-output-with-additional-input-data/20462) using `torch.cat()`, and my code as below: ```auto #---------------------------------------- Start dynamic combination ---------------------------- for layer in self.layers: src = layer(src, src_mask) # I have three layers and I expect 3 vectors src = torch.cat([src]).view(-1) src = self.fc_o(src) #src = [batch size, src len, hid dim] ``` ```auto class Encoder(nn.Module): def __init__ (self, input_dim, hid_dim, n_layers, n_heads, pf_dim, dropout, device, max_length = 100): super(). __init__ () self.device = device self.n_layers = n_layers self.tok_embedding = nn.Embedding(input_dim, hid_dim) self.pos_embedding = nn.Embedding(max_length, hid_dim) self.layers = nn.ModuleList([EncoderLayer(hid_dim, n_heads, pf_dim, dropout, device) for _ in range(n_layers)]) self.fc_o = nn.Linear(BATCH_SIZE, input_dim, hid_dim) self.dropout = nn.Dropout(dropout) self.scale = torch.sqrt(torch.FloatTensor([hid_dim])).to(device) #self.fc_o = nn.Linear(BATCH_SIZE, input_dim, hid_dim) def forward(self, src, src_mask): #src = [batch size, src len] #src_mask = [batch size, src len] batch_size = src.shape[0] src_len = src.shape[1] pos = torch.arange(0, src_len).unsqueeze(0).repeat(batch_size, 1).to(self.device) #pos = [batch size, src len] src = self.dropout((self.tok_embedding(src) * self.scale) + self.pos_embedding(pos)) #src = [batch size, src len, hid dim] #---------------------------------------- Start dynamic combination ---------------------------- outputs = [] for layer in self.layers: outputs.clear() for layer in self.layers: src = layer(src, src_mask) outputs.append(src) src = torch.cat(outputs, dim=1) src = self.fc_o(src) return src ``` Actually, I’m not sure about my implementation also I got an error: `RuntimeError: size mismatch, m1: [1 x 753664], m2: [256 x 256] at /opt/conda/conda-bld/pytorch_1573049301898/work/aten/src/THC/generic/THCTensorMathBlas.cu:290` Any suggestions to fix this error and to improve the implementation? Kind regards, Aiman Solyman ---
**Author:** ![Unity05](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/unity05/32/20944_2.png) [@Unity05](https://discuss.pytorch.org/u/Unity05)\ **Post date:** [March 9, 2021, 2:43pm UTC](https://discuss.pytorch.org/t/dynamic-layers-combination/114176/2 "2021-03-09T14:43:07Z")
Hi, in your linear layer definition, you should not specify the batch size (` *in_features* , *out_features* , *bias=True*` [[Docs]](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html)). This does most likely cause your error. However, your concatentation does not make sense to me as your just do `view(-1)` effectively. You could concat compressed layer outputs (as you have to pay attention to not increase the size to fast) seperatly. (Just regarding the encoder part right now.) Regards, Unity05 ---
**Author:** ![Aiman\_Mutasem-bellh](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aiman_mutasem-bellh/32/14771_2.png) [@Aiman\_Mutasem-bellh](https://discuss.pytorch.org/u/Aiman_Mutasem-bellh)\ **Post date:** [March 10, 2021, 12:14am UTC](https://discuss.pytorch.org/t/dynamic-layers-combination/114176/3 "2021-03-10T00:14:46Z")
Thank you sir for your valuable comment. I have tried to apply the encoder without forward layer forward `src = self.fc_o(src)`, and I got the same error. Here, the issue is how to concatenate layers output dynamically. The updated code as below: ```auto outputs = [] for layer in self.layers: outputs.clear() for layer in self.layers: src = layer(src, src_mask) outputs.append(src) x = torch.cat(outputs, dim=1) x = self.fc_o(x) ``` Moreover, the objective of this type of combination is that I’m trying to apply routing by agreement using capsule network following this [paper](https://arxiv.org/abs/1902.05770). The first step is to aggregate the encoder layers output dynamically, then feed them to CapsNet. ![Untitled_2](https://discuss.pytorch.org/uploads/default/original/3X/9/4/9429dce3e8b468d33af3ccd0c5efee1a7e1c0e59.png) Could you help with this case? ---
**Author:** ![Unity05](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/unity05/32/20944_2.png) [@Unity05](https://discuss.pytorch.org/u/Unity05)\ **Post date:** [March 10, 2021, 10:27pm UTC](https://discuss.pytorch.org/t/dynamic-layers-combination/114176/4 "2021-03-10T22:27:38Z")
Hi, Ah okay, now I see what you try to do. Your updated code snipped looks fine to me. Now, implementing a routing mechanism like the `Dynamic Routing` mentioned in the paper should be pretty straight forward. Regards, Unity05