[MAdd]: Dropout is not supported!

nlp
1

I am trying to reproduce this code snipped from PyTorch.When I pass the input to the model it returns the following warnings. Will it effect my outputs ?

Warning

[MAdd]: Dropout is not supported!
[Flops]: Dropout is not supported!
[Memory]: Dropout is not supported!
[MAdd]: LayerNorm is not supported!
[Flops]: LayerNorm is not supported!
[Memory]: LayerNorm is not supported!

I am using the following code

decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
memory = torch.rand(10, 32, 512)
tgt = torch.rand(20, 32, 512)
out = transformer_decoder(tgt, memory)
2

I haven’t seen these warnings before. Could you post a complete, minimal and executable code snippet as well as the entire output log including the stacktrace, please?

3

@ptrblck Please find attached complete code snippet:

import numpy as np
import torch
from torch import nn
from torchinfo import summary
decoder_layer = torch.nn.TransformerDecoderLayer(d_model=128, nhead=8,dim_feedforward=256)
Decoder = torch.nn.TransformerDecoder(decoder_layer, num_layers=1)
src = torch.rand(1, 128)
tgt = torch.rand(1, 128)
out = Decoder(tgt,src)

This code generates a vector of [1x128] with the following messages:

[MAdd]: Dropout is not supported!
[Flops]: Dropout is not supported!
[Memory]: Dropout is not supported!
[MAdd]: LayerNorm is not supported!
[Flops]: LayerNorm is not supported!
[Memory]: LayerNorm is not supported!

image
image1017×487 30.4 KB

4

Are these warning still raised if torchinfo is not imported?

5

@ptrblck No, they occurred with only torchinfo.

6

In that case I would recommend to try to update to the latest release of torchinfo and to check if these warnings are still raised. If so, you might want to create a GitHub issue in their repository.

1 Like