How to fix Conv1d RNN inconsistencies

1

Can I suggest the addition of an arg features_last to Conv1d?
It should be False by default, and when True it will expect input of shape (batch_size, seq_len, features).

That would go some way to making Conv1d usable as a drop-in replacement for an RNN. For completeness, it would also need another argument batch_first that would default to True, and an easy way to add causal padding.

The addition of Dropout1d could be done along the same lines, though with the right input dimension ordering Dropout2d and Dropout3d both work fine in the 1d setting.

2

Another idea for Dropout would be to add a dims argument to nn.Dropout. Legal values would be None meaning drop individual elements, or a tuple of dimensions meaning do dropout along those dimensions.

Then we would have …

Dropout2d == Dropout(dims=(0, 1))