Is there a "PyTorch-ic" way to do padding and batching?

[eugene]

I’m new to using PyTorch for RNNs and loving it so far. Nonetheless, my model is only training on individual sequences (i.e., batch size = 1).

As I move towards training in batch, I’ve been going through many different implementations for padding and batching the tensors, such as:

• https://medium.com/@sonicboom8/sentiment-analysis-with-variable-length-sequences-in-pytorch-6241635ae130
• https://github.com/hunkim/PyTorchZeroToAll/blob/master/13_2_rnn_classification.py

I also noticed PyTorch has pad_sequence which seems useful, though it does not return a PackedSequence object.

Is there a PyTorch recommended way to do padding and batching?

[aplassard (Andrew Plassard)]

Simple working example how to use packing for variable-length sequence inputs for rnn

Hi, Updated - here’s a simple example of how I think you use pack_padded_sequence and pad_packed_sequence, but I don’t know if it’s the right way to use them? import torch import torch.nn as nn from torch.autograd import Variable batch_size = 3 max_length = 3 hidden_size = 2 n_layers =1 # container batch_in = torch.zeros((batch_size, 1, max_length)) #data vec_1 = torch.FloatTensor([[1, 2, 3]]) vec_2 = torch.FloatTensor([[1, 2, 0]]) vec_3 = torch.FloatTensor([[1, 0, 0]]) batch_in[0] = vec_1…