Hello. I am having trouble over a small piece of code.
import torch import torch.nn as nn import math class Segmentation(torch.nn.Module): def __init__(self, input, hidden, output, layers, bidirectional, kernal, dropout_prob): super().__init__() assert(kernal%2==1 and kernal%2==1), 'Convolution kernal sizes should be odd for keeping same dimensions.' padding = (math.floor(kernal/2), math.floor(kernal/2)) # cnn input = [N * C * H * W] H=time, W=Frequency self.conv_1 = nn.Conv2d(1, 1, kernel_size=kernal, padding=padding) self.mp_1 = nn.MaxPool2d(kernel_size=kernal, stride=1, padding=padding) self.conv_2 = nn.Conv2d(1, 1, kernel_size=kernal, padding=padding) self.mp_2 = nn.MaxPool2d(kernel_size=kernal, stride=1, padding=padding) self.dropout = nn.Dropout(p=dropout_prob) self.gru = nn.GRU(input, hidden, num_layers=layers, dropout=dropout_prob, bidirectional=bidirectional, batch_first=True) in_features = hidden*2 if bidirectional else hidden self.linear = nn.Linear(in_features=in_features, out_features=output) self.log_softmax = nn.LogSoftmax(dim=-1) def forward(self, input, input_lens): # inputs must be length wise sorted # input is already padded input = self.mp_1(self.conv_1(input)) input = self.mp_2(self.conv_2(input)) input = self.dropout(input) # now pack the sequences. # input -> [batch, ch=1, height=time, width=frequency] input = input.squeeze(1) packed_input = nn.utils.rnn.pack_padded_sequence(input, input_lens, batch_first=True) packed_output, _ = self.gru(packed_input) output, _ = nn.utils.rnn.pad_packed_sequence(packed_output, batch_first=True) output = self.linear(output) return output
The problem occurs on line 39
packed_output, _ = self.gru(packed_input). After the tesnor is passed to the gru layer, the system just freezes. I didn’t understand what was happening at first. Then I opened the debugger and system monitor and found that after coming to this line, the RAM consumption starts increasing. My RAM is 8GB and the programme alone consumes more than 7 GB, which is really weird, and therefore the system crushes. Every time I run this script, I had to force restart my machine. I tried googling the issue, NO LUCK!
the input to the
self.gru(packed_input is a packed sequence.
The only reason I can think of this kind of behavior is an infinite for loop. But How can that happen on the forward layer of a built in library?
Or can it?