Hello. I am having trouble over a small piece of code.
import torch
import torch.nn as nn
import math
class Segmentation(torch.nn.Module):
def __init__(self, input, hidden, output, layers, bidirectional, kernal, dropout_prob):
super().__init__()
assert(kernal[0]%2==1 and kernal[1]%2==1), 'Convolution kernal sizes should be odd for keeping same dimensions.'
padding = (math.floor(kernal[0]/2), math.floor(kernal[1]/2))
# cnn input = [N * C * H * W] H=time, W=Frequency
self.conv_1 = nn.Conv2d(1, 1, kernel_size=kernal, padding=padding)
self.mp_1 = nn.MaxPool2d(kernel_size=kernal, stride=1, padding=padding)
self.conv_2 = nn.Conv2d(1, 1, kernel_size=kernal, padding=padding)
self.mp_2 = nn.MaxPool2d(kernel_size=kernal, stride=1, padding=padding)
self.dropout = nn.Dropout(p=dropout_prob)
self.gru = nn.GRU(input, hidden, num_layers=layers,
dropout=dropout_prob, bidirectional=bidirectional, batch_first=True)
in_features = hidden*2 if bidirectional else hidden
self.linear = nn.Linear(in_features=in_features, out_features=output)
self.log_softmax = nn.LogSoftmax(dim=-1)
def forward(self, input, input_lens):
# inputs must be length wise sorted
# input is already padded
input = self.mp_1(self.conv_1(input))
input = self.mp_2(self.conv_2(input))
input = self.dropout(input)
# now pack the sequences.
# input -> [batch, ch=1, height=time, width=frequency]
input = input.squeeze(1)
packed_input = nn.utils.rnn.pack_padded_sequence(input, input_lens, batch_first=True)
packed_output, _ = self.gru(packed_input)
output, _ = nn.utils.rnn.pad_packed_sequence(packed_output, batch_first=True)
output = self.linear(output)
return output
The problem occurs on line 39 packed_output, _ = self.gru(packed_input). After the tesnor is passed to the gru layer, the system just freezes. I didn’t understand what was happening at first. Then I opened the debugger and system monitor and found that after coming to this line, the RAM consumption starts increasing. My RAM is 8GB and the programme alone consumes more than 7 GB, which is really weird, and therefore the system crushes. Every time I run this script, I had to force restart my machine. I tried googling the issue, NO LUCK!
the input to the self.gru(packed_input is a packed sequence.
The only reason I can think of this kind of behavior is an infinite for loop. But How can that happen on the forward layer of a built in library?
Or can it?