Hello Forum
Im currently building a CRNN (CNN followed by RNN) which needs to classify ship-types according to their movement/behavoir. Im using AIS data which is transformed into a [lat, lon, time] data sequence.
The idea is to use the CNN as feature extraction network and then use the RNN to classify from found features.
The network i have is unfortunately not working. I trained it on 1000 Cargo ship tracks, 1000 Passenger ship tracks and 1000 Fishing ship tracks. The result is an accurancy of 30% which is the same as the network essentially just guessing same class over and over.
My Net is the following: First i have 3 convolutional layers, then 4 recurrent layers.
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
### RNN ###
self.rnn1 = nn.GRU(input_size=32, #input is the output from CNN
hidden_size=hidden_size,
num_layers=1)
self.rnn2 = nn.GRU(input_size=hidden_size,
hidden_size=hidden_size,
num_layers=1)
self.rnn3 = nn.GRU(input_size=hidden_size,
hidden_size=hidden_size,
num_layers=1)
self.rnn4 = nn.GRU(input_size=hidden_size,
hidden_size=hidden_size,
num_layers=1)
self.activation = nn.ReLU()
### END ###
self.dense1 = nn.Linear(hidden_size, 3)
### CNN ###
self.conv1 = nn.Sequential(
nn.Conv1d(
in_channels=3,
out_channels=8,
kernel_size=5,
stride=1,
padding=2,
),
nn.ReLU(),
#nn.MaxPool1d(kernel_size=2), # reduce dimension of sequece by half
)
self.conv2 = nn.Sequential(
nn.Conv1d(
in_channels=8,
out_channels=16,
kernel_size=5,
stride=1,
padding=2,
),
nn.ReLU(),
#nn.MaxPool1d(kernel_size=2), # reduce dimension of sequece by half
)
self.conv3 = nn.Sequential(
nn.Conv1d(
in_channels=16,
out_channels=32,
kernel_size=5,
stride=1,
padding=2,
),
nn.ReLU(),
#nn.MaxPool1d(kernel_size=2), # reduce dimension of sequece by half
)
def forward(self, x, hidden, batch_size):
x = self.conv1(x.double()) #inputs (1,3,batch_size)
x = self.conv2(x.double())
x = self.conv3(x.double())
#Reshape batch for RNN training:
x = x.reshape(batch_size,1,32)
x, hidden = self.rnn1(x, hidden) #inputs (seq_len,1,3)
x = self.activation(x)
x, hidden = self.rnn2(x, hidden)
x = self.activation(x)
x, hidden = self.rnn3(x, hidden)
x = self.activation(x)
x, hidden = self.rnn4(x, hidden)
#x = x.select(0, maxlen-1).contiguous()
x = x.view(-1, hidden_size)
x = F.relu(self.dense1(x))
return x, hidden #Returns prediction for all batch_size timestamps. i.e [batch_size, 3]
def init_hidden(self):
weight = next(self.parameters()).data
return Variable(weight.new(1, 1, hidden_size).zero_())
My optimizer and criterion:
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0001, weight_decay=0.5)
and my training phase:
def train():
print("Training Initiated!")
model.train()
hidden = model.init_hidden() #Initiate hidden
for step, data in enumerate(train_set_all):
X = data[0] #Entire sequence
y = data[1] #[1,0,0] or [0,1,0] or [0,0,1]
y = y.long()
#print(y.size())
### Split sequence into batches:
batch_size = 50 # split sequence into mini-sequences of size 50
max_batches = int(X.size(2)/batch_size)
for nbatch in range(max_batches):
model.zero_grad()
output, hidden = model(X[:,:,nbatch*batch_size:batch_size+nbatch*batch_size], Variable(hidden.data), batch_size)
loss = criterion(output, torch.max(y[:,nbatch*batch_size:batch_size+nbatch*batch_size,:].reshape(batch_size,3), 1)[1])
loss.backward()
optimizer.step()
print(step)
my question is. Does it makes sense? I know this is quite a question. At the moment i use batches of 50 time samples. This is so that the convolutional part of the network have something to convolve around. Ideally i would just feed it a single timestamp at a time but the result was the same (30 % accurancy).
Am i missing something between the networks? I.e. between the CNN and the RNN. Right now i just reshape the data so it fits the RNN requirements. Do i need anything else?
I cant seem to find any good tutorials on CRNNs only a few examples of source codes. But i find those hard to rewrite into my example when i have no information other than the code.
My labels are simply [0, 1, 0] or [1, 0, 0] or [0,0,1]. Is this correct? Should i use [1,2,3] or something of the like? And does it make sense to use the criterion that i use which i have labels as that? Is it possible to get a probability out as output from the network? Such that class 1 might be 20, class to might be 30 and class 3 might be 50? With all summing to 100? I think that would be ideal. 
Any help on CRNNs are highly appreciated.
