Hello, I am trying to create an RNN that will be able to detect anomalies in time-series data. In particular, looking for glitches in voltage/time plots. I currently am trying to implement a very simple version of this to just make sure that it is doable, but I continue to run into issues when trying to create and train the model.
Unlike other anomaly detection rnn’s that I have come across, I am particularly interested in not just being able to predict if a plot has an anomaly in it, but the time stamps that correlate to the anomaly. So, for each datapoint per plot in my training data, there is a corresponding label of 0 if there is no glitch at the current point, and a 1 if there is a glitch occuring.
Below is the code for my model:
class MorphModel(Module):
def __init__(self, n_layers):
# Define LSTM layer
self.lstm = nn.LSTM(2, 3 * 2 * 4, n_layers, batch_first=True)
# Define Linear layer
self.l_s = nn.Linear(3 * 2 * 4, 1)
# Define Sigmoid function (Output Layer)
self.sigmoid = nn.Sigmoid() # TODO look into best way of initializing this
def forward(self, input):
print("forward inputs: ", list(input[0].size())[0])
print("lstm in")
# lstm input: sequence length, batch size, 1
lstm_out, _ = self.lstm(torch.Tensor(list(input[0].size())[0],2,2)) # we want the hidden outputs (h_n)
print("lstm output")
# do some manipulation to be able to feed into linear?
print("lstm out: ", lstm_out.size())
lstm_out.reshape(-1, 2 * 24)
print("lstm out after size: ", lstm_out.size())
sig_input = self.l_s(lstm_out.view(13,2,-1))
print("linear layer out")
print(sig_input.size())
# will be batch size * len of input
# may need two outputs
output = self.sigmoid(sig_input)
print("sigmout out")
print(output.size())
return output
I set up my learner as such:
learn = Learner(dls, MorphModel(2), loss_func=CrossEntropyLossFlat())
#metrics=accuracy, cbs=ModelResetter)
learn.summary()
And, the summary shows:
forward inputs: 13
lstm in
lstm output
lstm out: torch.Size([13, 2, 24])
lstm out after size: torch.Size([13, 2, 24])
linear layer out
torch.Size([13, 2, 1])
sigmout out
torch.Size([13, 2, 1])
MorphModel (Input shape: 2)
============================================================================
Layer (type) Output Shape Param # Trainable
============================================================================
[]
LSTM
____________________________________________________________________________
2 x 2 x 1
Linear 25 True
Sigmoid
____________________________________________________________________________
Total params: 25
Total trainable params: 25
Total non-trainable params: 0
Optimizer used: <function Adam at 0x7fa0f2e31710>
Loss function: FlattenedLoss of CrossEntropyLoss()
Callbacks:
- TrainEvalCallback
- Recorder
- ProgressCallback
However, when I run fine_tune() on my learner, I get the following output:
forward inputs: 13
lstm in
lstm output
lstm out: torch.Size([13, 2, 24])
lstm out after size: torch.Size([13, 2, 24])
linear layer out
torch.Size([13, 2, 1])
sigmout out
torch.Size([13, 2, 1])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-12-ca61b3aa75fc> in <module>()
----> 1 learn.fine_tune(1)
21 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
2260 if input.size(0) != target.size(0):
2261 raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
-> 2262 .format(input.size(0), target.size(0)))
2263 if dim == 2:
2264 ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
ValueError: Expected input batch_size (26) to match target batch_size (4).
For my “proof-of-concept” data, each consists of 13 datapoints, which is why there are 13 forward inputs.
However, I really am not sure how to fix this ValueError. I also am not too confident in this approach as RNN’s are a very new concept to me. So, any advice is incredibly helpful. thanks!