Storing activation of only one hidden layer in a NN

1

Though this is in the concatenation of my previous question, but as the concept is different so I feel like asking another one here.

For better suggestions for my question let me rewrite the query again. I am using a Shallow neural network (a dense NN with a single hidden layer) and after the model, training needs to store the activations of the hidden layer. I got that answer in the previous question as you all can see. But now my next question is how to save the final outputs as I was training in batches. Actually, I want to store these output layer activations for all the samples in my dataset (2000). I think that is quite impossible as the model validation gives the classification results on the basis of trained coefficients only, but can we store their output activations too for that only hidden layer I am using in the network @ptrblck :slight_smile: sir I know you can help here.

Please guide.

2

You could iterate the dataset and append all outputs in e.g. a list to store them. Make sure to call detach() on the output tensor to avoid storing the entire computation graph in this list, which would otherwise increase the memory usage.