at which layer of a neural network, do I know the pattern that it has learnt, for example if I pass an image of cat to a neural network, and it gives it label of cat, by giving high probability at the final layer, which layer would show the pattern of cat it has learnt?
Similarly if I pass image of dog to the neural network, and it correctly labels it as dog, which layer would show the pattern it has learnt?
same thing for capsule networks also, how do I see what pattern has a capsule learnt?
The kernels and activations do not directly show a cat or dog feature.
However, you could e.g. try to visualize the gradients using e.g. flashtorch.