Analyzing the inference of a of a model

Yes, it’s possible to check the intermediate activations e.g. via forward hooks as described here.