Hello PyTorch forums, and thanks for all the help you have provided me so far!
I’m trying to visualize the features (filters) of my Resnet CNN when applied to a binary classification problem. The goal is to see somehow how my model is interpreting images of sawn timber when classifying them as either A or B. Ideally I would like to see a feature map highlighting (big weights) things like knots as the deciding factor for the classification.
My current attempt is using the torch-cam package from github, which to my understanding is the go-to method, yes? My code with some comments/thoughts is:
!pip install torchcam from torchcam.cams import SmoothGradCAMpp # model.eval() is exectured prior to the code in this post. model.to('cpu') # Using a fixed sample image img = Image.open('/content/drive/My Drive/data/Lundgrens collection/A/63546298.png') # Transformations are resize, to tensor, and normalize. # Shape becomes [1, 3, 50, 783] which is very wide! img_tensor = torch.unsqueeze(transformations(img),0) # The image colors look very strange, I think this is due to the normalization, which is ok I guess. fig, ax = plt.subplots(figsize=(36, 6)) ax.imshow(np.transpose(img_tensor, (1, 2, 0)), interpolation='nearest') ### Layer 1 # Hook your model before the forward pass cam_extractor = SmoothGradCAMpp(model, 'layer1') # What is "layer1" here? See below. # By default the last conv layer will be selected out = model(img_tensor) # Retrieve the CAM activation_map = cam_extractor(out.squeeze(0).argmax().item(), out) fig, ax = plt.subplots(figsize=(36, 6)) ax.imshow(activation_map, interpolation='nearest')
The problem is that if I run this code repeatedly the feature visualization is never the same. It is completely different, seemingly as if a random image is being sent to the model. Furthermore, I don’t think I understand the hooking of “layer1”, since as shown below layer1 consists of several sub-layers. So maybe I just don’t understand how to use the torch-cam package? Reading the source code is a bit too complicated to me and I don’t understand it enough to tell what is wrong here.
conv1.weight bn1.weight bn1.bias layer1.0.conv1.weight layer1.0.bn1.weight layer1.0.bn1.bias layer1.0.conv2.weight layer1.0.bn2.weight layer1.0.bn2.bias layer1.1.conv1.weight layer1.1.bn1.weight layer1.1.bn1.bias layer1.1.conv2.weight layer1.1.bn2.weight layer1.1.bn2.bias layer2.0.conv1.weight ...
Thanks for any help or feedback you could provide! I’m no expert on pytorch so don’t hold back if I’m doing something silly.