# Representation projection to 2D plot

Let’s say I have a model, AlexNet:

``````class AlexNet(nn.Module):

def __init__(self, num_classes: int = 1000) -> None:
super(AlexNet, self).__init__()
self.features = nn.Sequential(
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2),
nn.ReLU(inplace=True),
nn.ReLU(inplace=True),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3, stride=2),
)
self.classifier = nn.Sequential(
nn.Dropout(),
nn.Linear(256 * 6 * 6, 4096),
nn.ReLU(inplace=True),
nn.Dropout(),
nn.Linear(4096, 4096),
nn.ReLU(inplace=True),
nn.Linear(4096, num_classes),
)

def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.features(x)
### I want to extract `x` and plot the representation in a 2D space
x = self.avgpool(x)
x = torch.flatten(x, 1)
x = self.classifier(x)
return x
``````

Is there any way to visualize the weights learn from `self.features` layer and plot into a heatmap that look like this (taken from umap):

Where each color is the class labels of the data points from the `test` dataloader. Preferably with Python packages and plot with matplotlib and without tensorboard.

Yes, you could use forward hooks to store the activation from a specific layer (I assume you meant activation instead of weight) as described here and use the linked `umap` library to reduce the dimensionality. Once this is done, you could use `matplotlib` to create a scatter plot (take a look at the examples at the bottom of the docs).

1 Like