How to make a heatmap of weights?

1

I would like to look at my decoder weight matrix ( as a heatmap). I am a bit confused about the shapes, shouls I user the 5th matrix? ( the size can be seen in the model.parameters().

Model(
  (encoder): Encoder(
    (model): ModuleList(
      (0): Linear(in_features=6000, out_features=100, bias=True)
      (1): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU()
      (3): Dropout(p=0.2)
    )
  )
  (decoder): Decoder(
    (model): ModuleList(
      (0): Linear(in_features=100, out_features=6000, bias=True)
      (1): BatchNorm1d(6000, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU()
      (3): Dropout(p=0.2)
    )
  )
)

model parameters size:


torch.Size([100, 6000])
torch.Size([100])
torch.Size([100])
torch.Size([100])
torch.Size([6000, 100])
torch.Size([6000])
torch.Size([6000])
torch.Size([6000])
2

You could get more information, if you print the name and parameter using model.named_parameters().
Most likely the first and 5th outputs are the weight matrices from the linear layers.
The other (1dim parameters) would be the bias of the linear layers, weight and bias of the batch norm layers.

3

Thank you !
When I did what you said, the namings are as follow:

encoder.model.0.weight
encoder.model.0.bias
encoder.model.1.weight
encoder.model.1.bias
decoder.model.0.weight
decoder.model.0.bias
decoder.model.1.weight
decoder.model.1.bias

Just to make sure that I got it right, the encoder.model.1.weight encoder.model.1.bias corresponds to the Relu activation?

4

No, ReLU does not contain any parameters. They correspond to the nn.Batchnorm layer in the encoder.
The index in encoder.model.1 is corresponding to the ModuleList index when you print the model.

1 Like