I am printing summary of a model using summary module from torch.summary.But in the result of column ‘output shape’ of MNIST dataset i am getting output as
The -1 in your example would represent an arbitrary number for the batch dimension.
While the other shapes are fixed to the shown values, you can use any batch size for this model.
I am not sure that is the case. When using the torch.summary, the batch size is shown by the second position, right? In this example, the second column of the tensor appears to be digits power of two.and the other two correspond to H and W of the images in MNIST.
I do not have the answer, I actually have the same question. Can it be related with some flattening operation instead?
I tested my own architecture with several torch vision datasets and for all of them I obtain this -1 on the first column no matter the bs I choose.
For a new dataset that I am interested on, and for which I needed to write a custom dataset class, I get weird results, and in addition, torch.summary shows that my layers lack that initial -1, and I am trying to get it, reason why I need to understand where does it come from.
I’m assuming that summary() outputs the tensor shapes in the default format.
For 2-dimensional layers, such as nn.Conv2d and nn.MaxPool2d, the expected shape is given as [batch_size, channels, height, width]. dim1 would therefore correspond to the channels, which are often chosen to be powers of 2 for performance reasons (“good” indexing is easier for powers of 2).
The -1 would therefore be the batch dimension, which is flexible in PyTorch. I.e. you don’t have to specify the batch size for your model and it will take variable batches as long as you have enough memory etc.
Could you post your result and (if possible) your model definition, so that we could have a look?
#Def the Network and forward prop
dim = 12288 #64*64*3channels
n_h = 7
n_classes = 2 # 2 classes
n_y = 5
class Net(nn.Module):
def __init__(self):
super(Net,self).__init__()
self.input_layer = nn.Linear(dim,n_h)
self.hidden_layer = nn.Linear(n_h,n_y)
self.output_layer = nn.Linear(n_y,n_classes)
def forward(self, x):
x = x.view(-1, dim) #flatten image input
x = F.relu(self.input_layer(x))
x = F.relu(self.hidden_layer(x))
x = self.output_layer(x)#no ReLU to the last layer.
return F.log_softmax(x, dim=1) #convert logits to probabilities that sum up to 1.
net= Net()
print(net)
summary(Net(), input_size=(1,dim), batch_size=bs, device="cpu")
net.state_dict()
type or paste code here
Thanks for the code!
In your first example you are passing the batch_size=bs directly to sumamry(), which will then print it out.
If you skip it and leave it as -1 (default), you’ll get: