How do I check the number of parameters of a model?

For finding the total number of parameter elements (if you are interested in the total size of the parameter space rather than the number of parameter tensors), I use sum(p.numel() for p in model.parameters())

1 Like

@hughperkins Is this stack overflow answer a reasonable way to handle not double-counting shared parameters?

from prettytable import PrettyTable

def count_parameters(model):
    table = PrettyTable(["Modules", "Parameters"])
    total_params = 0
    for name, parameter in model.named_parameters():
        if not parameter.requires_grad: 
            continue
        param = parameter.numel()
        table.add_row([name, param])
        total_params+=param
    print(table)
    print(f"Total Trainable Params: {total_params}")
    return total_params

count_parameters(model)
1 Like

Final answer:

PyTorch doesn’t have a function to calculate the total number of parameters as Keras does, but it’s possible to sum the number of elements for every parameter group:

pytorch_total_params = sum(p.numel() for p in model.parameters())

If you want to calculate only the trainable parameters:

pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

Answer inspired by this answer on PyTorch Forums.

Note: I’m answering my own question. If anyone has a better solution, please share with us.


For those who are curious about .numel():

https://pytorch.org/docs/stable/generated/torch.numel.html

torch. numel ( input ) → int
Returns the total number of elements in the input tensor.


(copy pasted) reference: python - Check the total number of parameters in a PyTorch model - Stack Overflow

10 Likes

If you use torch_model.parameters(), the layers batchnorm in torch only show 2 values: weight and bias, while in tensorflow, 4 values of batchnorm are shown, which are gamma, beta, (similar to weight and bias in torch) and moving mean and moving var (similar to running mean and running var in torch batchnorm).
The 2 later values (running mean and running var) are non-trainable parameters and they don’t show up in the torch_model.parameters(). If you compare torch_model.parameters() and tf_model.trainable_variables, they should be equal.

1 Like

why would there be double counting? How does one deal with it?