Hello Haroune and Henry!
I don’t think that there is a simple, one-stop-shopping way of doing
this that is completely reliable. Note that activations also inherit
from Module
.
Losses, however, (appear to) inherit (possibly indirectly) from
torch.nn.modules.loss._Loss
(which, in turn, inherits from
Module
). You could look for that.
You might consider counting parameters()
on the theory that any
self-respecting model has parameters (although some activation
functions also have parameters).
Also, you have to be clear about what you mean by a “model.”
Would you consider a single Linear
to be a model? Arguably
it would be. How about a single Softmax()
. This seems more
of a stretch, but how does it behave any differently than a
Linear
? (One difference is that it doesn’t have any parameters.)
This script illustrates some of these points:
import torch
print (torch.__version__)
model_linear = torch.nn.Linear (3, 5)
model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
loss_mse = torch.nn.MSELoss()
loss_ce = torch.nn.CrossEntropyLoss()
act_softmax = torch.nn.Softmax()
act_prelu = torch.nn.PReLU()
print (sum (1 for _ in model_linear.parameters())) # count parameters
print (sum (1 for _ in model_sequential.parameters())) # count parameters
print (sum (1 for _ in loss_mse.parameters())) # count parameters
print (sum (1 for _ in loss_ce.parameters())) # count parameters
print (sum (1 for _ in act_softmax.parameters())) # count parameters
print (sum (1 for _ in act_prelu.parameters())) # count parameters
print (model_linear.__class__.__bases__) # immediate superclass
print (model_sequential.__class__.__bases__) # immediate superclass
print (loss_mse.__class__.__bases__) # immediate superclass
print (loss_ce.__class__.__bases__) # immediate superclass
print (act_softmax.__class__.__bases__) # immediate superclass
print (act_prelu.__class__.__bases__) # immediate superclass
print (model_linear.__class__.__mro__) # full class hierarchy
print (model_sequential.__class__.__mro__) # full class hierarchy
print (loss_mse.__class__.__mro__) # full class hierarchy
print (loss_ce.__class__.__mro__) # full class hierarchy
print (act_softmax.__class__.__mro__) # full class hierarchy
print (act_prelu.__class__.__mro__) # full class hierarchy
Here is the output:
>>> import torch
>>> print (torch.__version__)
1.6.0
>>> model_linear = torch.nn.Linear (3, 5)
>>> model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
>>> loss_mse = torch.nn.MSELoss()
>>> loss_ce = torch.nn.CrossEntropyLoss()
>>> act_softmax = torch.nn.Softmax()
>>> act_prelu = torch.nn.PReLU()
>>> print (sum (1 for _ in model_linear.parameters())) # count parameters
2
>>> print (sum (1 for _ in model_sequential.parameters())) # count parameters
2
>>> print (sum (1 for _ in loss_mse.parameters())) # count parameters
0
>>> print (sum (1 for _ in loss_ce.parameters())) # count parameters
0
>>> print (sum (1 for _ in act_softmax.parameters())) # count parameters
0
>>> print (sum (1 for _ in act_prelu.parameters())) # count parameters
1
>>> print (model_linear.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (model_sequential.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (loss_mse.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.loss._Loss'>,)
>>> print (loss_ce.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.loss._WeightedLoss'>,)
>>> print (act_softmax.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (act_prelu.__class__.__bases__) # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (model_linear.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.linear.Linear'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (model_sequential.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.container.Sequential'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (loss_mse.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.loss.MSELoss'>, <class 'torch.nn.modules.loss._Loss'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (loss_ce.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.loss.CrossEntropyLoss'>, <class 'torch.nn.modules.loss._WeightedLoss'>, <class 'torch.nn.modules.loss._Loss'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (act_softmax.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.activation.Softmax'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (act_prelu.__class__.__mro__) # full class hierarchy
(<class 'torch.nn.modules.activation.PReLU'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
Best.
K. Frank