Hi, I’m trying tensor subclassing examples in subclass_zoo/empty_tensor.py at main · albanD/subclass_zoo (github.com). The custom tensor class EmptyTensor does not have an actual tensor storage but returns dummy data on the fly. The behavior is implemented by __torch_dispatch__.
The custom tensor class behaves as expected. However, I am confused with a behavior when it is passed to torch.nn.Parameter.
rt = torch.randn(2, 2)
et = EmptyTensor(rt)
empty_weight = torch.nn.Parameter(et)
print(f"empty_weight={empty_weight.__class__} param?={isinstance(empty_weight, torch.nn.Parameter)}")
This displays:
empty_weight=<class 'empty_tensor.EmptyTensor'> param?=True
The expected class of empty_weight is torch.nn.Parameter. In addition, isinstance says it is an instance of torch.nn.Parameter though EmptyTensor is not a subclass of torch.nn.Parameter (it is a subclass of Tensor.
You can do the same with a normal tensor.
rt = torch.randn(2, 2)
real_weight = torch.nn.Parameter(rt)
print(f"real_weight={real_weight.__class__}")
As expected, the result is
real_weight=<class 'torch.nn.parameter.Parameter'>
Can anyone help me understand what is going here?