Torch tensor of objects

If I have a class

class Operation():
    def __init__(self):
        self.layer = nn.Linear(16*16,8,bias=True)

    def eval(self,input):
        return = torch.relu(self.layer(input))

How can I create a 2D torch tensor of this type ? For example, the equivalent of the follwing in numpy;

matrix = np.empty((x_dim,y_dim),dtype=object)
 for x in range(x_dim):
    for y in range(y_dim):
        matrix[x,y] = Operation()

I have tried in pytorch but seems like object is not a valid data type…

If it is really not possible is there another way it can be done ? Using ModuleList or ModuleDict while being able to refer to objects within as coordinates ?