Is it possible to have trainable and non-trainable parameters in the same tensor?

autograd
2

No, the requires_grad attribute is set for the tensor itself and you might need to create different subtensors and recreate the final tensor as seen in this example.