Hi,
If you have an input Tensor a, you should replace torch.FloatTensor(2,2) by a.new(2,2) to create a Tensor of the same type as a.
If you want to created tensor to be zeroed out, you can do b = a.new(2,2).zero_().
This will work for a being any type (cuda included).
If your second tensor already exist, you can also look into the type_as method to change the type of a tensor to the type of another tensor.
In you case it was not working because torch.Tensor.new(X.data, torch.zeros(2, 2)) is equivalent, for X.data being a cuda.FloatTensor to torch.cuda.FloatTensor(torch.FloatTensor(2,2).zero_()) meaning that you try to create a cuda tensor from a cpu tensor which is not allowed.