Should custom Dataset always return tensors?

I got no issue when just returning an integer so I am not sure why you are getting the error.

Maybe this post will make things clear about backprop problem I mentioned.