If you want the tensor to have one column repeated all over, you can use expand:
x = torch.arange(5)[None, None, None, :].expand(2, 3, 4, 5)
or, if you prefer,
x = torch.arange(5).view(1, 1, 1, -1).expand(2, 3, 4, 5)
Both create singleton dimensions where you want the expansion to go to (with indexing or view) and then expand (with stride 0) to the desired size (similar to what is done in broadcasting).
If you want the exact same result (without strides) as the stacking, you can add .contiguous() at the end.