Manager.queue get() error

RuntimeError: invalid device pointer: 0x204aa4200 at /opt/conda/conda-bld/pytorch_1544173631724/work/aten/src/THC/THCCachingAllocator.cpp:301

When I use torch.multiprocessing.manager.queue to share some data generated from different processes to a background thread, then I get the error when the background thread try to use queue.get() data from the shared queue. The shared data is Cuda tensors. Any suggestions? Thanks.