Using torch.Tensor over multiprocessing.Queue + Process fails

eacousineau · September 8, 2020, 3:20pm

Hi @Yi_Zhang, it will be the latter:

For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

This is the toy example I had used to check it for myself:

gist.github.com

https://gist.github.com/EricCousineau-TRI/4cedd0847ffc5acdd992403beffbc7b1

does_indeed_work.py

# Using torch==1.4.0

import numpy as np
import torch
import torch.multiprocessing as mp

torch.set_grad_enabled(False)

def target(inputs, outputs):
    x = inputs.get()

This file has been truncated. show original

does_not_work.py

# Using torch==1.4.0

import numpy as np
import torch
import torch.multiprocessing as mp

torch.set_grad_enabled(False)

def target(inputs, outputs):
    x = inputs.get()

This file has been truncated. show original

Feel free to modify it to test for multi-input multi-output queue processing.