Cannot create tensors from image in subprocess

vision
1

I try to run some parallel subprocesses to do some (cpu-based) post processing of network outputs. However, my attempts fail already at the data loading step. I cannot turn the images into torch tensors within the subprocesses. A minimal failing example is this:

import numpy as np
import torch
import torch.multiprocessing as mp
from skimage import io

def show_np():
    img_name = '000000000139.jpg'
    image = io.imread(img_name)
    print(image.shape)


def show_torch():
    img_name = '000000000139.jpg'
    image = io.imread(img_name)
    tensor = torch.tensor(image)
    print(tensor.shape)


show_np()
show_torch()


processes = []
for i in range(3):
    p = mp.Process(target=show_np)
    p.start()
    processes.append(p)

for p in processes:
    p.join()

processes = []
for i in range(3):
    p = mp.Process(target=show_torch)
    p.start()
    processes.append(p)

for p in processes:
    p.join()

This code prints the size when I run the function with only the numpy array or without subprocesses, but when I try to transom into torch.tensor in the subprocess evaluation just stops.

To run the code you need to put an image with the given name into the current folder of course.

Any help is very welcome!

2

You might need to use the if-clause guard as given in the last example here. These docs also mention common issues with deadlocks etc.