I want to know how to call the loader function of Pytorch in the Python/C API

I want to use Pytorch’s loader function using the Python/C API.
The following steps (1) to (3) are used to process. (3) is the problem.

(1) Call a python file from C++.
C++ file is a ConsoleApplication1.cpp.

(2) Load the loader function of pytorch from the python file.
Python files are test1.py and DataSetSample.py.

(3) Repeated processing does not work. (Pytorch’s loader function doesn’t work properly.)
Iteration in line 23 of test1.py doesn’t work. (line 24 works.)
Curiously, up to line 22 of test1.py is infinitely processed.
If you comment out the matplotlib in line 7 of test1.py, it stops at line 22.(test1.py)

I would like to know how to execute the Pytorch loader function in line 23.(test1.py)


Environment

◆OS
 Windows10
◆Environment related to C++
 C++ environment was created in Visual Studio Community 2017.
◆Environment related to python
 python   3.6.8
 torch       1.4.0
 torchvision 0.5.0
 matplotlib  3.2.1


This file is ConsoleApplication1.cpp.

#include <stdio.h>
#include <Python.h>
#include <iostream>


using namespace std;

int main() {
	PyObject *pName, *pModule, *pTmp, *pFunc;
	char *sTmp;
	int data;

	Py_Initialize();
	PyObject *sys = PyImport_ImportModule("sys");
	PyObject *path = PyObject_GetAttrString(sys, "path");
	PyList_Append(path, PyUnicode_DecodeFSDefault("executable file")); //Path of executable file
	pName = PyUnicode_DecodeFSDefault("test1");
	pModule = PyImport_Import(pName);
	Py_DECREF(pName);

	if (pModule != NULL) {
		pFunc = PyObject_GetAttrString(pModule, "test");
		pTmp = PyObject_CallObject(pFunc, NULL);
		int pp = 0;
		cin >> pp;

	}
	Py_Finalize();

	return 0;
}

This file is test1.py

# coding: UTF-8

print("call test1")
import torch

import DataSetSample
import matplotlib # this must affect the loop 

#### data loader ####
# Data
print('==> Preparing data..')
dataset = DataSetSample.DataLoad()
loader = torch.utils.data.DataLoader(
    dataset,
    batch_size=5,
    num_workers=1,
    shuffle=True
)
#########################

def test():
    print("========================= forの前")
    for batch_idx, (_, _, _) in enumerate(loader): #can't enter loop
    #for _ in range(5):                            #enter loop
        print("test")

if __name__ == "__main__":
    test()

This file is DataSetSample.py

class DataLoad():
    def __init__(self):
        print("start")

    def __getitem__(self, index):
        return 1, 2, 3
    
    def __len__(self):
        return 3

The result of running the executable after compiling ConsoleApplication1.cpp.
call test1
==> Preparing data..
start
========================= forの前
call test1
==> Preparing data..
start
========================= forの前
call test1
==> Preparing data..
start
========================= forの前
call test1
==> Preparing data..
start
========================= forの前

Repeated processing...

Please help me.
Thank you for reading to the end.

I’m not familiar with this particular use case of executing a Python application from C++, but could you set num_workers=0 and retry it again, as you might be running into multiprocessing issues?

Thank you for your response.
It was really helpful.

I tried reducing num_workers to 1 at the time, but I couldn’t do it.
I followed the advice and set num_workers to 0 and it worked!