How to do profiling on pytorch data loader worker process?

1

I use a simple profiling code to profile my training process.

	import cProfile, pstats
	cProfile.run("main()", "{}.profile".format(__file__))
	s = pstats.Stats("{}.profile".format(__file__))
	s.strip_dirs()
	s.sort_stats("time").print_stats(10)

and got something like this

image
image902×270 75.1 KB

When I did some reading on {method ‘acquire’ of ‘_thread.lock’ objects}, apparently this just shows what the parent process is calling and not the child process. So is it possible to do profiling on the data loader worker process?

3 Likes
2

Hello, did you find the solution to your question? I’m having the same problem about pytorch data loader worker process profiling like you.