I wanted to share a blog post that I (along with my colleagues at Hugging Face) have written on the torch.profiler. It covers profiling a matmul and addition operation. I think this could serve as a great tutorial for anyone beginning their journey into profiling, optmization, or even kernel developement.
Link: Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler
Any feedback, criticisms, or observations are very welcome!
PS: I don’t know if the category was set correctly or not, please feel free to let me know if that is not the case.