I am wondering what is the best way to accelerate Pytorch or what implementation is most commonly used?
write C/C++? pytorch lightning? or others?
what is the ecosystem in this field? people often choose what method to accelerate their pytorch training?
Pytorch is “accelerated” and it’s really fast. The major problem is not about pytorch but the python code you write around it.
Check this blogpost Object Detection from 9 FPS to 650 FPS in 6 Steps | paulbridger.com to have an idea where the problems come from. The only thing you can optimize is the dataloader. Still depends a lot on the field.
The Performance Guide could also be useful for general best practices.
Thanks!! it helps a lot!!