Hi PyTorch community,
I recently published a modified PyTorch Adam optimizer implementation that achieves up to a +40% CUDA and +160% Mac MPS speedup in step execution and training throughput.
Free size-limited fully working benchmark example and code at GitHub - yangofzeal/adam: 40% faster replacement for torch Adam Optimizer on cpu and GPU · GitHub