Custimize your own optimizer?

I want to code my own optimizer for the CNN like Particle Swarm Optimization (PSO) and use it instead of the others(Adam, SGD …), how can I start I am stuck.
Any suggestion would be much appreciated.

I am also new to Pytorch but as I understood in this thread people implemented custom optimizers.

1 Like