Hi, I implementing both Genetic algorithms and Particle Swarm as a optimizer for my network. I just curious that since GA and PSO dont need gradient for searching for an optimum that mean the tensor of weight in model also should be the tensor that requires_grad is False right and do I still need to put with torch.no_grad() ?? and do is there anything i should be aware of when using non-gradient method in pytorch?Thank you.
1 Like