When I add weight decay to achieve L2 regularization via torch.optim, what is the constant before alpha in the formula below, is it 2 * alpha or 1/2 * alpha or simply alpha? How can I find the official document for this?
I refer to the code, it seems is alpha
