How does PyTorchs SGD work?

[tataganesh]

The “stochastic” part in SGD comes from computing the gradient for mini-batches of the dataset, since Gradient Descent involves calculating the gradient for the full dataset.
This forum post might be helpful.

Is the SGD in Pytorch a real SGD?

After looking up the code of Pytorch’s SGD: Pytorch’s SGD it seems (excuse me in advance if my assertion is wrong) that the SGD is not a real SGD. Indeed, the way gradients are accumulated and more especially the order wherein they are accumulated is up to the user. Thereupon where is the randomness or stochastic character of this gradient descent? According to Wikipedia: SGD the examples must be randomly shuffled in the training set after being used to compute the gradient. However here th…