# Minimal working example of optim.SGD

**URL:** <https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623>\
**Category:** Uncategorized\
**Created:** [December 28, 2017, 2:57am UTC](https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623 "2017-12-28T02:57:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kq3rHCt9gK](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/e47c2d/32.png) [@kq3rHCt9gK](https://discuss.pytorch.org/u/kq3rHCt9gK)\
**Post date:** [December 28, 2017, 2:57am UTC](https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623/1 "2017-12-28T02:57:10Z")

</div>

I’d like to learn how to use SGD. But I have not found a minimal working example. It must be a piece of code working on its own. And it should be minimal in the sense that anything that can be deleted without affecting the usage of SGD should be deleted.

By this definition, the example at [http://pytorch.org/docs/master/optim.html](http://pytorch.org/docs/master/optim.html) is not working.

The example at the following URL is not minimal.  
[http://pytorch.org/tutorials/beginner/nlp/deep\_learning\_tutorial.html#optimization-and-training](http://pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html#optimization-and-training)

Could anybody show a MWE? Thanks.

---

<div class="post-metadata">

**Author:** ![richard](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/richard/32/1861_2.png) [@richard](https://discuss.pytorch.org/u/richard)\
**Post date:** [December 28, 2017, 3:32am UTC](https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623/2 "2017-12-28T03:32:24Z")

</div>

Do you want to learn about why SGD works, or just how to use it?

I attempted to make a minimal example of SGD. I hope this helps!

```auto
import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable

# Let's make some data for a linear regression.
A = 3.1415926
b = 2.7189351
error = 0.1
N = 100 # number of data points

# Data
X = Variable(torch.randn(N, 1))

# (noisy) Target values that we want to learn.
t = A * X + b + Variable(torch.randn(N, 1) * error)

# Creating a model, making the optimizer, defining loss
model = nn.Linear(1, 1)
optimizer = optim.SGD(model.parameters(), lr=0.05)
loss_fn = nn.MSELoss()

# Run training
niter = 50
for _ in range(0, niter):
	optimizer.zero_grad()
	predictions = model(X)
	loss = loss_fn(predictions, t)
	loss.backward()
	optimizer.step()

	print("-" * 50)
	print("error = {}".format(loss.data[0]))
	print("learned A = {}".format(list(model.parameters())[0].data[0, 0]))
	print("learned b = {}".format(list(model.parameters())[1].data[0]))

```

---

<div class="post-metadata">

**Author:** ![kq3rHCt9gK](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/e47c2d/32.png) [@kq3rHCt9gK](https://discuss.pytorch.org/u/kq3rHCt9gK)\
**Post date:** [December 29, 2017, 1:07am UTC](https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623/3 "2017-12-29T01:07:25Z")

</div>

Here is a more minimal MWE.

```auto
import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable

N = 64

x0 = torch.randn(N, 1)
x = Variable(x0)
y = Variable(x0, requires_grad=False)

A = Variable(torch.randn(1, 1), requires_grad=True)
b = Variable(torch.randn(1), requires_grad=True)

optimizer = optim.SGD([A, b], lr=1e-1)
for t in range(10):
	print '-' * 50 
	optimizer.zero_grad()
	#print A.grad, b.grad
	y_pred = torch.matmul(x, A) + b
	loss = ((y_pred - y) ** 2).mean()
	print(t, loss.data[0])
	loss.backward()
	optimizer.step()
	#print [A, b]

```

---

<div class="post-metadata">

**Author:** ![Thomas\_Poulsen](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thomas_poulsen/32/7309_2.png) [@Thomas\_Poulsen](https://discuss.pytorch.org/u/Thomas_Poulsen)\
**Post date:** [November 24, 2018, 6:19pm UTC](https://discuss.pytorch.org/t/minimal-working-example-of-optim-sgd/11623/4 "2018-11-24T18:19:49Z")

</div>

If you add a line of code to this. “print(predictions.grad)” right after the backward call. Then it will print “None” all the way through. Is that supposed to be? If you read the Autograd documentation i get the impression that a tensor is supposed to come out of it.