# How can i train input (not weight)?

**URL:** <https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569>\
**Category:** Uncategorized\
**Created:** [May 2, 2017, 9:51am UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569 "2017-05-02T09:51:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![yunjey](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yunjey/32/1386_2.png) [@yunjey](https://discuss.pytorch.org/u/yunjey)\
**Post date:** [May 2, 2017, 9:51am UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/1 "2017-05-02T09:51:34Z")

</div>

In GAN, i want to find latent vector z corresponding to the real image. One way to do this is to train z to minimize the error between sampled image and real image.

However, when i ran the code below, Error message appeared: “ValueError: can’t optimize a non-leaf Variable”.

```
targets # target images of shape (batch_size, 3, 64, 64)

z = Variable(torch.randn(batch_size, 100), requires_grad=True).cuda()
optim = torch.optim.Adam([z], lr=0.01) # This causes an error. (if i delete cuda() above, it solve the problem)

samples = generator(z) # sampled images
loss = torch.mean((targets - samples)**2)
loss.backward()

```

How can i solve this problem?

Thanks,

---

<div class="post-metadata">

**Author:** ![SherlockLiao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sherlockliao/32/6672_2.png) [@SherlockLiao](https://discuss.pytorch.org/u/SherlockLiao)\
**Post date:** [May 2, 2017, 10:27am UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/2 "2017-05-02T10:27:47Z")

</div>

you can refer to this article [http://pytorch.org/tutorials/advanced/neural\_style\_tutorial.html#sphx-glr-advanced-neural-style-tutorial-py](http://pytorch.org/tutorials/advanced/neural_style_tutorial.html#sphx-glr-advanced-neural-style-tutorial-py)

---

<div class="post-metadata">

**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\
**Post date:** [May 3, 2017, 3:18am UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/3 "2017-05-03T03:18:13Z")

</div>

> [@yunjey](#):
>
> z = Variable(torch.randn(batch\_size, 100), requires\_grad=True).cuda()

you have a mistake in this code.  
It should be:

```auto
z = Variable(torch.randn(batch_size, 100).cuda(), requires_grad=True)

```

---

<div class="post-metadata">

**Author:** ![BestSonny](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bestsonny/32/1611_2.png) [@BestSonny](https://discuss.pytorch.org/u/BestSonny)\
**Post date:** [June 29, 2018, 8:49pm UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/4 "2018-06-29T20:49:50Z")

</div>

In pytorch 0.4.0, I want to know how do we train this model where i assign one additional tensor  
task\_parameters = torch.ones(2).cuda().requires\_grad\_()  
to learn the task weights via

precision = torch.exp(-1\*self.task\_parameters)  
total\_loss = loss \* precision[0] + self.task\_parameters[0]+ loss\_quality \* precision[1] + self.task\_parameters[1]

Now, how do I update task\_parameters in optimizer  
optimizer = optim.SGD(itertools.chain(list(model.parameters()),[task\_parameters]), lr=args.lr, momentum=args.momentum)???

---

<div class="post-metadata">

**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\
**Post date:** [June 30, 2018, 12:13pm UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/5 "2018-06-30T12:13:05Z")

</div>

@BestSonny `task_parameters` should be a leaf Tensor to be optimized, i.e. it’s `.grad` field should be filled up with gradients. Hence, you define it like this:

```auto
task_parameters = torch.ones(2, device='cuda', requires_grad=True)

```

OR

```auto
task_parameters = torch.ones(2).cuda() # what you have in your code
task_parameters.detach().requires_grad_() # makes it a leaf variable and sets requires_grad

```

---

<div class="post-metadata">

**Author:** ![BestSonny](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bestsonny/32/1611_2.png) [@BestSonny](https://discuss.pytorch.org/u/BestSonny)\
**Post date:** [July 1, 2018, 2:27pm UTC](https://discuss.pytorch.org/t/how-can-i-train-input-not-weight/2569/6 "2018-07-01T14:27:40Z")

</div>

Thank you very much.