How to assign gradients to model parameters manually?

reinforcement-learning
1

In A3C, the gradients of main thread parameters are copied from training threads.
But the gradients of pytorch nn.module cannot be reassigned?
So how to implement this procedure?

BTW. is there a function like tf.gather_nd() in pytorch?
to extract values in specific indexes of the tensor (multidimensional).
Cause right now, the torch.indes_select can only select from one dimension.

2

to answer your second question, you can create a mask tensor and index elements via this mask tensor:

example:

import torch

x = torch.randn(2, 3, 2)

mask = x.gt(0.5)
print(mask)

# mask is a torch.ByteTensor of same shape and size as x

print(x[mask])

Output:


(0 ,.,.) =
  0  0
  0  0
  0  1

(1 ,.,.) =
  0  1
  1  0
  1  0
[torch.ByteTensor of size 2x3x2]


 1.3140
 0.8220
 1.5169
 0.9264
[torch.FloatTensor of size 4]
1 Like
4

Thank you smth.
If I have an index list where same indexes may repeat for several times.

Then I think this method is useless.

5

Did you try gather?

6

But gather always returns a tensor with same dimension as the input right?

7

Yes, that’s the limitation.