Hi,
I need to asynchronously updates weights of several copies of one network in most of RL algorithms. I tried to write a class function in which an instance of toch::nn::seqential
exists.
Using named_parameters()
I can access the parameters in a network. Now, the question is can I assign another tensor, with same shape, to p.value()
?
For example assume I have tensor w
which has same characteristics as p.value()
does. Does p.value() = w
assigns values in w
to p.value()
?
I tested this procedure as below and it did not work for me:
torch::autograd::GradMode::set_enabled(false);
int m=0;
for (auto &p : net->named_parameters()) {
auto z = p.value(); // note that z is a Tensor, same as &p : net->parameters
auto w = torch::zeros_like(p.value());
if (z.dim()==1){
int first = m;
int last = m + z.size(0);
m += z.size(0);
auto v = slice(weights, first, last);
w+= torch::tensor(v);//.to(cpu_device);
p.value() = w;
}
else if (z.dim()==2){
int first = m;
int last = m + z.size(0)*z.size(1);
m += z.size(0)*z.size(1);
auto v = slice(weights, first, last);
w += torch::reshape(torch::tensor(v), {z.size(0),z.size(1)});//.to(cpu_device);
p.value() = w;
}
}
in which weights
is a std::vector<float>
, and slice
function returns the proper slice of the weights
vector.
I also tried this code
torch::autograd::GradMode::set_enabled(false);
int m=0;
for (auto &p : layers->named_parameters()) {
auto z = p.value(); // note that z is a Tensor, same as &p : layers->parameters
auto w = torch::zeros_like(p.value());
if (z.dim()==1){
int first = m;
int last = m + z.size(0);
m += z.size(0);
auto v = slice(weights, first, last);
w+= torch::tensor(v);//.to(cpu_device);
p->copy_(w);
}
else if (z.dim()==2){
int first = m;
int last = m + z.size(0)*z.size(1);
m += z.size(0)*z.size(1);
auto v = slice(weights, first, last);
w += torch::reshape(torch::tensor(v), {z.size(0),z.size(1)});//.to(cpu_device);
p->copy_(w);
}
}
which updates the values fine, but it breaks the gradients link and after this step I loose all gradients.
Thanks,
Afshin