I am using Libtorch for a genetic algorithm where I need to clone individuals and mutate the clones. I have written a method to do this. This method is one of the most used functions in my code and I have found that removing my clone operation in favour of a copy operation speeds up the code almost 5x.
So I think I might be doing something very wrong. I cannot use a simple copy operation as the changes in copied individuals will reflect on the original. I’ve added a small toy example to show-case this issue:
struct NodeTest : torch::nn::Module {
public:
NodeTest(){
value = register_parameter("val", torch::rand({1}, torch::TensorOptions().dtype(torch::kFloat)).requires_grad_(true));
}
void change_value(){
this->parameters()[0].data() = torch::rand({1}, torch::TensorOptions().dtype(torch::kFloat));
}
torch::Tensor value;
};
struct IndividualTest {
public:
IndividualTest() {
input_nodes.emplace_back(NodeTest());
}
IndividualTest clone(){
IndividualTest new_ind = IndividualTest();
new_ind = *this;
new_ind.input_nodes[0] = NodeTest();
new_ind.input_nodes[0].parameters()[0].data() = this->input_nodes[0].parameters()[0].data();
return new_ind;
}
void print(){
cout<<this->input_nodes[0].parameters()[0].data().item<float>()<<endl;
}
void change_value(){
this->input_nodes[0].change_value();
}
std::vector<NodeTest> input_nodes;
};
IndividualTest original = IndividualTest();
IndividualTest copy = original;
cout<<"original before: "<<endl;
original.print();
copy.change_value();
cout<<"copy: "<<endl;
copy.print();
cout<<"original after: "<<endl;
original.print();
IndividualTest clone = original.clone();
cout<<"#######new test#########"<<endl;
cout<<"original before: "<<endl;
original.print();
clone.change_value();
cout<<"clone: "<<endl;
clone.print();
cout<<"original after: "<<endl;
original.print();
In my actual code an individual contains many nodes, but the node structure is approximately the same. How can I avoid this costly cloning operation? The registered parameters all have the same memory address when I copy them (like a shared pointer?). I need them to be unique for each individual