Hi all. I have forked some processes to manipulate a shared model. When I do something in each process like following:
shared_model.fc.weight.data += 1
After joining all process, the final weight of shared_model varies every time. This tells us directly write the data of a shared model doesn’t guarantee your writing protection.
So I wonder other operations, like optimizer.step() to update all data of a model, has a writing protection? If not, we might need to write our our Lock to guarantee that.