I’m trying to run two autograd.backward() operations in parallel on two separate computational graphs - but for some reason it ‘skips’ without giving an error message.
def local2(target,inp_init):
inp_data_init=inp_init.detach()
optimizerloc2.zero_grad()
outputloc2 = network_local2(inp_data_init)
lossloc2 = torch.nn.functional.nll_loss(outputloc2, target)
print(“message1”)
lossloc2.backward()
print(“message2”)
optimizerloc2.step()
C=P[0]
def local1(target,data,inp_init):
back_prop_init=inp_init.detach()
optimizerloc1.zero_grad()
output_loc1=network_local1(data)
output_loc1.backward(torch.FloatTensor(back_prop_init))
print(“random”)
optimizerloc1.step()
output_loc1=network_local1(data)
D=X[-1]
for loop here:
p = mp.Process(target = local2, args=(target,D))
q = mp.Process(target = local1, args=(target,data,C))
q.start()
p.start()
p.join()
q.join()
This snippet will print “random” and “message1” but not “message2” - it appears to ‘skip’ the backward() part. If I use locks, the same thing happens except it gets stuck on the backward() part.
Edit: Found a solution : https://github.com/pytorch/pytorch/issues/3966