I am trying to parallelize my code on Google Colab like this:
from concurrent.futures import ProcessPoolExecutor
from time import sleep, time
from torch.autograd import Variable
def f(x):
chromosome_init = np.random.rand(30000)
return single_run(images, labels, num_epochs, chromosome_init, fitness)
L = list(range(1))
if __name__ == '__main__':
begin = time()
with ProcessPoolExecutor(max_workers=1) as pool:
# result = sum(pool.map(f, L))
result = np.sum(np.fromiter(pool.map(f, L), float))
end = time()
print(f"result = {result} and time = {end-begin}")
The function single_run is a training function and right now I simplified it to this:
def single_run(images, labels, num_epochs, chromosome, fitness):
with torch.no_grad():
for layer in cnn.state_dict():
cnn.state_dict()[layer].data.fill_(2)
return 1
When I run the code in Colab I get this error
I have identified that the error shows up when I assign weights as “cnn.state_dict()[layer].data.fill_(2)”. I have tried to reset CUDA calls. Any ideas how to debug/cure it are appreciated!
Thanks in advance!