I know there are similar posts. I nearly checked them all. Tried `.detach()`

. and `retain_graph=True`

options. However I couldn’t solve it for my training loop of DCGAN. This is a fairly complex training loop for me so I wouold be glad for any help. I also checked the PyTorch tutorial and created this version.

```
criterion = nn.BCELoss()
discriminator_optimizer = optim.Adam(netD.parameters(), lr=lr, betas=(beta1, 0.999))
generator_optimizer = optim.Adam(netG.parameters(), lr=lr, betas=(beta1, 0.999))
img_list = []
G_losses = []
D_losses = []
iters = 0
start = timeit.default_timer()
for epoch in tqdm(range(EPOCHS), position=0, leave=True):
generator.train()
discriminator.train()
for idx, data in enumerate(tqdm(dataloader, position=0, leave=True)):
img_data = data[0].to(device) # size [1024, 3, 64, 64]
dummy_labels = data[1] # size [1024]
real_labels = torch.full((dummy_labels.size()), 1., dtype=torch.float).to(device) # size [1024]
fake_labels = torch.full((dummy_labels.size()), 0., dtype=torch.float).to(device)
noise = torch.randn(dummy_labels.size()[0], INPUT_VECTOR_DIM, 1, 1).to(device) # size [1024, 100, 1, 1]
discriminator.zero_grad()
discriminator_real_out = discriminator(img_data).view(-1).to(device) # size [1024] .view(-1) to rid unnecessary dimensions
discriminator_real_loss = criterion(discriminator_real_out, real_labels)
discriminator_real_loss.backward()
generator_fake_out = generator(noise) # size [1024, 3, 64, 64]
discriminator_fake_out = discriminator(generator_fake_out.detach()).view(-1).to(device)
discriminator_fake_loss = criterion(discriminator_fake_out, fake_labels)
discriminator_fake_loss.backward()
discriminator_loss = discriminator_real_loss.item() + discriminator_fake_loss.item()
discriminator_optimizer.step()
generator_loss = criterion(discriminator_fake_out, real_labels)
generator_loss.backward()
generator_optimizer.step()
generator.zero_grad()
```

Problem is from this line `generator_loss.backward()`

. Also when I remove the line `discriminator_optimizer.step()`

the problem disappears.

Full error: `RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.`