I only backward once.Why this happen?RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward()

autograd
1

I only backward once.Why this happen?If I don’t add detach on radical_template,this code can not run.

 radical_template = []
            for step,img in enumerate(radical_dataloader):
                backbone1.train()
                img = torch.unsqueeze(img,dim=1)
                img = img.to(device)
                feature = backbone1(img)
                radical_template.append(feature)
            radical_template = torch.cat(radical_template,dim=0)
            for step, (imgs, lbs) in enumerate(train_dataloader):
                backbone.train(), classifier.train(),backbone1.train()
                character_template = self.template_manager.sample_random_template(self.opt.training.num_templates).to(device)
                imgs, lbs = imgs.to(device), lbs.to(device)
                out = backbone(imgs)
                out = out.permute(0, 2, 3, 1)
                out = out.view(out.size(0), 64, 128)  # 128 * 16 * 128
                sim = torch.matmul(out, radical_template.transpose(0, 1).detach())  # 128 * 16 * 260
                sim = sim.transpose(1, 2)  # 128 * 260 * 16
                sim = torch.softmax(sim, dim=2)
                output = torch.matmul(sim, out)  # 128 * 260 * 128
                output = torch.sum(output, dim=1)  # 128 * 128
                template = backbone1(character_template, dropout=False,template=True)
                logits,temp,temp1 = classifier(output, label=lbs, template=template)
                loss = torch.nn.functional.cross_entropy(logits, lbs)
                loss = loss + temp + temp1

                accelerator.backward(loss)
2

Hi Yupeng!

I assume that backbone1 is a model with trainable parameters. If so, at
this point radical_template depends on backbone1 and is attached to
a computation graph that leads back to those parameters.

At this point, sim has been detach()ed from any and all computation graphs,
so it connects back to neither backbone1 (through radical_template) nor
backbone (through out).

But without the .detach(), sim will be connected to backbone1.

Without the .detach(), loss is connected to backbone1 through output
and thus sim. (It is separately connected to backbone1 through template,
but that is not the cause of your problem.)

This backward() call – inside the for step loop – frees the full computation
graph attached to loss. Without the .detach(), this includes the portion of
the graph connects radical_template to backbone1. However, the
computation of radical_template occurs outside of the loop, so that part
of the graph is not rebuilt when you execute the loop a second time. Therefore
when you call accelerator.backward (loss) a second time, you will try to
backpropagate through the radical_template graph that has already been
freed (and not rebuilt). Hence the error.

How to fix this will depend on what you are trying to do. (You don’t show
your optimization step.) Are you trying to optimize backbone1 via loss
through the connection of radical_template to backbone1? In that case
you should move the computation of radical_tensor inside of the
for step loop in order to rebuild the graph. (backbone1 will have changed
because you have optimized it, so recomputing the graph would make sense.)

If you are not optimizing backbone1, you should probably detach() both
radical_template and template so that no graph connects back to
backbone1.

Best.

K. Frank

1 Like
3

Hi K. Frank!
Thanks for your time!
I have changed the code to solve the problem you mentioned.And it do work.
I still have a question about this sentence.

At this point, sim has been detach() ed from any and all computation graphs,
so it connects back to neither backbone1 (through radical_template ) nor
backbone (through out ).

Why sim is detached from out ?

Looking forward to your reply.

Best.

Zhou Yupeng

4

Hi Yupeng!

That’s my mistake – sim is not detached from out.

I misread the location of some of the parentheses in this line of code:

sim = torch.matmul(out, radical_template.transpose(0, 1).detach())

The second argument to matmul() is detached from its connection to
backbone1, so sim is not connected to backbone1. But sim is, indeed,
connected to out and hence to backbone.

Sorry for the confusion.

Best.

K. Frank

1 Like
5

Thanks for your valuable time again!

Best.

Zhou Yupeng