I am trying to implement a variation of VAEs in which the latent space variable z is derived from a Gibbs sampling routine. To generate z, I initialize it by sampling from a gaussian distribution. Then, I run the Gibbs sampling procedure 2000 times to get past the burn-in period. Then I again sample from the Gibbs sampler routine to get the z which I send to the decoder of the VAE. The problem is that I am unable to do backpropogation through the VAE anymore. I defined a variable flag_initialize=1 in the init function, and made it run the first time. Then, I had hoped to use that as the initial value for all further computations. The code in the forward function is the following:
if self.flag_initialize==1:
z1_prior,z2_prior = self.gibbs.initialize_prior_sample(self.g11,g22,self.g12)
self.flag_initialize=0
self.z1_prior =z1_prior
self.z2_prior =z2_prior
self.z1_prior,self.z2_prior = self.gibbs.prior_sample(self.z1_prior,self.z2_prior,self.g11,g22,self.g12)
When I run the code, I get an error which reads,
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.
Any help on how I can do a backpropogation with Gibbs sampling? Thank you!