Hi there,
I am trying to use a Bayesian inference toolbox where the prior must be given as a list of pytorch distributions. The toolbox grabs this list and samples from it.
The prior can be given either as a list of independent torch distributions or as a multivariate distribution. For example, instead of defining the prior as prior = [Normal(torch.tensor([0.0]), torch.tensor([1.0])), Normal(torch.tensor([0.0]), torch.tensor([1.0]))], I could define it as:
prior = MultivariateNormal(torch.zeros(2), torch.eye(2)).
So far so good. The problem is that my model is hierarchical.
My parameters’ priors are:
$$\log \pi ~ Unif(log(a),log(b))$$
$$\gamma_j | \pi ~ Bernoulli(\pi)$$
$$\beta_j | (gamma_j = 0) ~ \del_0$$ (here \del_0 can in practice be replaced by a normal distribution with very low variance)
$$\beta_j | (gamma_j = 1) ~ Normal(0,\sigma_{\beta}^2)$$,
where a, b and \sigma_{\beta} are known, and \del_0 is a kronecker delta = 1 at 0
I am not sure how to go about setting up a prior distribution that includes all of these hierarchical dependencies.
From searching around a bit I have found out that I can set the distribution for \pi as follows:
import torch.distributions as dist
class LogUniform(dist.TransformedDistribution):
def init(self, lb, ub):
super(LogUniform, self).init(dist.Uniform(lb.log(), ub.log()),
dist.ExpTransform())
REF: PyTorch Issue 11412
But I am not sure how to:
- Pass this distribution to the \gamma distribution
- Pass the \gamma distribution to the \beta distribution and drawing \beta_j in one of two ways, depending on \gamma_j
Any help with this would be greatly appreciated! I also take this opportunity to ask whether it is possible to write in LaTeX in these questions and if so, how, because I have tried inserting the dollar signs but they did not work.
Thank you in advance.