Jun 24, 2022 · With that in mind, the ELBO can be a meaningful lower bound on the log-likelihood: both are negative, but ELBO is lower. How much lower? The KL divergence from the conditional distribution. I don’t see where you think the figure is indicating that it should be positive. The bottom of the diagram isn’t 0.

Oct 1, 2022 · Here is my question: how does maximizing ELBO lead to a good/correct posterior predictive distribution ...

Jan 29, 2025 · In general for any continuous probability density function (PDF), by convention small case letter like $\mathbf{x}_0$ is always 'realization' of its corresponding random variable(s) like $\mathbf{X}_0$, though their realized value is unknown expressed as a symbol.

Jan 22, 2024 · ELBO is a quantity used to approximate the log marginal likelihood of observed data, after applying ...

Nov 30, 2023 · Stack Exchange Network. Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Oct 19, 2022 · $\begingroup$ thanks! so if the ELBO itself is tractable - why does rocca show that we are optimizing the KL divergence? he shows that we can develop the KL divergence between the approximate posterior and the true posterior (which is indeed unknown) as a sum of the data likelihood and the KL divergence between the approximate posterior and the prior, and then proceeds to optimize that.

Sep 7, 2020 · But it still does not solve my problem of understanding: mu_bob and sigma_bob define a gaussian just like mu_alice and sigma_alice. Usually, a N(0,1) prior on he latent space is used and encouraged by the regularization term of the ELBO objective.

Oct 25, 2019 · The ELBO $\mathcal{L}(\phi)$ can be written as the difference between the log evidence and the KL divergence between the variational distribution and true posterior: $$\mathcal{L}(\phi) = \log p(x) - D_{KL} \Big( q_\phi(\theta) \parallel p(\theta \mid x) \Big)$$ Take the gradient of both sides w.r.t. the variational parameters.

Aug 10, 2024 · Yet on the wikipedia page for the ELBO & on the highly cited paper for Bayes-by-backprop it shows the ELBO using the KL "in reverse" (i.e. true distribution is the second argument): What bothers me even more is that the cost function you actually optimize (below) seems like it would work (better?) even if you didn't use the reversed KL for the ...

Apr 9, 2022 · Tour Start here for a quick overview of the site