1. The document proposes methods for estimating the marginal log-likelihood of latent variable models in an unbiased manner. 2. It discusses using Monte Carlo methods like MCMC and importance sampling to estimate the intractable integral in the marginal log-likelihood. Multilevel Monte Carlo can provide an unbiased estimate with fewer samples than standard Monte Carlo. 3. Stochastically Unbiased Marginalization Objective (SUMO) is introduced to provide an unbiased estimate of the marginal log-likelihood using a single sample. This involves weighting the importance weighted bound with a geometric distribution.

1. Unbiased Gradient Estimation for Marginal Log-likelihood
Shohei Taniguchi, Matsuo Lab

2. എ‫ܠ‬
• ਂ૚ੜ੒Ϟσϧͷֶश͸ɼपลର਺໬౓ ͷ࠷େԽͰఆࣜԽ͞ΕΔ
જࡏม਺Ϟσϧʢe.g., VAE)ɿ
ΤωϧΪʔϕʔεϞσϧɿ
• ଟ͘ͷ৔߹Ͱɼपลର਺໬౓͸ղੳతʹ‫͍ͳ͖Ͱࢉܭ‬
log pθ (x)
log pθ (x) = log
∫
pθ (x, z) dz
log pθ (x) = − Eθ (x) − log
∫
exp (−Eθ (x)) dx
2

3. എ‫ܠ‬
ྫ̍ɿVAE
ର਺पล໬౓ͷม෼ԼքͰۙࣅ
log pθ (x) = log
∫
pθ (x, z) dz
≥
𝔼
q(z) [
log
pθ (x, z)
q (z) ]
= ℒ (θ, q)
3

4. എ‫ܠ‬
ྫ̎ɿEBM
ର਺पล໬౓ͷޯ഑ΛMCMCͰۙࣅ
͸MCMCΛ εςοϓճͨ͠෼෍Ͱɼ Ͱ౳߸੒ཱ
∇log pθ (x+) = − ∇Eθ (x+) +
𝔼
x−∼pθ(x) [∇Eθ (x−)]
≈ − ∇Eθ (x+) +
𝔼
xT∼q(xT) [∇Eθ (xT)]
q (xT) T T → ∞
4

5. എ‫ܠ‬
• ม෼Լք΍༗‫ݶ‬εςοϓͷMCMCʹΑΔޯ഑ͷۙࣅʹ͸ɼόΠΞε͕͋Δ
όΠΞεɿਪఆྔͷ‫ظ‬଴஋ͱਅ஋ͱͷͣΕ
ɹɹɹɹɹόΠΞε͕0ͷਪఆྔΛෆภਪఆྔͱ͍͏
• Ͱ͖Ε͹ɼର਺पล໬౓ͷޯ഑ͷෆภਪఆྔΛ࢖ֶͬͯश͍ͨ͠
• ର਺पล໬౓Λෆภਪఆ͢Δख๏ͷྫʢա‫ڈ‬ͷྠಡʣ
https://www.slideshare.net/DeepLearningJP2016/dlsumo-unbiased-estimation-
of-log-marginal-probability-for-latent-variable-models-250013351
5

6. Outline
पลର਺໬౓ͷෆภਪఆ๏
1. पลର਺໬౓ࣗମΛਪఆ͢Δํ๏
• On Multilevel Monte Carlo Unbiased Gradient Estimation for Deep Latent
Variable Models (AISTATS 2021)
• Efficient Debiased Evidence Estimation by Multilevel Monte Carlo Sampling (UAI
2021)
2. पลର਺໬౓ͷޯ഑Λਪఆ͢Δํ๏
• Unbiased Contrastive Divergence Algorithm for Training Energy-Based Latent
Variable Models (ICLR 2020)
6

7. Outline
पลର਺໬౓ͷෆภਪఆ๏
1. पลର਺໬౓ࣗମΛਪఆ͢Δํ๏
• On Multilevel Monte Carlo Unbiased Gradient Estimation for Deep Latent
Variable Models (AISTATS 2021)
• Efficient Debiased Evidence Estimation by Multilevel Monte Carlo Sampling (UAI
2021)
2. पลର਺໬౓ͷޯ഑Λਪఆ͢Δํ๏
• Unbiased Contrastive Divergence Algorithm for Training Energy-Based Latent
Variable Models (ICLR 2020)
7

8. IWAE
Importance Weighted Autoencoder
ͰVAEͱҰகɼ Ͱ౳߸੒ཱ
log pθ (x) = log
𝔼
z(1),…,z(k)∼q(z)
[
1
k
k
∑
i=1
pθ (x, z(i)
)
q (z(i)
) ]
≥
𝔼
z(1),…,z(k)∼q(z)
[
log
1
k
k
∑
i=1
pθ (x, z(i)
)
q (z(i)
) ]
=
𝔼
z(1),…,z(k)∼q(z) [ℒk (θ, q)]
k = 1 k → ∞

9. पลର਺໬౓ͷ‫਺ڃ‬ද‫ه‬
ͱ͓͘ͱɺ‫਺ڃ‬ ͷ‫ظ‬଴஋͸ର਺पล໬౓ͱҰக͢Δ
Δk =
{
ℒ1 (θ, q) (k = 1)
ℒk (θ, q) − ℒk−1 (θ, q) (k ≥ 2)
∞
∑
k=1
Δk
𝔼
[
∞
∑
k=1
Δk
]
=
𝔼
[ℒ∞ (θ, q)] = log pθ (x)

10. Russian Roulette Estimator
ҎԼͷΑ͏ͳ Λߟ͑Δ
1. ֬཰ Ͱද͕ग़ΔίΠϯΛৼΔ
2. ද͕ग़ͨΒ Ҏ߱Λ‫͠ࢉܭ‬ɺ Ͱׂͬͨ΋ͷΛ ʹ଍͢
ཪ͕ग़ͨΒ ͚ͩΛ‫͢ࢉܭ‬Δ
̂
y
̂
y = Δ1 +
∑
∞
k=2
Δk
μ
⋅ b, b ∼ Bernoulli (μ)
μ
k = 2 μ Δ1
Δ1

11. Russian Roulette Estimator
͸ ͷෆภਪఆྔͰ͋Δ͜ͱ͕Θ͔Δ
̂
y
∞
∑
k=1
Δk
̂
y = Δ1 +
∑
∞
k=2
Δk
μ
⋅ b, b ∼ Bernoulli (b; μ)
𝔼
[ ̂
y] = Δ1 +
∑
∞
k=2
Δk
μ
⋅
𝔼
[b] =
∞
∑
k=1
Δk

12. Russian Roulette Estimator
ಉ͜͡ͱΛ Ҏ߱΋‫܁‬Γฦ͢ͱɺҎԼͷ ΋ ͷෆภਪఆྔʹͳΔ
͸࠷ॳʹཪ͕ग़Δ·ͰʹίΠϯΛৼͬͨճ਺ʢ‫ز‬Կ෼෍ʹै͏ʣ
͜ͷ Λ࢖͑͹ɺର਺पล໬౓ͷෆภਪఆྔ͕ಘΒΕΔ
k = 2 ̂
y
∞
∑
k=1
Δk
̂
y =
K
∑
k=1
Δk
μk−1
, K ∼ Geometric (K; 1 − μ)
K
̂
y

13. Single Sample Estimator
ಉ༷ʹͯ͠ɼҎԼͷ ΋ ͷෆภਪఆྔʹͳΔ
̂
y
∞
∑
k=1
Δk
̂
y =
ΔK
p (K)
=
ΔK
μK−1
(1 − μ)
𝔼
[ ̂
y] =
∞
∑
K=1
p (K) ⋅
ΔK
p (K)
=
∞
∑
k=1
Δk

14. SUMO
Stochastically Unbiased Marginalization Objective
log pθ (x) =
𝔼
K∼p(K)
[
K
∑
k=1
Δk
μk−1 ]
= ℒ1 (θ, qϕ) +
𝔼
K∼p(K)
K
∑
k=2
ℒk (θ, qϕ) − ℒk−1 (θ, qϕ)
μk−1
VAEͱಉ͡
ิਖ਼߲

15. SUMOͷ՝୊
• ਪఆྔͷ෼ࢄ͕େ͖͘ͳΓ΍͍͢
• ࠷ѱͷ৔߹ɼ෼ࢄ͕ແ‫ݶ‬େʹൃࢄ͢Δ
• ෼ࢄ͸ ͷબͼํͰ੍‫͖Ͱޚ‬Δ͕ɼ෼ࢄΛԼ͛Α͏ͱ͢Δͱɼ
͕େ͖͘ͳΓɼ‫͕ྔࢉܭ‬େ͖͘ͳΔ
• ෼ࢄ͕༗‫ྔࢉܭ͔ͭݶ‬ͷ‫ظ‬଴஋΋༗‫͋Ͱݶ‬Δ͜ͱ͕ཧ૝
p (K)
K
15

16. पลର਺໬౓ͷ‫਺ڃ‬ද‫ه‬ʢ࠶ʣ
ͱ͓͘ͱɺ‫਺ڃ‬ ͷ‫ظ‬଴஋͸ର਺पล໬౓ͱҰக͢Δ
Δk =
{
ℒ1 (θ, q) (k = 1)
ℒk (θ, q) − ℒk−1 (θ, q) (k ≥ 2)
∞
∑
k=1
Δk
𝔼
[
∞
∑
k=1
Δk
]
=
𝔼
[ℒ∞ (θ, q)] = log pθ (x)
ͷઃ‫ํܭ‬๏͸ଞʹ΋ߟ͑ΒΕΔ
Δk

17. पลର਺໬౓ͷ‫਺ڃ‬ද‫ه‬ʢվʣ
͜ͷ৔߹΋ɼ‫਺ڃ‬ ͷ‫ظ‬଴஋͸ର਺पล໬౓ͱҰக͢Δ
͜ΕΛ࢖ͬͯߏ੒ͨ͠ਪఆྔ͸ɼ෼ࢄ༗‫ྔࢉܭ͔ͭݶ‬༗‫ݶ‬Λ࣮‫͖Ͱݱ‬Δ
Δk =
ℒ1 (θ, q) (k = 1)
ℒ2k (θ, q)−
1
2 (ℒ(1)
2k−1 (θ, q) + ℒ(2)
2k−1 (θ, q)) (k ≥ 2)
∞
∑
k=1
Δk

18. ࣮‫ݧ‬
ը૾ੜ੒
ఏҊ๏͸IWAE΍SUMOΑΓ΋ੑೳ͕վળ͢Δ

19. Outline
पลର਺໬౓ͷෆภਪఆ๏
1. पลର਺໬౓ࣗମΛਪఆ͢Δํ๏
• On Multilevel Monte Carlo Unbiased Gradient Estimation for Deep Latent
Variable Models (AISTATS 2021)
• Efficient Debiased Evidence Estimation by Multilevel Monte Carlo Sampling (UAI
2021)
2. पลର਺໬౓ͷޯ഑Λਪఆ͢Δํ๏
• Unbiased Contrastive Divergence Algorithm for Training Energy-Based Latent
Variable Models (ICLR 2020)
19

20. EBMͷֶश
ର਺पล໬౓ͷޯ഑ΛMCMCͰۙࣅ
͸MCMCΛ εςοϓճͨ͠෼෍Ͱɼ Ͱ౳߸੒ཱ
͕༗‫ͱͩݶ‬ɼޯ഑ͷਪఆʹόΠΞε͕ͷΔ
∇log pθ (x+) = − ∇Eθ (x+) +
𝔼
x−∼pθ(x) [∇Eθ (x−)]
≈ − ∇Eθ (x+) +
𝔼
xT∼q(xT) [∇Eθ (xT)]
q (xT) T T → ∞
T
20

21. ޯ഑ͷ‫਺ڃ‬ల։
Ͱ͋Δ͜ͱΛ༻͍Δͱɼ‫਺ڃ‬ͷ‫ʹܗ‬ม‫͖Ͱܗ‬Δ
𝔼
x−∼pθ(x) [∇Eθ (x−)]
=
𝔼
x∞∼q(x∞) [∇Eθ (x∞)]
=
𝔼
xk∼q(xk) [∇Eθ (xk)] +
∞
∑
t=k+1
𝔼
xt+1∼q(xt+1) [∇Eθ (xt+1)] −
𝔼
xt∼q(xt) [∇Eθ (xt)]
pθ (x) = q (x∞)
21

22. ޯ഑ͷ‫਺ڃ‬ల։
ͨͩ͠ɼ ͸ ͱಉ͡पล෼෍ʹै͏
𝔼
x−∼pθ(x) [∇Eθ (x−)]
=
𝔼
xk∼q(xk) [∇Eθ (xk)] +
∞
∑
t=k+1
𝔼
xt+1∼q(xt+1) [∇Eθ (xt+1)] −
𝔼
xt∼q(xt) [∇Eθ (xt)]
=
𝔼
xk∼q(xk) [∇Eθ (xk)] +
∞
∑
t=k+1
𝔼
xt+1∼q(xt+1) [∇Eθ (xt+1)]−
𝔼
yt∼q(yt) [∇Eθ (yt)]
yt xt
22

23. ޯ഑ͷ‫਺ڃ‬ల։
ͨͩ͠ɼ ͸ ͱಉ͡पล෼෍ʹै͏
𝔼
x−∼pθ(x) [∇Eθ (x−)]
=
𝔼
xk∼q(xk) [∇Eθ (xk)] +
∞
∑
t=k+1
𝔼
xt+1∼q(xt+1) [∇Eθ (xt+1)]−
𝔼
yt∼q(yt) [∇Eθ (yt)]
=
𝔼
[
∇Eθ (xk) +
∞
∑
t=k+1
(∇Eθ (xt+1) − ∇Eθ (yt))]
yt xt
23

24. ޯ഑ͷ‫਺ڃ‬ల։
ͨͩ͠ɼ ͸ ͱಉ͡पล෼෍ʹै͍ɼ Ͱ Ͱ͋Δͱ͢Δ
𝔼
x−∼pθ(x) [∇Eθ (x−)]
=
𝔼
[
∇Eθ (xk) +
∞
∑
t=k+1
(∇Eθ (xt+1) − ∇Eθ (yt))
]
=
𝔼
[
∇Eθ (xk) +
τ−1
∑
t=k+1
(∇Eθ (xt+1) − ∇Eθ (yt))
]
yt xt t ≥ τ xt+1 = yt
24

25. ޯ഑ͷ‫਺ڃ‬ల։
ແ‫਺ڃݶ‬Λ༗‫ݶ‬࿨ʹॻ͖‫͑׵‬ΒΕͨʂ
͔͠͠ɼ Ͱ Λຬͨ͢ ͸ͲͷΑ͏ʹ࡞Δʁ
𝔼
x−∼pθ(x) [∇Eθ (x−)]
=
𝔼
[
∇Eθ (xk) +
τ−1
∑
t=k+1
(∇Eθ (xt+1) − ∇Eθ (yt))]
t ≥ τ xt+1 = yt yt
25

26. ΧοϓϦϯά
• ૬ؔͷ͋Δ2ͭͷMCMCΛճͯ͠
1εςοϓͣΕͨαϯϓϧಉ͕࢜
Ұக͢Δ·Ͱճ͢
• MCMCΛಠཱʹճͯ͠͠·͏ͱ
αϯϓϧಉ͕࢜Ұக͢Δ͜ͱ͕΄ͱΜͲ‫͜ى‬Βͳ͍ͷͰ
͏·͘૬ؔͤ͞Δ͜ͱ͕ٕज़తͳ؊
26

27. ΧοϓϦϯά
• ͱ ΛͦΕͧΕ ʹै͏֬཰ม਺ͱ͢Δ
• ͨͩ͠ɼ྆ऀ͸ಠཱͰ͋Δඞཁ͸ͳ͍
• ͜ͷͱ͖ɼ ͱ ͕Ұக͢Δ֬཰ ʹ͍ͭͯɼҎԼͷෆ౳͕ࣜ੒ཱ
ξ η p (ξ), q (η)
ξ η P (ξ = η)
P(ξ = η) ≤ 1 − ∥p − q∥TV =
∫
min{p(x), q(x)}dx
27
∫
min{p(x), q(x)}dx

28. Maximal Coupling
• ҎԼͷखॱͰ ͱ ΛαϯϓϦϯά͢Δͱɼ͜ͷ্քΛୡ੒Ͱ͖Δ
P(ξ = η) ≤ 1 − ∥p − q∥TV =
∫
min{p(x), q(x)}dx
ξ η
28
https://commons.wikimedia.org/wiki/File:Total_variation_distance.svg

29. Maximal Coupling
खॱ
1. ෼෍ ͔Β ΛαϯϓϦϯά
2. ֬཰ Ͱ ͱ͢Δ
• ਤͷ੺͍෦෼͔Β Λαϯϓϧ
3. ֬཰ Ͱ ͔Βαϯϓϧ͢Δ
• ͷ࢒Γͷ෦෼͔Β Λαϯϓϧ
p ξ
α = min (q (ξ)/p (ξ), 1) η = ξ
η
1 − α q̃ (η) ∝ min (q (η) − p (η), 0)
q η
29
https://commons.wikimedia.org/wiki/File:Total_variation_distance.svg

30. Maximal Coupling
ྫɿͲͪΒ΋ඪ४ਖ਼‫ن‬෼෍ͷ৔߹
1. ͔Βαϯϓϧ ΛಘΔ
2. ͱͯ͠ɼ ͔Βͷ
αϯϓϧͱ͢Δ
• ͜ͷͱ͖ɼ ͱ ͸ͱ΋ʹඪ४ਖ਼‫ن‬෼෍ʹै͍ɼ֬཰1Ͱ
p ξ
η = ξ q
ξ η ξ = η
30
https://colcarroll.github.io/couplings/static/maximal_couplings.html

31. Maximal Coupling
ͦͷଞͷྫ
31
https://colcarroll.github.io/couplings/static/maximal_couplings.html

32. ΧοϓϦϯά
• MCMCͷ֤εςοϓΛmaximal couplingͰαϯϓϦϯά͢Δ͜ͱͰ
ߴ͍֬཰Ͱ2ͭͷαϯϓϧΛҰகͤ͞Δ͜ͱ͕Ͱ͖Δ
• ΧοϓϦϯάΛ࢖ͬͨMCMCͷෆภԽͷॳग़͸[Jacob, et al., 2017]
https://arxiv.org/abs/1708.03625
• ͜ͷ࿦จ͸ɼ͜ΕΛEBMʢಛʹRBMʣͷ
ֶशʹ༻͍ͨ΋ͷ
32

33. ࣮‫ݧ‬
τΠσʔλ
33
ී௨ͷMCMCͰ͸్த͔Βੑೳ͕ྼԽ͢Δ͕ఏҊ๏͸ͦΕ͕‫͜ى‬Βͳ͍
ΧοϓϦϯάʹ͔͔Δεςοϓ਺͸ଟͯ͘10εςοϓఔ౓

34. ࣮‫ݧ‬
ը૾ੜ੒ʢFashion MNISTʣ
34
ը૾ੜ੒Ͱ΋ಉ༷ͷ܏޲

35. ·ͱΊ
• पลର਺໬౓͓Αͼͦͷޯ഑Λෆภਪఆ͢ΔͨΊͷςΫχοΫΛ঺հ
1. ϩγΞϯϧʔϨοτਪఆΛ༗‫ݶ‬෼ࢄɾ༗‫ํ͏ߦͰྔࢉܭݶ‬๏
2. ΧοϓϦϯάΛ࢖ͬͯMCMCΛෆภʹ͢Δํ๏
• ͲͪΒ΋ςΫχΧϧʹ໘ന͘ɼ৭ʑԠ༻͕ޮ͖ͦ͏
• ࣮‫͕ݧ‬খ‫ن‬໛ͳͷͰɼେ͖͍ϞσϧͰ΋࢖͑Δͷ͔͕‫ͳʹؾ‬Δ
35