Description
1. [2 points] Adapted from Exercise 3.1 of FCMA p.133: For α, β = 1, the beta distribution becomes uniform between 0 and 1. In particular, if the probability of a coin landing heads is given by r and a beta prior is placed over r, with parameters α = 1, β = 1, this prior can be written as follows: p(r) = 1 (0 ≤ r ≤ 1) Using this prior, compute the posterior density for r if y heads are observed in N tosses (i.e., multiply this prior by the binomial likelihood and manipulate the result to obtain something that looks like a beta density). Solution. 2. [2 points] Adapted from Exercise 3.2 of FCMA p.134: Repeat the previous exercise for the following prior, also a particular form of the Beta density: p(r) = ( 2r 0 ≤ r ≤ 1 0 otherwise What are the values of the prior parameters α and β that result in p(r) = 2r? Solution. 3. [3 points] Adapted from Exercise 3.5 of FCMA p.134: If a random variable R has a beta density p(r) = Γ(α + β) Γ(α)Γ(β) r α−1 (1 − r) β−1 , derive an expression for the expected value of r, Ep(r){r}. You will need the following identity for the gamma function: Γ(n + 1) = nΓ(n). Hint: use the fact that Z r=1 r=0 r α−1 (1 − r) b−1 dr = Γ(a)Γ(b) Γ(a + b) . Solution. 4. [5 points; Required only for Graduates] Adapted from Exercise 3.12 of FCMA p.135: When performing a Bayesian analysis of the Olympics data, we assumed that σ 2 was known. If instead we assume that w is known and an inverse Gamma prior is placed on σ 2 p(σ 2 |α, β) = β α Γ(α) (σ 2 ) −α−1 exp − β σ 2 , the posterior over σ 2 will also be inverse Gamma. Derive the posterior parameters. Solution. 2 5. [5 points] Adapted from Exercise 4.2 of FCMA p.165-166: In Chapter 3, we computed the posterior density over r, the probability of a coin giving heads, using a beta prior and a binomial likelihood. Recalling that the beta prior, with parameters α and β, is given by p(r|α, β) = Γ(α + β) Γ(α)Γ(β) r α−1 (1 − r) β−1 and the binomial likelihood, assuming y heads in N throws, is given by p(y|r, N) = y N r y (1 − r) N−y compute the Laplace approximation to the posterior. (Note, you should be able to obtain a closed-form solution for the MAP value, ˆr, by setting the log posterior to zero, differentiating, equating to zero and solving for r.) Solution. 6. [4 points] Adapted from Exercise 4.3 of FCMA p.166: In the previous exercise you computed the Laplace approximation to the true beta posterior. In this problem, plot both the true beta posterior and the Laplace approximation for the three sets of values: 1. α = 5, β = 5, N = 20, and y = 10, 2. α = 3, β = 15, N = 10, and y = 3, 3. α = 1, β = 30, N = 10, and y = 3. Be sure to clearly indicate the values in your plot captions. Include how the two distributions (the true beta posterior and the Laplace approximation) compare in each case. Include the python script you use to generate these plots; the script should be named plot laplace approx.py. Suggestion: for plotting the beta and Gaussian (Normal) distributions, use scipy.stats.beta and scipy.stats.normal to create the beta and Gaussian random variables, and use the pdf(x) method for each to generate the curves. Note that for scipy.stats.normal, the mean is the location (loc) parameter, and the sigma is the scale parameter. Also, scipy.stats.normal expects the scale parameter to be the standard deviation (i.e., take the square root: math.sqrt(x)) of the variance you’ll compute for the Laplace approximation. Solution. 7. [4 points] Adapted from Exercise 4.4 of FCMA p.166: Given the expression for the area of a circle, A = πr2 , and using only uniformly distributed random variates, devise a sampling approach for computing π. Describe your method in detail and provide you script to do the estimation – this script should be called pi sample estimate.py. Solution.
