Final Exam Study Guide

Our final exam is Monday, April 27 from 9:00 AM to 12:00 PM. You are allowed both sides of two 8.5” x 11” hand-written sheets of notes, prepared by you and you alone.The exam will cover the following topics:

  1. Probability spaces and rules
  2. Counting
  3. Conditional probability and independence
  4. Discrete random variables
  5. Continuous random variables
  6. The moment-generating function
  7. Transformations of random variables
  8. Joint distributions
  9. Maximum likelihood estimation
  10. Bayesian inference

There will be approximately 1 problem per topic. The problems on topics 1-8 will be similar to the midterms. Since you’ve had more time to live with this material, I reserve the right to include problems that might stretch you a bit. For the MLE and Bayesian problems, I assure you there will be no surprises. You can expect problems that are exactly like the examples you’ve seen in lecture, lab, Problem Set 7, and this study guide. The distributions may be different, and the details might change, but the format and the steps are the same. If you work every practice problem from scratch on your own and then study the solutions carefully, you are ready.

TipWhat special distribution families should I know?

Binomial, geometric, Poisson, continuous uniform, normal, gamma (and its special cases like exponential), beta.

Anything else will be introduced from scratch within the exam as if you had never seen it before.

NOTE: the distribution helpers are the authoritative resource for pdf/pmf/cdf formulas. Other Internet sources may lead you astray because some of the distribution families have alternative parameterizations, e.g. \(\frac{1}{\beta}e^{-\frac{x}{\beta}}\) for Exp\((\beta)\) (which is NOT the way we define it).

NoteReminder

If your final exam score is better than your lowest midterm score, I will replace the low midterm with the final.

1. Probability spaces and rules

Problem 1

Suppose the events \(A\) and \(B\) are disjoint. Under what conditions are \(A^{c}\) and \(B^{c}\) also disjoint events?

Problem 2

Let \(A_1,\,A_2,\,...,\,A_n\subseteq S\) be a finite collection of possibly overlapping events in some probability space. Show that

\[ P\left(\bigcap_{i=1}^n A_i\right)\geq \sum\limits_{i=1}^nP(A_i)-n+1. \]

Want more review?

2. Counting

Problem 3

Four people are going to play bridge, which begins with an entire 52-card deck being dealt at random, 13 cards per player. What is the probability that each player is dealt exactly one ace?

TipStill confused about ordered versus unordered sampling?

There are several ways to do this calculation. You should try it on your own, but then, go read Example 1.8.9 and the subsequent discussion on pages 37 - 39 of DeGroot & Schervish.

Problem 4

An evil cult has its secret meetings in the basement of a building that has 15 above-ground floors. When the meeting is over, the nine cult members enjoy coffee and donuts and then depart to sow the seeds of villainy. They board the elevator, and each person randomly and independently selects one of the 15 floors to visit. Compute the probability that…

  1. No two passengers share a floor;
  2. At least two passengers share a floor;
  3. Exactly two passengers share a floor;
  4. No passengers get off at the top floor;
  5. At least one passenger gets off at floor 1;
  6. All passengers get off on consecutive floors;
  7. All passengers get off below floor 10

Want more review?

3. Conditional probability and independence

Problem 5

A JP Morgan recruiter is walking around campus with three bone-colored business cards. Two of the cards are normal; they have writing on one side and they are blank on the other. The third card was misprinted; the writing was duplicated on both sides. The recruiter drops one of the cards and orders his valet to pick it up. When the valet reaches for the card, he sees writing facing up. What is the probability that the recruiter dropped the misprinted card?

Problem 6

Suppose that \(k\) events \(B_1\), \(B_2\), …, \(B_k\) form a partition of the sample space \(S\), and \(A\subseteq S\) is some event with positive probability \(P(A)>0\). Show that if \(P(B_1\mid A)<P(B_1)\), then that guarantees \(P(B_i\mid A)>P(B_i)\) for at least one \(i\in\{1,\,2,\,...,\,k\}\).

Want more review?

4. Discrete random variables

Problem 7

Consider a sequence of \(n\) independent coin flips of a fair coin. Define a streak of heads as an uninterrupted run of heads, and efine a streak of tails analogously. We are interested in the number of streaks in a sequence of flips. For example, consider the sequence below:

\[ hhhttthhttthhhhhthtthhhh. \]

It contains 9 streaks. Here is the sequence again with the streaks separated:

\[ hhh\quad ttt\quad hh\quad ttt\quad hhhhh\quad t\quad h\quad tt\quad hhhh. \]

In a sequence of \(n\) flips of a fair coin, let \(Y\) be the number of streaks. What is the range of \(Y\), and what is its PMF?

Problem 8

Let \(X\sim\text{Exponential}(\lambda)\) for some arbitrary \(\lambda>0\). Define a new random variable \(Y=\lceil X\rceil\). Recall that the ceiling function \(\lceil\cdot\rceil\) is the function that rounds a number up to the next integer. So \(\lceil0.5\rceil=1\), \(\lceil13.1\rceil=14\), and so on.

  1. What is the range of \(Y\)?
  2. What is the PMF of \(Y\)?
  3. Does \(Y\) belong to a familiar distribution family?

Want more review?

5. Continuous random variables

Problem 9

Imagine \(X\) has this cdf:

\[ F(x)=\frac{1}{1+\exp\left(-\frac{x-\mu}{s}\right)},\quad x\in\mathbb{R}. \]

\(\mu\in\mathbb{R}\) and \(s>0\) are just constants.

  1. What is the pdf of \(X\)?
  2. What is the mean of \(X\)?

Problem 10

Here is the goofy pdf of some nonnegative random variable \(X\):

\[ f(x) = e^{1-e^x+x},\quad x\geq 0. \]

  1. Compute the cdf of \(X\);
  2. Compute the median of \(X\).

Want more review?

6. The moment-generating function

Problem 11

A certain random variable \(X\) has moment-generating function

\[ M(t)=\frac{e^{2t}}{1-3t},\quad t<1/3. \]

  1. Compute the mean and variance of \(X\).
  2. What is the moment generating function of the new random variable

\[ Y=\frac{\pi}{2}-\frac{X}{3}. \]

Problem 12

Consider the following:

\[ \begin{aligned} N&\sim\text{Poisson}(\lambda)\\ X_1,\,X_2,\,X_3,\,...&\overset{\text{iid}}{\sim}M\\ S&=\sum\limits_{i=1}^NX_i. \end{aligned} \]

\(N\) is independent of all of the \(X_i\). The \(X_i\) are an infinite sequence of iid random variables each sharing a common moment-generating function \(M(t)=E(e^{tX_1})\). \(S\) is a random sum of random variables. The terms are random, and so is \(N\), the number of terms being summed.

  1. What is the MGF of \(S\)?
  2. If I had an iid collection of \(S_1\), \(S_2\), …, \(S_m\) each possessing the same distribution that you just derived, what would be the distribution of \(T=S_1+S_2+...+S_m\)?

Want more review?

7. Transformations of random variables

Problem 13

\(X\) possesses the goofy distribution if its PDF is:

\[ f(x)=k\frac{x^{k-1}}{\theta}\exp\left(-\frac{x^k}{\theta}\right),\quad x>0. \]

The constants \(k\) and \(\theta\) are both positive parameters. We denote this \(X\sim\text{GF}(k,\,\theta)\)

  1. If we define a new random variable \(Y=\ln X\), what is its distribution?
  2. If we define a new random variable \(Z = X^k\), what is its distribution?

Problem 14

Let \(X\sim\text{Gamma}(\alpha,\,\beta)\), and define a new random variable \(Y=X^r\) for any positive constant \(r>0\).

  1. What is the density of \(Y\)?
  2. Derive a formula for \(E(X^n)\) that works for any \(n\in\mathbb{N}\).

Want more review?

  • Lab 7;
  • Problem Set 6: problem 6;
  • Midterm 2 Study Guide: problem 7;
  • Midterm 2: problem 3;
  • Read DeGroot & Schervish Chapter 3.8 and do the odd-numbered exercises.

8. Joint distributions

Problem 15

\(X\) and \(Y\) are jointly absolutely continuous with joint density

\[ f_{XY}(x,\, y) = \frac{1}{8} (y^2-x^2) e^{-y} ,\quad y>0 ;\, -y<x<y. \]

  1. Sketch \(\textrm{Range}(X,\, Y)\).
  2. Compute the marginal density of \(X\).
  3. Compute the marginal density of \(Y\).
  4. Compute the conditional density of \(X\) given \(Y = y\).
  5. Compute the conditional density of \(Y\) given \(X = x\).

Problem 16

Consider a random pair \((Q,\,Z)\) of continuous random variables whose joint distribution is given by this hierarchy:

\[ \begin{aligned} Q&\sim \text{Gamma}(a,\,b)\\ Z\mid Q=q&\sim\text{GF}(k, 1/q). \end{aligned} \]

So conditionally, \(Z\) has the distribution introduced in Problem 13 above, with \(1/q\) serving as the second parameter. \(k>0\) is just a constant throughout.

  1. What is the joint density of \((Q,\,Z)\)?
  2. What is the marginal distribution of \(Z\)?
  3. What is the conditional distribution of \(Q\) given \(Z=z\)?

Want more review?

  • jointly discrete examples from lecture;
  • jointly contnuous examples from lecture;
  • mixed Poisson-gamma example from Lab 9;
  • Problem Set 6: problems 7 - 10;
  • Midterm 2 Study Guide: problems 8 - 9;
  • Midterm 2: problem 4;
  • Odd-numbered exercises in DeGroot & Schervish Chapters 3.4 - 3.6.

9. Maximum likelihood estimation

Problem 17

Consider these data:

\[ X_1,\,X_2,\,...,\,X_n\overset{\text{iid}}{\sim}\text{N}(\theta,\,1). \]

  1. What is the maximum likelihood estimator of \(\theta\in\mathbb{R}\)?
  2. What is the sampling distribution of the estimator?
  3. What is the MSE of the estimator?
  4. Based on the MSE, what are the statistical properties of this estimator?

Problem 18

Let \(X_1\), \(X_2\), …, \(X_n\) be iid from some member of the parametric family that you met on Problem Set 6:

\[ f(x\,|\,\theta)=\theta(x+1)^{-(\theta+1)},\quad x>0. \]

  1. What is the maximum likelihood estimator of \(\theta>0\)?
  2. What is the sampling distribution of the estimator?
  3. What is the MSE of the estimator?

Problem 19

Consider these data:

\[ X_1,\,X_2,\,...,\,X_n\overset{\text{iid}}{\sim}\text{GF}(k,\,\theta). \]

So again, we are recycling the distribution family from Problem 13 above. Throughout, just treat \(k>0\) as fixed and known.

  1. What is the maximum likelihood estimator of \(\theta\in\mathbb{R}\)?
  2. What is the sampling distribution of the estimator?
  3. What is the MSE of the estimator?
  4. Based on the MSE, what are the statistical properties of this estimator?

Want more review?

10. Bayesian inference

Problem 20

Consider this Bayesian model:

\[ \begin{aligned} \theta&\sim\text{N}(m_0,\,\tau_0^2)\\ X_{1:n}\mid\theta&\overset{\text{iid}}{\sim}\text{N}(\theta,\,1). \end{aligned} \]

  1. What is the posterior distribution for \(\theta\) conditional on the data?
  2. Derive the posterior mean and show that it is a convex combination of the prior mean \(m_0\) and the MLE.
  3. The posterior mean can be thought of as an alternative estimator, in a classical sense. So, what are its mean, variance, bias, and MSE?
  4. Is it possible for this new estimator to be superior (lower MSE) than the pure MLE?

Problem 21

Consider this Bayesian model:

\[ \begin{aligned} \theta&\sim\text{IG}(a_0,\,b_0)\\ X_{1:n}\mid\theta&\overset{\text{iid}}{\sim}\text{N}(0,\,\theta). \end{aligned} \]

  1. What is the posterior distribution for \(\theta\) conditional on the data?
  2. Derive the posterior mean and show that it is a convex combination of the prior mean and the MLE;
  3. The posterior mean can be thought of as an alternative estimator, in a classical sense. So, what are its mean, variance, bias, and MSE?
  4. Is it possible for this new estimator to be superior (lower MSE) than the pure MLE?

Problem 22

Let’s consider the same distribution family as in Problem 18:

\[ f(x\mid \theta)=\theta(x+1)^{-(\theta+1)},\quad x>0. \]

We’ve done MLE for this, now let’s go Bayes. Consider this model:

\[ \begin{aligned} \theta&\sim\text{Gamma}(a_0,\,b_0)\\ X_{1:n}\mid\theta&\overset{\text{iid}}{\sim}f(x\mid \theta). \end{aligned} \]

  1. What is the posterior distribution for \(\theta\) conditional on the data?
  2. Derive the posterior mean and show that it is a convex combination of the prior mean and the MLE;

Want more review?