HTSEFP: Midterm

Part one of the "How to solve every problem" series for MATH 324. Covers chapter 1, 2 and 3 of the syllabus, which is what will be tested on the midterm.

1F-distributions¶

Given two random samples from a chi-square distribution - $S_1^2$, with $n_1$ values and a variance of $\sigma_1^2$, and $S_2^2$, with $n_2$ values and a variance of $\sigma_2^2$ - find:

a) a number $b$ such that $\displaystyle P\left ( \frac{S_1^2}{S_2^2} \leq b \right ) = x$ (where $x$ is some number between 0 and 1);
b) a number $a$ such that $\displaystyle P \left ( a \leq \frac{S_1^2}{S_2^2} \right ) = y$ (where $y$ is again some number between 0 and 1);
c) $\displaystyle P \left ( a \leq \frac{S_1^2}{S_2^2} \leq b \right )$.

1.1General solution¶

This is explained in the textbook, starting from definition 7.3. First, we notice that

$$\frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2}$$

is an F-distribution with $n_1 - 1$ numerator degrees of freedom and $n_2-1$ denominator degrees of freedom. Now, we substitute in the variance for each random sample (it may be that the ratio and not the values themselves is given) to obtain something in the form

$$\alpha \frac{S_1^2}{S_2^2}$$

where $\alpha$ is some constant specific to the question.

For part (a), we can then multiply both sides of the $\leq$ sign within the probability function by $\alpha$, resulting in

$$P\left ( \alpha \frac{S_1^2}{S_2^2} \leq \alpha b \right ) = x$$

Now, we simply need to use the provided table for F distributions (table 7 appendix 3 of the textbook) to identify the value of $\alpha b$ cutting off an upper-tail area of $1 - x$, and then divide that by $\alpha$ to obtain the true value of $b$.

For part (b), we first have to use the identity

$$P(U_1/U_2 \leq k) = P(U_2/U_1 \geq \frac{1}{k}$$

and after substituting and multiplying both sides by $\alpha$ we have

$$P\left ( \alpha \frac{S_2^2}{S_1^2} \geq \alpha \frac{1}{a} \right ) = P \left ( \alpha \frac{1}{a} \leq \alpha \frac{S_2^2}{S_1^2} \right )$$

Using the table mentioned above, we identify the value of $\alpha/a$ cutting off an upper-tail area of $1-y$, and divide $\alpha$ by that value to obtain the true value of $a$.

For part (c), the answer is $1 - (1-x) - (1-y)$. Should be obvious.

1.2Examples¶

• Assignment 1, question 1

2Finding the MLE¶

Given a probability density function, find the maximum likelihood estimate for the parameter $\theta$. Optionally:

a) Find the expected value and the variance for the MLE of $\theta$, $\hat \theta$;
b) Given a value for $\theta$, give the approximate bounds for the error of estimation;
b) Find the MLE for the variance.

2.1General solution¶

Find the likelihood function (given by the joint density of the random sample), then find the maximum of the logarithm of it (by finding the point at which the derivative is equal to 0).

a) The expected value and variance for $\hat \theta$ can be found using the MLE obtained above and the formulas for the expected value and variance of the given probability distribution, respectively. Use the rules of arithmetic manipulation of the expected value and variance functions if necessary.

b) The approximate bounds are given by $2\sqrt{V(\hat \theta)}$. Simply substitute the given value of $\theta$ in the formula for $V(\hat \theta)$.

c) Substitute the given value for the MLE in the formula for $V(Y)$.

2.2Examples¶

• Assignment 1, questions 2 and 3

3Method-of-moments estimators¶

Given random sample(s) from a distribution, find the method-of-moments estimators for something (mean, variance, some random parameter, etc).

3.1General solution¶

The first two moments for the distribution are $E(Y)$ and $E(Y^2)$, respectively; for instance, for the normal distribution, these come out to $E(Y) = \mu$ and $E(Y^2) = \sigma^2 + \mu^2$. The corresponding sample moments are $m_1' = \displaystyle \frac{1}{n} \sum_{i=1}^n Y_i = \overline Y$ and $\displaystyle m_2' = \frac{1}{n} Y_i^2$. To find the method-of-moments estimator for $\mu$ or $\sigma^2$, we simply equate $E(Y)$ and $m_1'$ and $E(Y^2)$ and $m_2'$ to solve for $\hat \mu$ and $\hat \sigma^2$, respectively.

If finding the estimator for a parameter $\alpha$, then first find the expected value by integrating $yf(y)$ (which should be a function of $\alpha$) and set that equal to the first sample moment, $\overline Y$, then solve for $\alpha$.

3.2Examples¶

• Assignment 1, questions 4 and 5

4Deriving the posterior distribution¶

Derive the posterior distribution for something

4.1General solution¶

wat

4.2Examples¶

• Assignment 1, question 6

5Unbiased estimators¶

Show that the estimator $\hat \theta$ is an unbiased estimator for $\theta$.

5.1General solution¶

To show that $\hat \theta$ is an unbiased estimator of $\theta$, we simply have to show that $E(\hat \theta) = \theta$. Assuming that $\hat \theta$ is a function of other estimators whose expected values are given, we can use the rules of arithmetic with the expected value function to show that $E(\hat \theta)$ simplifies to $\theta$.

5.2Examples¶

• Assignment 1, question 7 (a)

6Estimators and minimising variance¶

Given that the estimator $\hat \theta$ is a function of several other estimators whose variance is given, which are independent, and a constant $a$, how should the constant $a$ be chosen in order to minimise variance?

6.1General solution¶

First, we can determine the variance of the estimator $\hat \theta$ using the rules of arithmetic with the variance function. We should obtain the variance of $\hat \theta$ in terms of the variance for the other estimators and the constant $a$. Then, we find the minimum of this function, which can be done by finding the point at which the derivative of the function (with respect to $a$) is equal to 0.

6.2Examples¶

• Assignment 1, question 7 (b)

7Identifying estimator bias¶

Given several random samples from some distribution $f(y)$ and a number of estimators of $\theta$ based on those random samples, classify each estimator as biased or unbiased. Find the variance for each of the unbiased estimators.

7.1General solution¶

If the expected value of the estimator is equal to $\theta$, then it's unbiased; else, it is biased. We can find the expected value of any estimator by integrating $yf(y)$ (if the estimator involves only one of the aforementioned random samples) or by substituting the expected values of each of the estimators it's a function of (if the estimator involves several random samples). Note that if the estimator involves the min or max or two or more of the random samples, then it's going to be biased.

To find the variance of each random sample, use the identity $V(Y) = E(Y^2) - (E(Y))^2$ and calculate $E(Y^2)$ by integrate $y^2f(y)$. Then, determine the variance of each estimator using the rules of arithmetic with the variance function.

7.2Examples¶

• Assignment 1, question 8

8Computing efficiency¶

Given the formulas for two unbiased estimators, find the efficiency of one estimator to the other.

8.1General solution¶

The efficiency of $\hat \theta_1$ to $\hat \theta_2$ is given by:

$$\text{eff}(\hat \theta_1, \hat \theta_2) = \frac{V(\hat \theta_2)}{V(\hat \theta_1)}$$

So we just need to find the variance for each estimator. If min/max functions are involved, we can use order statistics and transform coordinates in order to get a more convenient integrand.

8.2Examples¶

• Assignment 2, question 1

9Consistent estimators¶

Show that something is a consistent estimator for some parameter.

9.1General solution¶

Take the limit as some variable in the estimator function goes to infinity. This should approach zero.

9.2Examples¶

• Assignment 2, question 2

10Determining sufficiency¶

Determine whether or not some estimator is sufficient for some parameter in some random sample from some probability distribution.

10.1General solution¶

Find the likelihood function (product of the joint densities). Split it as the product of two functions - one ($g(\overline y, p)$) that is a function of $\overline y$ and $p$, the other ($h(y_1, \ldots y_n)$) that is a function of the random sample stuff etc. If both are non-negative, then by theorem 9.4 in the textbook, we have that the estimator is sufficient for the relevant parameter.

10.2Examples¶

• Assignment 2, questions 3 and 4

11Finding an MVUE¶

Find an MVUE of some parameter $\theta$ using some function. Optionally: find the variance of the MVUE.

11.1General solution¶

Find the expected value of the given function (possibly by integrating, as is standard), which should be a function of $\theta$ and some other variables. Isolate $\theta$; the MVUE is the function whose expected value is equal to $\theta$.

If you need to show that something is an MVUE, recall that it must be a function of the sufficient statistic and must be unbiased.

Find the variance using the identities for the variance and expected value functions and then differentiating the moment-generating function 4 times (result of using the identities).

11.2Examples¶

• Assignment 2, questions 5 and 6

12Errors of estimation¶

Given $n$ random samples and values for the observed mean and standard deviation (or just raw values for calculating them), estimate the mean for the population as a whole and a place a 2-standard-error bound on the error of estimation.

12.1General solution¶

The best population mean estimate is the observed mean from the random samples. The 2-standard-error bound is given by $2\sigma_p$. This indicates that that the probability that the error of estimation is less than $2\sigma_p$ is approximately 0.95.

If the standard deviation is not given, note that $\sigma_p = \sqrt{p(1-p)/n}$ where $p$ is the mean or whatever.

See also example 8.2 from chapter 8 of the textbook.

12.2Examples¶

• Assignment 2, question 7

13Pivotal quantities¶

Show that some quantity is a pivotal quantity for some distribution. Use this to a find a particular lower/upper confidence limit for some parameter.

13.1General solution¶

Transform coordinates - set $U$ to the proposed pivotal quantity and rewrite the density function in terms of $u$. The result should be independent of the original variable, proving that the proposed quantity is a pivotal quantity.

Now, to find the lower confidence limit for some parameter, we need to find the value of $\alpha$ such that the probability of U being less than or greater than $\alpha$ is whatever the desired probability is. To solve for $\alpha$, we simply rewrite the probability function as an integration of the density function (in terms of $u$) with the bounds being 0 to $\alpha$ or whatever. Then, we rewrite the equation based on the relationship between $U$ and $Y$ to get the bound for $\theta$.

13.2Examples¶

• Assignment 2, question 8