University of Illinois  Econ 476  Department of Economics
Spring 2001  Final Exam Review  Roger Koenker

This handout includes a sampling of recent questions from 476 final exams. It should provide a reasonable sample to gauge the nature of the questions and to review the material covered in the course. It could be expected that two exam questions will come from these review questions.

*****

University of Illinois  Econ 476  Department of Economics
Spring 1998  FINAL EXAMINATION  Roger Koenker

1. Let Z be a U[0, 1] random variable and define (hint: draw pictures!)

X = Z − \frac{1}{2} Y = |2Z − 1| − \frac{1}{2} U = sgn (X) V = sgn (Y)

(a) Find the covariance matrix of the random vector (X, Y, U, V).

(b) Which pairs of these variables, if any, are stochastically independent?

2. An important recent paper on semiparametric estimation of panel data models begins with the observations that if two random variables X and Y are iid with common density f(·), then Z = Z − Y has a symmetric distribution centered at zero. Explain.

Spring 1997  Econ 476 Final Exam  Roger Koenker

1. Consider the problem of estimating the location parameter, θ, of the (standard) Cauchy

distribution with density

f(x) = π^{−1}(1 + (x − θ)^{2})^{−1}

(a) Using the “method of scoring” and the fact that

−π^{−1} ∫ \frac{2x^{2} − 2}{(1 + x^{2})^{3}} dx = \frac{1}{2}

find the maximum likelihood estimator for θ from the sample observations

{−13.71, 0.40, −2.55, 0.44, −0.57, −1.05, −0.87, −1.59, −4.11, −0.72}

Try to be very explicit about what you do here.

(b) Interpret the mysterious integral in part (a) in statistical terms.

(c) Compare the performance of the sample mean and sample median with the mle in the Cauchy case.

(d) Is the mle asymptotically normal in this problem? Briefly sketch why or why not.

(e) Suppose that you wanted to expand the approach in part (a.) so that instead of a scalar location parameter you had a “regression effect”, i.e., θ = x′_{i}β for the ith observation. Describe briefly how you might adapt your approach to computing the mle to this case.

2. Suppose X_{1}, …, X_{10} constitute a random sample from the density

f(x) = \frac{θ^{3}}{2} e^{−θx}x^{2} x > 0

Knowing only that X̄ = 1.9, find an estimate of θ, and test the hypothesis that θ = 1. If possible, try to suggest more than one test of the hypothesis, compare their conclusions, and discuss their relative merits from a theoretical standpoint.

3. In Lecture 18, I tried to connect spline smoothing with model selection via the notion of the “effective dimensionality” of the fitted function. In particular, it was suggested that choosing the smoothing parameter μ = λ^{−1} by minimizing

ν(μ) = n log(σ̂(μ)) + \frac{1}{2} log nk(μ)

where k(μ) = T r(A(μ)) might be reasonable. Suppose several years from now someone asks you about this. Try to explain the rationale for this expression briefly. Try to relate it to the application of model selection in regression. Suppose someone suggests that instead of using k(μ) = T r(A(μ)) you might want to use k(u) = n^{−1}T r(2A(μ) − A^{2}(μ)), can you suggest a rationale for this?

*****

University of Illinois  Official “Answer” to Q.3  Department of Economics
Spring 1997  Econ 476 Final Exam 1997  Roger Koenker

This question caused some difficulty in interpretation so I thought it might be useful to provide an “official answer.” As you will see there is some rather radical hand waving going on in this answer which would be desirable to make more rigorous.

Recall the GCV expression of Craven and Wahba

V (λ) = \frac{n^{−1} ∥ (I − A)y ∥^{2}}{(n^{−1}T r(I − A))^{2}}

The objective is to connect this expression to the proposed SIC expression. Note first that

σ̂^{2} = n^{−1} ∥ (I − A)y ∥^{2}

is the usual mle estimate of σ^{2} presuming that λ in A(λ) is fixed. Thus, taking logs we have

ν(λ) = log V (λ) = log(σ̂^{2}) − 2 log(n^{−1}T r(I − A))
= log(σ̂^{2}) − 2 log(1 − n^{−1}T r(A))

Now using the familiar expansion log(1 + a) ≅ a for small a we have, provided that T r(A) is small relative to n,

ν(λ) ≅ log(σ̂^{2}) + 2n^{−1}T r(A)

so minimizing ν(λ) is equivalent to maximizing

η(λ) = −\frac{n}{2} log(σ̂^{2}) − k(λ)

where the first term is recognizable as the usual log-likelihood evaluated at the mle, for given λ, and k(λ) = T r(A) is, or purports to be, a measure of the dimensionality of the fit. This is, in effect an interpretation of Craven and Wahba’s GCV criterion as AIC. The expression in the exam is simply derived from the above by rewriting the likelihood term as −n log(σ̂) and replacing k(λ) by 1/2 log nk(λ) as in Schwarz.

There is an extensive discussion about alternative definitions of k(λ) in Hastie and Tibshirani (1990). One way to motivate the suggested alternative to k(λ) = T r(A(λ)) is to consider the usual argument

E ∥ û ∥^{2} = E ∥ (I − A)u ∥^{2}
= σ^{2}T r(I − A)^{2}
= σ^{2}(n − T r(2A − A^{2}))

So in this approach T r(2A − A^{2}) is the dimension of the fitted model. Of course, if A were really idempotent, then 2A − A^{2} = A, but this is not (quite) the case for typical smoothers including the smoothing spline.

*****

University of Illinois Department of Economics

Spring 1995  Econ 476 Exam Review  Roger Koenker

1. Suppose X_{1}, …, X_{10} constitute a random sample from the density

f(x) = \frac{θ^{3}}{2} e^{−θx}x^{2} x > 0.

Knowing only that X̄ = 2, find an estimate of θ, and test the hypothesis that θ = 1. Briefly, try to explain any special virtues of the procedures you use.

2. In the simple regression through the origin model,

y_{i} = x_{i}β + u_{i}

with x_{i} and β scalar. Explain the significance of the condition

max_{i} |x_{i}|^{2}/ ∑ x_{i}^{2} → 0

for the asymptotic behavior of β̂ = (x′y)/x′x.

3. Consider the kernel density estimator

f̂_{h}(x) = n^{−1} ∑ K_{h}(x − X_{i}) = (nh)^{−1} ∑ K_{0} (\frac{x − X_{i}}{h} )

Explain briefly the following argument to compute the asymptotic bias of f̂_{n}(x):

Ef̂_{h}(x) = n^{−1} ∑ EK_{h}(x − X_{i})
= ∫ K_{h}(x − u)f(u)du
= ∫ K_{0}(s)f(x + sh)ds

Thus for h → 0 we have Ef̂_{h}(x) → f(x) and

Bias f̂_{h}(x) = ∫ K_{0}(s)f(x − sh)ds − f(x)
= \frac{h^{2}}{2} ∫ s^{2}K_{0}(s)dsf′′(x) + o(h^{2})

In particular, explain carefully any special assumptions on K or f that are implicitly being used.

4. The celebrated Wald proof of the consistency of the mle is based on the following Lemma. Let Z_{1}, ..., Z_{n} be a random sample from the density f(z, θ_{0}). Under regularity conditions, on f, for any θ ≠ θ_{0}, as n → ∞,

(∗) P_{θ_{0}}{Π_{i=1}^{n}f(Z_{i}|θ_{0}) > Π_{i=1}^{n}f(Z_{i}|θ)} → 1.

The proof may be slightly condensed to the following “one-liner”

E_{θ_{0}}log(f/f_{0}) < log(E_{θ_{0}}f/f_{0}) = 0.

a.) Explain why (∗) implies consistency of the mle. b.) Briefly describe the regularity conditions. c.) Explain briefly the role of the regularity conditions.

*****

University of Illinois Department of Economics

Spring 1996  Econ 476 Review  Roger Koenker

1. Let X_{1}, …, X_{n} be a random sample from the density f(x, θ_{0}) and θ̂_{n} denote the maximum likelihood estimator of θ_{o}, i.e., θ̂_{n} maximizes

l_{n}(θ) = ∑_{i=1}^{n} log f(X_{i}, θ)

over θ ∈ Θ ⊆ ^{|}R^{p}. Presuming sufficient regularity that θ̂_{n} is efficient in the sense that

\sqrt{n}(θ̂_{n} − θ_{0}) ⇝ η(0, I(θ_{0})^{−1})

where I(θ_{0}) denotes the Fisher Information of f. Sketch the argument for fact that

2(l_{n}(θ̂) − l_{n}(θ_{0})) ⇝ χ_{p}^{2}

2. Let X_{1}, …, X_{n} be a random sample from a df F (x−θ_{0}). Consider L-estimators of location

of the form

θ̂_{0} = ∫_{0}^{1} J(u)F_{n}^{−1}(u)du

where F_{n}(·) denotes the empirical distribution function constructed from the X_{i}’s. If F has finite Fisher information I(F) and a twice continuously differentiable log density, then the optimal L-estimator has weight function of the form

J^{∗}(F (x)) = −\frac{(log f(x))′′}{I(F)}

(a) Explain the observation “note that

∫_{−∞}^{∞} J^{∗}(F (y))dF (y) = ∫_{0}^{1} J^{∗}(u)du = 1

and therefore θ̂_{n} is location equivariant.”

(b) The optimality of θ̂_{n} may be seen by computing the influence function of the general L-estimator as follows: I. The IF of the u^{th} sample quantile is

IF (x, F^{−1}(u), F) = \frac{d}{dε}F_{ε}^{−1}(u) = \frac{u − δ_{x}(F^{−1}(u))}{f(F^{−1}(u))}

which may be shown by differentiating the identity

F_{ε}(F_{ε}^{−1}(u)) = u

where F_{ε}(y) = (1 − ε)F (y) + εδ_{x}(y) to obtain

0 = −F (F_{ε}^{−1}(u)) + δ_{x}(F_{ε}^{−1}(y)) + f_{ε}(F_{ε}^{−1}(u))\frac{d}{dε}F_{ε}^{−1}(u)

and evaluating at ε = 0.

II. Thus

IF (x, θ̂_{n}, F) = ∫_{0}^{1}(J^{∗}(u)(u − δ_{x}(F^{−1}(u)))/f(F^{−1}(u))du
= ∫_{−∞}^{∞} J^{∗}(F (y))(F (y) − δ_{x}(y))dy
= ∫_{−∞}^{x} J^{∗}(F (y))dy − ∫_{−∞}^{∞}(1 − F (y))J^{∗}(F (y))dy
= −I(F)^{−1} ∫_{−∞}^{x}(log f)′′(y)dy
= −I(F)^{−1}(log f)′(x)

III. Setting ψ(x) = −(log f)′(x) = −f′(x)/f(x) we conclude that \sqrt{n}(θ̂_{n} − θ_{0}) ⇝ η(0, EIF^{2}) where

EIF^{2} = ∫ (ψ^{2}(x)/I(F)^{2})dF (x)
= I(F)^{−1}

Explain briefly the foregoing result. Focus on the following aspects (i) How to compute θ̂_{n}. (ii) How does θ̂_{n} differ from the mle. (iii) What does the IF tell us about θ̂_{n}.

3. Explain the following S function. What does it do and how does it do it?

lprq function(x, y, h, alpha){
fv <- numeric(length(y))
der <- fv
for(i in 1:length(y)) {
     z <- x - x[i]
     wx <- dnorm(z/h)
     r <- rq(cbind(1, z) * wx, y * wx, alpha, int = F)
     fv[i] <- r$coef[1]
     der[i] <- r$coef[2]

}

res <- y - fv
list(fv = fv, res = res, der = der)

}