Chapter 17_Maximum Likelihood Estimation(计量经济学-西安交通大学,李庆男)

T
−[(T − 1)/2] log(2π ) − [(T − 1)/2] log(σ 2 ) −
t=2
(yt − c − φyt−1 )2 . 2σ 2
(6)
3
1.2
Evaluating the Likelihood Function Using (Vector) Joint Density
A different description of the likelihood function for a sample of size T from a Gaussian AR(1) process is some time useful. Collect the full set of observations in a (T × 1) vector, y ≡ (Y1 , Y2 , ..., YT ) . The mean of this (T × 1) vector is E (Y1 ) E (Y2 ) . E (y) = . . E (YT )
which might loosely be viewed as the probability of having observed this particular sample. The maximum likelihood estimate (M LE ) of θ is the value for which this sample is most likely to have been observed; that is, it is the value of θ that maximizes (1). This approach requires specifying a particular distribution for the white noise process εt . Typically we will assume that εt is gaussian white noise: εt ∼ i.i.d. N (0, σ 2 ).
t=2
fYt |Yt−1 (yt |yt−1 ; θ ). (4)
The log likelihood function (denoted L(θ)) is therefore
T
L(θ ) = log fY1 (y1 ; θ ) +
t=2
log fYt |Yt−1 (yt |yt−1 ; θ ).
(5)
1
1
1.1
MLE of a Gaussian AR(1) Process
Evaluating the Likelihood Function Using (Scalar) Conditional Density
A stationary Gaussian AR(1) process takes the form Yt = c + φYt−1 + εt , (2)
1 V= (1 − φ2 )
. φT −1 . φT −2 . . . . . . . 1

.
(6) and (7) must represent the identical likelihood function. 4
Then (7) becomes
Ch. 17 Maximum Likelihood Estimation
The identification process having led to a tentative formulation for the model, we then need to obtain efficient estimates of the parameters. After the parameters have been estimated, the fitted model will be subjected to diagnostic checks . This chapter contains a general account of likelihood method for estimation of the parameters in the stochastic model. Consider an ARM A (from model identification) model of the form Yt = c + φ1 Yt−1 + φ2 Yt−2 + ... + φp Yt−p + εt + θ1 εt−1 +θ2 εt−2 + ... + θq εt−q , with εt white noise: E (εt ) = 0 E (εt ετ ) = σ 2 f or t = τ . 0 otherwise
σ2 . 1 − φ2
Since {εt }∞ t=−∞ is Gaussian, Y1 is also Gaussian. Hence, fY1 (y1 ; θ) = fY1 (y1 ; c, φ, σ 2 ) 1 1 {y1 − [c/(1 − φ)]}2 = √ . exp − · 2 σ 2 /(1 − φ2 ) 2π σ 2 /(1 − φ2 ) Next consider the distribution of the second observation Y2 conditional on the observing Y1 = y1 . From (2), Y2 = c + φY1 + ε2 . (3)
= fY3 |Y2 ,Y1 (y3 |y2 , y1 ; θ )fY2 |Y1 (y2 |y1 ; θ )fY1 (y1 ; θ ).
In general, the value of Y1 , Y2 , ..., Yt−1 matter for Yt only through the value Yt−1 , and the density of observation t conditional on the preceding t − 1 observations is given by fYt |Yt−1 ,Yt−2 ,...,Y1 (yt |yt−1 , yt−2 , ..., y1 ; θ )
The log likelihood for a sample of size T from a Gaussian AR(1) process is seen to be 1 {y1 − [c/(1 − φ)]}2 1 L(θ ) = − log(2π ) − log[σ 2 /(1 − φ2 )] − 2 2 2σ 2 /(1 − φ2 )
= fYt |Yt−1 (yt |yt−1 ; θ ) 1 1 (yt − c − φyt−1 )2 = √ . exp − · 2 σ2 2πσ 2
The likelihood of the complete sample can thus be calculated as
T
fYT ,YT −1 ,YT −2 ,...,Y1 (yT , yT −1 , yT −2 , ..., y1 ; θ ) = fY1 (y1 ; θ ) ·
with εt ∼ i.i.d. N (0, σ 2 ) and |φ| < 1 (How do you know at this stage ?). For this case, θ = (c, φ, σ 2 ) . Consider the p.d.f of Y1 , the first observations in the sample. This is a random variable with mean and variance E (Y1 ) = µ = V ar (Y1 ) = c and 1−φ
This chapter explores how to estimate the value of (c, φ1 , ..., φp , θ1 , ..., θq , σ 2 ) on the basis of observations on Y . The primary principle on which estimation will be based is maximum likelihood estimation. Let θ = (c, φ1 , ..., φp , θ1 , ..., θq , σ 2 ) denote the vector of population parameters. Suppose we have observed a sample of size T (y1 , y2 , ..., yT ). The approach will be to calculate the joint probability density fYT ,YT −1 ,...,Y1 (yT , yT −1 , ..., y1 ; θ ), (1)
=


µ µ . . . µ

= µ, . . . . . . . φT −1 . φT −2 . . . . . . . 1
= σ2V
The sample likelihood function is therefore the multivariate Gaussian density: 1 fY (y; θ ) = (2π )−T /2 |Ω−1 |1/2 exp − (y − µ) Ω−1 (y − µ) , 2 with log likelihood L(θ ) = (−T /2) log(2π ) + 1 1 log |Ω−1 | − (y − µ) Ω−1 (y − µ). 2 2 (7)

where
where µ = c/(1 − φ). The variance -covariance of y is 1 φ . φ 1 φ 1 . . . Ω = E [(y − µ)(y − µ) ] = σ 2 2 . . (1 − φ ) . . . . T −1 φ . . 1 φ . . . φT −1 φ 1 . . . . . φ . . . . . . . . . .