8-Fundamental of estimation theory

Unbiased
• The sample variance
m 1 2 2 ˆ ˆ yi m i 1
UESTC-何子述等
9
However, From(10.9)-(10.14)
2 m
1 2 ˆ E yi E m i 1 m m 1 2 m m
In this case
Depends only on the parameter estimate & p
UESTC-何子述等
Independent of parameter
15
Both are sufficient estimates!
10.5.5 Minimum-variance estimates
13
Consider the sample p mean where the individual random variables are independent and each is normally ll distributed di t ib t d with ith mean and d variance i 2 joint pdf p of measurements is given g by y . The j
p y1 , ,... ym ; g T y1 ,... , ym , h y1 ,... , ym
Depends p only y on the UESTC-何子述等 estimate & parameter Independent of parameter
2


2
2
m 1 2 2 ˆ lim E lim m m m
2
The sample variance is biased but asymptotically unbiased estimate estimate.
UESTC-何子述等 10
10.5.2 Consistent estimates
p y1 , ,... ym ;
2

i
1
As
m 2 i 1 i
2
2
m 2 2
m yi 2 exp p 2 i 1 2
2
y y y y
i 1 m
ˆ y α
.
UESTC-何子述等
7
10.5.1 Unbiased estimates
ˆ An estimate y is unbiased, , if
ˆ is i a constant vector ˆ E = is i a random d vector t
UESTC-何子述等 17
An alternative form of the Cramer-Rao inequality Is
ˆ V 1
2 ln p y1 ,..., ym ; E
The equality condition is obtained when
The Proof for CRLB Proof:
p y; dy = 1
p y; p y; d y dy = 0
Th pdf The df can be b expressed d as
p y1 , ,... ym ; 2
h y1 , y2 , , ym 1
2 2 s y 1 y exp m 1 exp m 2 m /2 2 2 2 2 2
T y is a sufficient statistic for if the conditional pdf p y1 , y2 ,, ym T y , does not depend on .
A method used to determine whether or not an estimate is a sufficient estimate is to see if the Fisher factorization theorem is satisfied. satisfied
The variance of an unbiased estimate is a estimate performance measure . A i An important t t bound b d on the th variance i is i the th Cramer-Rao inequality q y CRB or CRLB (Cramer-Rao Low Bound)
Asymptotically unbiased estimate

m
UESTC-何子述等
ˆ lim E =
8

10.5.1 Unbiased estimates
In the example 10.1 • the sample mean
1 m ˆ y yi ˆ =α m i 1
2
In practice practice, it is more useful. useful
UESTC-何子述等
12
10.5.4 Estimators based on sufficient statistics
ˆ T y in the y ? Whether the full information is used to estimate
UESTC-何子述等 2
10.2 Formulation of the general g parameter estimation problem
The parameter estimation problem is to find a parameter estimate vector that depends on the Estimator observables.
• The Th parameter t α may be b random d vector t or a
nonrandom parameter vector. • The parameter estimation occur depending on whether pdf is known or unknown.
ˆ V 1 2 E 2 ln p y1 ,..., ym ;
Fisher information
UESTC-何子述等 16
Therefore, the inequality q y can be expressed p as
ˆ F 1 V
ˆ m is consistent if it converges An estimate g stochastically to as m becomes large. A explicit An li it statement t t t of f consistency it is i that th t for f ˆm every y small number, , 0 , the estimate concentrates on the parameter as m becomes l large
The estimate can be defined by
A direct estimation way 1 m ˆ y yi ˆ =α m i 1
UESTC-何子述等
1 m 2 ˆ yi ˆ m i 1
(估计子)
ˆ y to estimate a parameter! Construct a function α
UESTC-何子述等 3
• The p parameter estimate α p on the ˆ y depends
random observables and is itself a random variable.
m
ˆ m 1 lim P
ˆ m lim P 1 m
or
UESTC-何子述等
11
10.5.2 Consistent estimates
Mean square consistent estimate
ˆm 0 lim E m
UESTC-何子述等 4
Example 10.1
Constant number p plus zero mean noise are
yi ni ,
1 ˆ yi m i 1
m
i 1,...., m
2 y 1 i p yi ; exp 2 2
where h
2 F E 2 ln p y1 ,..., ym ;
If the estimate of the parameter satisfies the Cramer-Rao bound with equality, then it is termed a most efficient estimate.(优效估计)
Chapter 10 Fundamentals of Estimation Theory
UESTC-何子述等
1
10.1 Chapter highlights
• Parameter estimation problem formulation • Types of estimation problems • Bayes yes es estimation o • Minimax estimation • Maximum-likelihood estimation • Comparison of estimators of parameters