统计学课件--概率论

Outline
• Probability Concepts
– Basic Concepts – Conditioning Concepts
• Law of Total Expectation/Variance, Bayes’ Theorem
• Limit Theorems
– Modes of Convergence – Law of Large Numbers (LLN) – Central Limit Theorem (CLT)
• Intersection (and) 交集 & &&
– e.g., A=heads on first, B=heads on second – A ∩ B = {H,H}
• Complement 补集 !
– sets of all outcomes not in A – e.g., A={T,T}, Ac={H,H},{H,T},{T,H}
Random Variable
Discrete Random Variable Continuous Random Variable
11
Condit百度文库oning Concepts
• Conditioning Concepts
– Conditional Probability
• Law of Total Probability • Bayes’ Theorem
probability of the event – the probability of A, P(A), regardless of whether B has occurred.
• Joint probability 联合概率: the probability of two
events in conjunction – P(A∩B), P(AB), P(A,B) – If there are two possible outcomes for r.v. X with events B and Bc, then P(A)=P(A∩B)+ P(A∩Bc) Law of Total Probability
• conditional probability density function (pdf) of Y given X
f XY ( x, y ) fY | X ( y | x ) f X ( x)
14
Conditioning Concepts
• Comments:
– Conditional distribution can be viewed as a probability distribution defined on a reduced sample space.
– Conditional Expectation
• Law of Total Expectation
– Conditional Variance
• Law of Total Variance
12
Conditioning Concepts
• Marginal probability 边际概率: the unconditional
9
Basic Concepts
• If A and B are disjoint,
– they have no element in common. – and if A occurs, then B cannot occur.
• P(A ∩B) =0 • P(A U B) = P(A) + P(B) A Black Cards B Red Cards
• Probability 概率论
(mathematical foundations of statistics)
Inductive 归纳 (particular → general)
Given the information in the box, what is in your hand?
– Assume known population and parameters, and compute the probability of drawing a particular sample. Deductive 演绎 (general → particular) 2
13
Conditioning Concepts
• Conditional probability
– the probability of A, given the occurrence of B
• P(A|B) = P(A∩B)/P(B) → P(A∩B) = P(A|B)P(B) Rule of multiplication 乘法原则 • Independence P(A|B)=? Disjoint events P(A|B)=?
Lecture 3: Review of Probability
第三讲：概率论回顾
1
Statistics VS Probability
• Statistics 统计学
Given the information in your hand, what is in the box?
– Assume unknown population and parameters, and sample is used to infer the values of the parameters.
– i.e., a function from events to probability levels
6
Basic Concepts
• Union (or) 并集 | ||
– e.g., A=heads on first, B=heads on second – A U B= {H,T},{H,H},{T,H}
7
Basic Concepts
• Rule of addition 加法原则
–AUBUC=?
8
Basic Concepts
• Independent events ≠ Disjoint (mutually exclusive) events 独立≠不相交/互斥
• If A and B are independent (A⊥B),
– the occurrence of A/B makes it neither more nor less probable that B/A occurs,
• P(A ∩ B) = P(A) * P(B) • P(A U B) = P(A) + P(B) – P(A ∩B) – Independent events A = heads on one toss of fair coin B = heads on second toss of same coin – Dependent events A = rain forecasted on the news B = take umbrella to work
• Limit Theorems
– Modes of Convergence – Law of Large Numbers (LLN) – Central Limit Theorem (CLT)
• Distribution and Statistic
• Chebyshev’s Theorem
4
Basic Concepts
16
Conditioning Concepts
• Conditional expectation 条件期望
– the expected value of a r.v. with respect to a conditional probability distribution p ( xy ) E[Y | X x] y pY | X ( y | X x) y p X ( x) y y
E[XY] = E[X]E[Y|X] = E[Y]E[X|Y]
17
Conditioning Concepts
• Law of Total/Iterated Expectation (LIE) 全/重期望律
E[X]=E[E[X|Y]]
• Applications of LIE – Other versions: E[X|I1]=E[E[X|I2]|I1] where the value of I1 is determined by that of I2.
– Discrete
• conditional probability mass function (pmf) of Y given X
P( X x Y y) pY | X ( y | x) pY | X (Y y | X x) P( X x)
– Continuous
– a mathematical construct that models a real-world experiment consisting of states that occur randomly – consists of three parts
• a sample space • a set of events • the assignment of probabilities to the events
p
y
Y|X
( y | x) 1
f
y
Y|X
( y | x) 1
15
Conditioning Concepts
• Bayes’ theorem 贝叶斯定理
– relates conditional and marginal probabilities of events A and B, where B has a non-zero probability:
• e.g., S={H,H},{H,T},{T,H},{T,T}
• Event (A or F) 事件
– a set of outcomes (subset of S).
• e.g., no heads A={T,T}
5
Basic Concepts
• Probability Space 概率空间
A card cannot be Black and Red at the same time.
10
Basic Concepts
• Random Variable (X) 随机变量 r.v.
– represents a possible numerical value from a random event.
– P(A) = prior probability 先验概率
• the probability of A given only initial information
– P(A|B) = posterior probability 后验概率
• the updated conditional probability of A, given initial information and the outcome of B
• Random Experiment 随机试验
– procedure whose outcome cannot be predicted in advance
• e.g., toss a coin twice
• Sample Space (S or
) 样本空间
– the set of all possible outcomes
f XY ( x, y ) E[Y | X x] y fY | X ( y | x)dy y dy f X ( x) y y
– Conditional expectation & regression 回归:
regression line: y = E[Y|X]
– Independence & Disjoint: E[Y|X=x] = ? E[XY] = ?
• Distribution and Statistic
• Chebyshev’s Theorem
3
Outline
• Probability Concepts
– Basic Concepts – Conditioning Concepts
• Law of Total Expectation/Variance, Bayes’ Theorem