24 Est Testing

Stat310 Estimation + Testing

Hadley Wickham
Saturday, 11 April 2009

1. What makes a good estimator?
2. Recap & general strategy
3. Non-symmetric distributions
4. Testing


Low bias, low variance Low bias, high variance

High bias, low variance High bias, high variance


Can combine both
together to get mean
squared error

ˆ = E[(θ − θ)2 ]
ˆ
MSE(θ)
ˆ = Var(θ) + Bias(θ, θ)2
ˆ ˆ
MSE(θ)


Recap
(Z + 1)/5 ~ SomeDistribution(θ, β)
What, mathematically, is a 95%
conﬁdence interval around Z?
Write down the steps you’d take to
generate such an interval if you knew θ
and β


Problem

Y = g(X) Y ~ F(θ) (g has an inverse)
Find a 1 - α conﬁdence interval for X.

i.e. Find a and b so that
P(a < X < b) = 1 - α


Solution
1. Find a 1 - α conﬁdence interval for Y.
P(c < Y < d) = 1 - α
a. If F is symmetric, then the bounds will
-1(α/2) and d = F-1(1 - α/2)
be c = F
b. If F isn’t symmetric then it’s harder
2. a = g -1(c), b= g -1(d)


Example
340 333 334 332 333 336 350 348 331
344 (mean: 338, sd: 7.01)
Find a 95% conﬁdence interval for μ

¯n − µ
X
√ ∼ tn−1
s/ n

More complicated case

(n − 1)S 2
X ∼ χ (n − 1)
2
X= 2
σ
Find 95% conﬁdence interval for standard
deviation in previous case
(sd = 7.01, n = 10)


Standard deviation

Find confidence interval for X ~ χ2(9).
Generally want the shortest confidence
interval, but hard to find when not
symmetric.
Any of the following are correct. The best
has the smallest interval.


0.10

0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


0.10
(0.05, 1)
(3.33,Inf)
Length: Inf
0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


0.10
(0.03, 0.99)
(2.85,21.67)
Length: 18.8
0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


0.10
(0.025, 0.975)
(2.7,19.0)
Length: 16.3
0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


0.10
(0.01, 0.96)
(2.09,17.61)
Length: 15.5
0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


0.10
(0, 0.95)
(0.0,16.9)
Length: 16.9
0.08

0.06

0.04

0.02

0.00

0 5 10 15 20 25 30


Your turn
Find 95% conﬁdence interval for the
standard deviation (sd = 7.01, n = 10)

P(2.09 < X < 17.61) = 0.95

(n − 1)S 2
X= 2
σ

Testing


Testing

Very closely related to estimation
(particularly conﬁdence intervals)
But point is to answer a yes/no question:
Is the mean of the distribution equal to 0?
Do X and Y have the same mean?


Your turn
The following values have been drawn
from a normal distribution with standard
deviation 1.
2.9 2.1 3.0 3.2 1.2 3.0 3.3 1.2 2.3 1.5
(mean: 2.13)
Is it possible they came from a normal
distribution with mean 1.5?


Example
Create 95% conﬁdence interval.
Is it inside?
Create 90% conﬁdence interval.
Is it inside?
…
Or we can look up the value directly,
using the cdf


Testing jargon
No: Null hypothesis. Nothing is
happening. (Thing we want to disprove)
Yes: Alternative hypothesis. Something
interesting is happening.

Major complication:


Absence of
evidence is not
evidence of absence


Implication

Means we never “accept” the null
hypothesis, just “fail to reject” it.

Null distribution is usually simple case for
which we know the distribution


Your turn
Null hypothesis: μ = 1.5
Alternative hypothesis: μ > 1.5 OR μ < 1.5
Under the null hypothesis what is the
distribution of the mean?
How does what we saw compare to the
null distribution? Is it likely or not?


P-value
P value gives us the probability, under the
null hypothesis, that we would have seen a
value equal to or more extreme than the
value we observed.
Strength of evidence for rejecting the null
hypothesis.
But we need a cut off to make a yes-no
decision. How do we choose that cut off?


Errors
What are the possible errors we can
make?
False positive. Choose alternative when
null is correct. (aka Type 1)
False negative. Choose null when
alternative is true. (aka Type 2)


Terminology
Probability of a false positive called α
Probability of false negative called 1 - β

How are the two related?
Usually care more about false positives


Testing overview

Write down null and alternative
hypotheses.
Compute test statistic.
Convert to p-value.
Compare p-value to alpha cut off.


Next time

Some speciﬁc tests.
i.e. for common situations what is the
distribution under the null-hypothesis


24 Est Testing

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (17)

Mais de Hadley Wickham

Mais de Hadley Wickham (20)

Último

Último (20)

24 Est Testing