Inferential Statistics Hypothesis tests

Inferential Statistics Hypothesis tests

Eva Riccomagno, Maria Piera Rogantin

DIMA – Universit`a di Genova

riccomagno@dima.unige.it rogantin@dima.unige.it

Part F

Hypothesis tests for the equality of two means

a) paired samples

b) two-samples

Review

Exercise. Chicago Tribune Chicago land’s technology professionals get local technology news from various newspapers and magazines. A marketing company claims that 25% of the IT professionals choose the Chicago Tribune as their primary source for local IT news. A survey was conducted to check this claim. Among a sample of 750 IT professionals in the Chicago land area, 23.47% of them prefer the Chicago Tribune. Can we conclude that the claim of the marketing company is true?

The random variable modeling the preference of the Chicago Tribune is X ∼ B(1, p)

Test statistic: P = X; sample value: ˆ^b p = 0.2347

Large sample size (n = 750). Using CLT P ∼ N^{b }p, ^p(1−p)_n ^ H₀ : p = 0.25 H₁ : p 6= 0.25 or p < 0.25?

p-value computation in R using t.test

> np=750*0.2347

> prop.test(np,750,0.25)

1-sample proportions test with continuity correction data: np out of 750, null probability 0.25

X-squared = 0.85654, df = 1, p-value = 0.3547

alternative hypothesis: true p is not equal to 0.25 95 percent confidence interval:

0.2051343 0.2670288 sample estimates:

p 0.2347

> prop.test(np,750,0.25,"less")

1-sample proportions test with continuity correction data: np out of 750, null probability 0.25

X-squared = 0.85654, df = 1, p-value = 0.1774 alternative hypothesis: true p is less than 0.25 95 percent confidence interval:

0.0000000 0.2617696 sample estimates:

p 0.2347

In both cases there is not evidence to reject H₀ (p = 0.25)

Test for the equality of two means

A common application is to test if a new process or treatment is superior to a current process or treatment

The data may either be paired or unpaired

a) Paired samples When there is a one-to-one correspondence between the values in the two samples. That is, if X₁, X₂, . . . , X_n and Y₁, Y₂, . . . , Y_n are the two sample variables, then X_i cor- responds to Y_i

b) Unpaired samples The sample sizes for the two samples may or may not be equal

a) Paired samples

Let X and Y be two random variables modeling a characteristic of the same population

Example. Drinking Water

(from https://onlinecourses.science.psu.edu Penny State University)

Trace metals in drinking water affect the flavor and an unusually high concentration can pose a health hazard

Ten pairs of data were taken measuring zinc concentration in bottom water and surface water

water

bottom surface 1 0.430 0.415 2 0.266 0.238 3 0.567 0.390 4 0.531 0.410 5 0.707 0.605 6 0.716 0.609 7 0.651 0.632 8 0.589 0.523 9 0.469 0.411 10 0.723 0.612

●

●

●

●

●●

●

●

●

●

0.2 0.3 0.4 0.5 0.6 0.7

0.30.40.50.60.7

bottom

surface

attach(water)

m=min(surface,bottom) M=max(surface,bottom) plot(surface~bottom, asp=1,pch=16,

xlim=c(m,M),ylim=c(m,M), cex.axis=1.5,cex.lab=1.5) abline(0,1,col="red",lwd=2)

Assume X ∼ N (µ_X, σ_X² ) and Y ∼ N (µ_Y , σ_Y²) Test hypotheses

H₀ : µ_X = µ_Y and H₁ : µ_X 6= µ_Y

or equivalently H₀ : µ_X − µ_Y = 0 and H₁ : µ_X − µ_Y 6= 0 (simple or composite hypotheses – one/two sided) Let (X₁, Y₁), . . . , (X_n, Y_n) be the n paired sample variables Consider the sample random variables D₁, . . . , D_n with

D_i = X_i − Y_i Consider the sample mean of D

D ∼ N (µ_D, σ_D² /n)

with µ_D = µ_X − µ_Y and σ_D² = σ_X² + σ_Y² − 2Cov(X, Y ), usually un- known and estimated by the unbiased estimator S_D²

The test for mean equality of becomes a Student’s t test on µ_D, with H₀ : µ_D = 0

Example. Drinking Water (continue)

> D=surface-bottom;D

[1] 0.015 0.028 0.177 0.121 0.102 0.107 0.019 0.066 0.058 0.111

• Hypotheses: H₀ : µ_D = 0 and H₁ : µ_D 6= 0

• Two-sided: R₀ = (−∞, c₁) ∪ (c₂, +∞)

• Sample size: n = 10

• Sample variables: D₁, . . . , D₁₀ i.i.d. D_i ∼ N (0, σ_D² ) with σ_D² estimated by S_D²

• Test statistic under H₀: T = ^D

S_D/√

n ∼ t₉

• α = 0.05

The thresholds of the rejection region c₁ and c₂ are such that 0.025 = P(T < c1 | µ_D = 0) 0.025 = P(T > c2 | µ_D = 0) Observe that, because of the symmetry w.r.t. 0 of the Student’s t density

c₁ = −c₂

In the sample: d = 0.0804 s = 0.052

The sample value of the test statistic, under H0, is 4.86

> d_m=mean(D);d_m; s=sd(D);s [1] 0.0804

[1] 0.05227321

> t=d_m/(s/sqrt(10));t [1] 4.863813

The rejection region is R0 = (−∞, −2.262) ∪ (2.262, ∞). The p-value is 0.0009

> c1=qt(0.025,9) [[1] -2.262157

> 2*(1-pt(t,9)) [1] 0.0008911155

The direct computation in R produces

> t.test(surface,bottom,paired=TRUE) Paired t-test

data: surface and bottom

t = -4.8638, df = 9, p-value = 0.0008911

alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:

-0.117794 -0.043006 sample estimates:

mean of the differences -0.0804

There is experimental evidence to reject H0

b) Unpaired samples

Example. Prey of two species of spiders

(from https://onlinecourses.science.psu.edu Penny State University)

The feeding habits of two species of net-casting spiders are stud- ied. The species, the deinopis and menneus, coexist in eastern Australia. The following data were obtained on the size, in mil- limeters, of the prey of random samples of the two species.

The spiders were selected randomly and thus we assume inde- pendent measurements.

> d=c(12.9,10.2,7.4,7.0,10.5,11.9,7.1,9.9,14.4,11.3)

> m=c(10.2,6.9,10.9,11.0,10.1,5.3,7.5,10.3,9.2,8.8)

> mean(d);mean(m) [1] 10.26

[1] 9.02

d m

68101214

Normal distribution

Assume the prey size of the two population (denoted by A and B) follow a Normal distribution

X_A ∼ N (µ_A, σ_A²) X_B ∼ N (µ_B, σ_B² )

Let n_A and n_B be the size of the two independent sample of X_A and X_B. In the example n_A = n_B = 10.

We want to test H₀ : µ_A = µ_B and H₁ : µ_A 6= µ_B

or equivalently H₀ : µ_A − µ_B = 0 and H₁ : µ_A − µ_B 6= 0 The two sample mean random variables are

X_A ∼ N µ_A, σ_A² n_A

!

X_B ∼ N µ_B, σ_B² n_B

!

The random variable difference of the two sample mean random variables follows the Normal distribution

X_A − X_B ∼ N µ_A − µ_B, σ_A²

n_A + σ_B² n_B

!

The original test becomes a test on the mean of one Normal random variable

1. The variances σ_A² and σ_B² are known Fixed α, a usual z-test is carried out

2. The variances σ_A² and σ_B² are unknown, and assumed equal and estimated by the unbiased estimators S_A² e S_B²

A unbiased estimator of the variance of the random variable X_A − X_B is S² = (n_A − 1)S_A² + (n_B − 1)S_B²

(n_A + n_B − 2) · n_A + n_B

n_A n_B (Pooled variance) In particular, if n_A = n_B, then S² = S_A² + S_B²

/n_A

The test statistic is T =

X_A − X_B^ − (µ_A − µ_B)

S with T ∼ t_n

A+n_B−2

Fixed α, a usual Student’s t test is carried out 3. The unknown variances σ_A² and σ_A² are not equal

A hypothesis test based on the t distribution, known as Welch’s t-test, can be used

Example. Prey of two species of spiders (continue)

• Hypotheses: H₀ : µ_D = µ_M and H₁ : µ_D 6= µ_M

• Two-sided: R₀ = (−∞, c₁) ∪ (c₂, +∞)

• Sample size: n_D = n_M = 10

• First, assume σ_D² = σ_M² . Pooled variance estimator:

S² = ^S_D² + S_M^{2 } /n_D

• Test statistic under H₀:

T =

X_D − X_M^

S ∼ t_2n

D−2

• α = 0.05

The thresholds of the rejection region c₁ and c₂ are such that 0.025 = P(T < c1 | µ_D = µ_M) 0.025 = P(T > c2 | µ_D = µ_M)

The sample means of the two groups are:

x_D = 10.26 x_M = 9.0.2

The sample difference of means is: x_D − x_M = 1.24 The sample pooled variance is: s² = 1.01

The sample value of the test statistic, under H₀, is 1.18

> diff_m=mean(d)-mean(m);diff_m [1] 1.24

> s2=(sd(d)^2+sd(m)^2)/10;s2 [1] 0.9915556

> t=diff_m/sqrt(s2);t [1] 1.245269

The rejection region is R₀ = (−∞, −2.1) ∪ (2.1, ∞) The p-value is 0.25

> c1=qt(0.025,18);c1 [1] -2.100922

> 2*(1-pt(t,18)) ## note 2*( ) -- two sided test [1] 0.2290008

There is no experimental evidence to reject H₀

Can we assume equal variances?

A specific test can be performed (based on the Fisher distribu- tion). Here we do not give the details. Compute in R

> var.test(m, d, ratio = 1)

F test to compare two variances data: m and d

F = 0.56936, num df = 9, denom df = 9, p-value = 0.4142

alternative hypothesis: true ratio of variances is not equal to 1 95 percent confidence interval:

0.1414206 2.2922339 sample estimates:

ratio of variances 0.5693585

We can assume the σ_D² = σ_M² , although the ratio of variances is 0.57. This apparent inconsistency is due to the small sample sizes

Direct computation in R of the test

H₀ : µ_D = µ_M and H₁ : µ_D 6= µ_M, assuming σ_D² = σ_M²

> t.test(d,m,var.equal=T) Two Sample t-test data: d and m

t = 1.2453, df = 18, p-value = 0.229

alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:

-0.8520327 3.3320327 sample estimates:

mean of x mean of y 10.26 9.02

If the equality of the variances is rejected, we use the Welch Two Sample t-test

In such a case the polled variance s² and the degrees of freedom are compute in an another manner

Compute in R

t.test(d,m)

Welch Two Sample t-test data: d and m

t = 1.2453, df = 16.74, p-value = 0.2302

alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:

-0.8633815 3.3433815 sample estimates:

mean of x mean of y 10.26 9.02

The problem of making inference on means when vari- ances are unequal, is, in general, quite a difficult one. It is known as the Behrens-Fisher Problem

(G. Casella, R.J. Berger, Statistical Inference, 2nd ed., Duxbury, Ex.

8.42)

Notes and generalisations

• The Wald test. If the two random variables are not normally distributed and the sample size is “large” a Wald test can be performed

• Threshold different from zero. In some applications, you may want to adopt a new process or treatment only if it exceeds the current treatment by some threshold. In this case, the difference between the two mean is not compared with 0 but with the chosen threshold