Inferential Statistics(1) - Introduction

1. Power(1-β)

power is the probability of correctly rejecting H0.(1-β)

• Type I error(false positive): H0 is rejected when H0 is true.

  • Solution: decrease significance level α(0.05->0.01):larger Z/T-scorewider CI harder to reject H0H0 more likely to be true.

• Type II error(false negative)(β): H0 is accepted when H0 is false.

  • Solution: lower standard errorless β; more power(1-β)narrower CIharder to accept H0H0 more likely to be false \(se=\frac{S}{\sqrt{(n)}}\)

    • smaller standard deviation

      • more reliable instruments
      • more homogeneous sample

    • larger sample

      • selecting a stronger manipulation
      • more responsive sample

    • parametric test> non-parametric test: assumptions about population distribution and parameters.(more power(1-β))

    • 非参数检验（parametric test）

      参数检验通常是假设总体服从正态分布，样本统计量服从T分布的基础之上，对总体分布中一些未知的参数，例如总体均值、总体方差和总体标准差等进行统计推断。如果总体的分布情况未知，同时样本容量又小，无法运用中心极限定理实施参数检验，推断总体的集中趋势和离散程度的参数情况。这时，可以用非参数检验，非参数检验对总体分布不做假设，直接从样本的分析入手推断总体的分布。

      与参数检验相比，非参数检验适用范围广，特别适用于小样本数据、总体分布未知或偏态、方差不齐及混合样本等各类型数据。二者的对比如下图：

      

    • 非参数检验常用的检验方法表

      

• Estimate power

  1. post-hoc: estimate power after the data have been collected.

  2. a priori: estimate power before we collect any data.

     • standard effect size(S/M/L)

       Effect size is a quantitative measure of the magnitude of the experimenter effect. The larger the effect size the stronger the relationship between two variables.BI

       • 安利一款超好用的效应量（effect size）计算工具
     • helpful
       • research design
       • test selection
       • feasibility of research

  

2. Two independent proportions

• z-test

• z-test 2 proportions

  1. Assumptions

     • independence(random assignment and/or selection)
     • sufficient observations in each sample
       • one-side: 10 negative; 10 positive;
       • two-side: 5 negative; 5 positive;

  2. Hypotheses

     1. H~0~: p1-p2 = 0
     2. H~a~: p1-p2 ≠ 0

  3. Test statistic

     

     \[z = \frac{\hat{p}_1-\hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})({1\over{n_1}}+{1\over{n_2}})}}\\ where \ \hat{p}=\frac{n_1{\hat{p}_1}+n_2{\hat{p}_2}}{n_1+n_2}\]

1. P-value

   Two-tail probability from standard normal distribution of values even more extreme than observed z test statistic presuming the null hypothesis is true.

2. Conclusions

   • if p <= α, reject H~0~.
     • if p > α, accept H~0~.

• Confidence Interval

\[CI= ({\hat{p}_1-\hat{p}_2}) \pm Z_{\alpha} * \sqrt{\frac{\hat{p}_1*(1-\hat{p}_1)}{n_1}+\frac{\hat{p}_2*(1-\hat{p}_2)}{n_2}}\]

• Relative risk

  equals ratio of two proportions.

  This is especially useful if the proportions are very small and the relevance of the absolute difference is hard to evaluate.

  eg. medical sciences.

3. Two independent means

• t-test

• t-test 2 means

  1. Assumption

     • independent observations(random assignment/selection)
     • normally distributed?(one-sided test) (usually > 30)

  2. Hypotheses

     • H~0~: μ1-μ2 = 0
     • H~a~: μ1-μ2 ≠ 0

  3. Test statistic

     

  4. P-value & Conclusion

     

• Alternative calculation

  1. Assumption

     • independent observations(random assignment/selection)
     • normally distributed?(one-sided test) (usually > 30)
     • equal variances

  2. Hypotheses

     • H~0~: μ1-μ2 = 0
     • H~a~: μ1-μ2 ≠ 0

  3. Test statistic

     

  4. P-value & Conclusion

• two ways comparison

  • unequal variance(usually preferred)
    • safer
    • just as good
  • equal variance
    • larger df
    • smaller
    • larger chance to correctly reject H~0~

• Confidence Interval between two means

  

• Permutation test 置换检验

  This test derives the sampling distribution by considering all possible (or a large sample of random) permutations of the original data. A permutation is one particular way of dividing the (n~1~+n~2~) observations into two groups of size n~1~ and n~2~, respectively. For each such permutation, the test statistic (e.g., the difference in the sample means) is computed from the resulting dataset.

  • Assumptions
    • A quantitative response variable observed in each of two groups.
    • Independent random samples, either from random sampling or a randomized experiment. For an experiment, the assumption of a random sample of subjects is not necessary as long as the subjects are assigned to the two groups.
  • Hypotheses
    • H~0~: μ~1~-μ~2~ = 0
    • H~a~: μ~1~-μ~2~ ≠ 0
  • Test Statistic
    • The difference (x~1~-x~2~) of sample means.
  • P-Value
    • The P-Value is the proportion out of all possible permutations with a test statistic as extreme or even more extreme than the observed test statistic.
      • When the number of all possible permutation is too large, a random sample of a large number of permutations( such as 10000) is considered.
  • Conclusion
    • Smaller P-Value give stronger evidence against H~0~ and supporting H~a~. Reject H~0~ if P-Value ≤ significance level(0.05)

4. Two dependent proportions

• pairs/same cases–>eliminates random errors

• McNemar’s test

  • Assumptions
    • sufficient observations
      • 1-sided: n~01~+n~10~≥30
      • 2-sided: no lower limit
  • Hypotheses
    • H~0~:p~1~=p~2~–>n~01~=n~10~
    • H~a~:p~1~≠p~2~–>n~01~≠n~10~
  • Test statistic: z-test
    • 
  • P-Value & Conclusion
    • 

5. Two dependent means

• CI

  

6 Controlling for other variables

• Controlling variables
  • experimental
    • randomization
    • keeping constant–> turn into constant 将变量变为常量。
  • statistical
    • measure and check at every level of control variable

• Possible explanations of other variables

  • Confounder (lurking variable)–A common cause of dependent and independent variables, resulting in a spurious relation.

    

  • Moderator–The relation becomes weaker or stronger.

    

  • Mediator–intermediate

    

• Simpson’s paradox(田忌赛马！不看总量，只看比率都是算流氓！)