Research Design

1. True Experiments

Randomized control trials/RCT

Maximize internal validity

• 3 essential ingredients for true experiments:

  • Manipulation

    • cause under control
    • eliminate temporal precedence: cause precedes effect

  • Comparison

    • control group
    • ensures effect does not occur naturally: maturation

  • Random assignment

    • ensures there are no other explanations for the effect
    • Replication
      • is required to ensure randomization works

2. Factorial Design 析因设计

• several independent variables–factors
  • investigated simultaneously
• one dependent variable

3. Repeated Measures

1. Between Factor 组间因素

   • the dependent variable measured after being exposed to the independent variable.
     • gender

2. Within Factor 组内因素

   • the dependent variable measured after being exposed to each level of the independent variable.

     • different level of dosages to one patient

   • Repeated measures design 重复测量设计

     • measured repeatedly
     • Short period
   • at least one within factor and one or more between factors

3. Longitudinal design 纵向设计

   • measured repeatedly

   • long period of time

   • rare

   • mostly correlation studies

   • sometimes (quasi-) experimental studies

4. Research Control

1. Manipulation

   • control over independent variable

     • value/level determined by researcher

   • control over external variables

     • rule out alternative explanations

   • terms: level;condition;group

   • experimental variable

     • independent variable that can be fully controlled

   • individual difference variables

     • intrinsic property of participant that cannot be controlled
     • age/gender

   • Seemingly non-manipulable variables that can be manipulated

     • a manipulation check

     • measuring after the dependent effect variables

2. Control of variables of disinterest

   ideal: only difference: independent variables

   • ceteris paribus: all other things are equal

   • much harder in social science to control

   • Control variables

   • solution:

     • Randomization随机化

       eliminates all systematic differences between participants in different conditions on average

       • Randomization failure: unequal distribution possible

       • solution:

         1. Randomization check: measure relevant variables to check whether randomization worked and the conditions are the same.

         2. Restricted randomization（限制性随机）

            guarantee randomization over selected variables

       • blocking（区组）: guarantees equal group sizes

         • pair participants in blocks of two
         • flip a coin to determine where the first participant goes
         • the second… participant is automatically assigned the other condition

       • stratified restricted randomization(分层限制性随机): equal distribution on participant characteristic

         • boys/girls
         • sex
         • age

       • Limits:

         • >2,3 variables–>too complicated
         • too many possibly relevant variables
         • Randomization with repeated measures

   • It is necessary!

   • avoiding maturation

     • counterbalancing: order randomized抵消平衡法
       • Within subject designs: the order in which subjects are exposed to the conditions should be randomized.
       • Subjects are assigned to one out of all possible orderings of the conditions possibly using blocking to ensure that no one ordering is over represented.列出所有可能的实验条件序列 并将受试者分配到每一组 尽量使用区组来确保各组人数相当

3. Matching

   • Random assignment impossible?
     • threat of selection to internal validity
     • pragmatic/ethical reasons
     • independent variable–>individual differences variable
   • Matching on relevant background variables
   • Threat: undermatching
     • measured with some error
     • related to the variables of interest
     • Due to regression to the mean!(均值回归)
       • 选择的对象没有代表性导致重复试验时向均值回归。
     • detrimental effect!

5. Lab vs Field

• Lab

  • often experimental
  • maximizes internal validity
  • low ecological validity? mundane realism?
    • not similar to natural setting
    • Low ecological validity doesn’t imply low construct validity or low external validity
  • experimental realism
    • experimental translation of natural setting

• Field

  • often non-experimental
  • maximize external validity
  • uncontrolled environment
  • often observing natural behavior

6. Experimental Designs

1. Two-group Design双组设计

   • Experimental group: the hypothesized cause is present

   • Control group: the cause is absent

   • Two-group pretest/post-test design

     

     • avoid maturation
     • compare the size of the increase or decrease in scores

   • pretest

     • sensitize 被试敏感化
     • practice effect 练习效应
       • higher score in post-test
     • alert participants to the purpose of the study

2. Solomon four-group design 所罗门四组设计

   • two group design+ two group pre/post-test design

     

     • the experimental/control condition are run twice
       • one with a pretest
       • one without a pretest
     • avoid practice effect

3. Repeated measure design with one within subject factor 单一主体因素重复测量设计

   • all participants are exposed to all levels of the independent variable. They experience all conditions.

   

7. Quasi-experimental designs

• non random assignment

  • random assignment can be: impractical/unethical

• sometimes no manipulation, or comparison

  • individual differences variable: non-manipulable
    • sex
    • disease

• investigates causal relation

• natural event/natural experiment

• values selected, not manipulated

  • like correlational studies
  • difference: 准实验研究检验的是因果假说 且会尽可能地极力控制额外变量.

• Types of designs

  1. Static groups comparison

     Static groups comparison

     • similar to the two group experimental design
     • a post-test after exposure to the independent variable
     • Difference: Non random assignment
       • internal selection threat
       • Comparison of groups lessens the threats of maturation and history

  2. Pre/post-test Non-equivalent control group 不等组前测/后测设计

     Pre/post-test Non-equivalent control group

     • firmly disambiguate the causal order
     • assess the size of any maturation effects
     • not always possible to use in control group

  3. One group pre/post-test 单组前测-后测设计

  4. Interrupted time-series design 中断时间序列设计

     

     • more observations/ tests before and after the treatment

     • avoid maturation

     • still history threat

       • solution:

         • replicated interrupted time-series design

           • control group
           • pre/post-test non-equivalent control group with more tests

         • redo the experiment at a different time

           

8 Correlational design相关性研究

• no manipulation of independent variable

• or no independent variable identified

• no causal direction specified

• don’t select/manipulate just measure

• types:

  

  1. Cross sectional design 横断面实验设计

     • one large group is considered at one specific point of time
     • a fairly large number of properties are measured at once
     • aim: to investigate the association between variables in a way that accurately describes a larger population
     • survey study or survey design

  2. time-series design 时间序列设计

     • one person is measured at several points in time, usually with many measurement moments in quick succession and in fixed intervals.

       • longitudinal design vs. time-series vs. repeated measures

         • longitudinal

           • general term for measuring repeatedly over long period of time

         • time-series

           • correlational: 1 person measured over time
           • quasi-experimental: people in conditions measured repeatedly
           • experimental: N=1 study: 1 person measured over time in all conditions

         • repeated measures

           • experimental: persons measured over time in all conditions where
           • one or more independent variables are manipulated

  3. Panel design 群组实验设计

     • also call time-series cross-sectional design(TSCS)
     • the same group of people is measured at several points in time
       • cohort design 队列实验设计
         • a special case of panel design
         • participants are all same age or started a new school or a job at the same time

9 Other designs

• case study

  • one person/group(as as whole)
  • quantitative: n=1, time-series
  • qualitative: generating hypotheses more than testing
  • negative analysis
    • contradictory evidence
    • counter-evidence against a hypothesis

• evaluation research 评价研究

  • applied research, non -experimetnal
  • broader scope
  • effect of policy/program
  • summative: outcome of a program
  • formative:process assessing how a program works
  • sociology, communication sciences, political sciences

• Intervention study 干预研究

  • experimental
  • effect of treatment on individuals
  • developmental/clinical psychology

• Validation study 有效性研究

  • assessing the quality of an measurement instrument
    • surveys
    • questionnaires
    • tests
  • statistical analysis
  • psychometrics, sociometrics