Social Science Quantitative methods(02) - The Scientific Method

 
  • 0 views

Quantitative methods(02) - The Scientific Method

  1. Empirical Cycle

    Empirical Cycle

    graph TB
A[Observation]-->B[Hypothesis]
B-->C[experiments]
C-->D[Analysis&Result]
D-->E[Evaluation: Hypothes supported, adjusted, or rejected]
E-->A

    1. Observation

      • function: sparks idea for hypothesis(anything we want to explain):
        • pattern
        • unexpected event
        • interesting relation
      • source: not important:
        • personal
        • shared
        • imagined
        • previous research findings
      • observing relation in one or more instances

    2. Induction

      graph TB
A[mutiple observation: true in specific cases] --> B[general rule: ture in all cases ]
      • general rule or hypothesis

    3. Deduction

      • hold true in new instances
      • expectation/prediction is deduced about new observations
      • determine research setup
      • define concepts, measurement instruments, procedures, sample
      • hypothesis is transformed with deductive reasoning and specification of research setup into prediction about new observations

    4. Testing

      • data collection
      • compare data to prediction
      • statistical processing
        • descriptive: summarize
        • inferential: decide

    5. Evaluation

      • Conclusion:
        1. Results confirm prediction–> Hypothesis provisionally supported
        2. Results disconfirm prediction–> Hypothesis not automatically rejected:
          • repeat with better research setup(empirical cycle)
          • adjust hypothesis
          • reject hypothesis

  2. Confirmation/Disconfirmation

    1. Confirmation

      • Prediction confirmed
      • Hypothesis not proven but provisionally supported
      • more support–>more credence

      No scientific empirical statement can ever be proven once and for all.

    2. Disconfirmation

      • Prediction disconfirmed
        • Plausible explanations for failure:
          1. methodological issues
          2. research design
          3. instruments inappropriate
          4. variables not controlled
      • Hypothesis
        • preserved
          • reject auxiliary assumptions on research design and measurement
          • come up with a better research setup
        • adjusted
          • by adding additional clauses
          • modified hypothesis less general
        • rejected/radically adjusted
          • very rare

  3. Criteria for evaluation

    1. Reliability

      • Replicability: repetition possible

    2. Validity

      hypothesized relation accurately reflects reality

      • Construct validity
        • prerequisite
        • constructs measured/manipulated accurately
        • instruments measure/manipulate intended properties accurately
        • biggest challenges！！！
      • Internal validity
        • hypothesized causal relation
        • observed effect due to hypothesized cause
        • threatened by: plausible alternative explanations
      • External validity
        • hypothesized relation holds in general?
        • results generalize to other people,groups,environments,times?

  4. Causality

    • Hume first list the criteria:
      1. cause and effect are connected
      2. cause precedes effect
      3. cause and effect co-vary consistently
        • correlation does not imply causation!
      4. no alternative explanations
        • the most difficult one!!!
        • methodology to minimize the alternatives

  5. Internal Validity Threats

    • Participants

      • Maturation

        • alternative explanation: natural change
        • solution: control group

      • Selection

        • alternative explanation: differences in participant characteristics
        • solution: randomized selection

      • Selection by maturation

        • alternative explanation: groups systematically differ in their rate of maturation
        • selected groups become mature during the experiment
        • solution: randomized assignment to groups

    • Instruments

      • Low construct validity

        • systematic bias/measures another construct
        • prerequisite for internal validity
        • solution:
          • valid instruments
          • valid manipulation methods
          • used consistently

      • Instrumentation

        • instruments changed during the process of the study
        • solution:
          • valid instruments
          • valid manipulation methods
          • used consistently

      • Testing

        • sensitization: measurement affects behavior
        • solution:
          • special design with multiple groups with/without pretest

    • Artificiality

      expectation changes behaviors

      • Experimenter expectancy

        • Researcher changes behavior(unconsciously) biasing effect researcher’s expectations
        • influences participants’ responses
        • solution: experimenter blind design

      • Demand characteristics

        • Participant changes behavior(unconsciously)

          biasing effect participant’s expectations

        • Solution: leave participants unaware of the real purpose of the study or at least which group they are in

          • Double Blind Research Design

        • solution: cover story

          • temporary deception–>needs to be necessary
          • risk of bias–>needs to be real
          • researcher–> needs to debrief participants afterwards

    • Research Setup

      • Ambiguous temporal precedence 时间先后模糊

      • solution: manipulation of cause

      • History

        • unforeseen event during study
          • provides alternative explanation
        • large-scale
          • solution: unavoidable
        • small-scale - solution: test subjects separately, if possible

    • Mortality

      • participants dropout
        • alternative explanation: differential dropout
      • solution: hard to avoid; document reasons for dropout for further research

  6. Relevant types of variables

    1. Construct

      • denotes property in general, abstract terms
        • loneliness&depression

    2. Variable

      • operationalized, concrete expression of construct.

        • measurable/manipulable
        • UNLA loneliness scale/ GDS depression survey

      • Vary+able!!

      • Types of variables

        1. Independent variable

          • In control of
          • cause; explanatory; input; predictor

        2. Dependant variable

          • influenced by/the result of dependent
          • effect; response; outcome; output

      • Variables of disinterest

        extraneous properties

        1. Confounders/ lurking variable

          潜在变数(Lurking variable):指对研究中其他变数间的关系有重要影响,却没有被列入研究范围的变数.（要么因为此存在变量不为人知,或者它的影响被认为是可以忽略而其实不能忽略,或者是数据无法获得）

          • related to independent/dependent variables
          • partially or entirely accounts for relationship
          • solution: keeping it constant or turn it into control variable

        2. Control variables

          • likely to be related to independent/dependent variables
          • effects can be controlled for (unlike confounder)
          • solution: checking relationship at each level or value

        3. Background variable

          • not relevant in relation between independent+dependent variable
          • relevant for determining representativeness:
            • age
            • gender
            • ethnic/cultural background
            • social status
            • education level
          • solution: change it into control variables
          • used to assess generalizability