Elements of the scientific method
Elements of the scientific method
The Scientific Method: The Gold Standard for Establishing CausalityChapter 4© 2019 McGraw-Hill Education. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or distribution without the prior written consent of McGraw-Hill EducationLearning ObjectivesRecall the elements of the scientific method. Explain how experiments can be used to measure treatment effects. Execute a hypothesis test concerning a treatment effect using experimental data. Construct a confidence interval for a treatment effect using experimental data. Differentiate experimental from nonexperimental data. Explain why using nonexperimental data presents challenges when trying to measure treatment effects.
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
The scientific method is a process designed to generate knowledge through the collection and analysis of experimental data.
A classic application is in medicine, where researchers run clinical trial to learn the impact of a new drug on patient’s health outcomes.
Scientific method effectively establishes causality.
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
The scientific method consists of the following six parts:
Ask a question
Do background research
Formulate a hypothesis
Conduct an experiment to test the hypothesis
Analyze the data from the experiment and draw conclusions
Communicate the findings
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method Process
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
Step 1: Ask a question. Deciding which question to ask is often motivated by interest in a particular outcome
Step 2: Do background research involves learning more about the issue surrounding the posed question. The purpose is to find information that will help identify a possible answer to the question
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
Step 3: Formulate a hypothesis involves hypothesizing a possible answer to the question.
Hypothesis
A proposed idea based on limited evidence that leads to further investigation.
Typically grounded in the background research and involves a positive statement about causality
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
Step 4: Run an experiment
Experiment
A test within a controlled environment designed to examine the validity of a hypothesis
Experimental data
Data that result from an experiment
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
For hypothesis about causality, the experiment generally involves allocating a binary treatment, or treatment levels, across two or more groups
Treatment
Something that is administered to members of at least one participating group
Treatment effect
The change in the outcome resulting from variation in the treatment
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method
Step 5: Analyze the data and draw conclusions.
Compare the measured outcomes between the group receiving the treatment and those who didn’t
Build a confidence interval for the treatment effect
Is there a causal relationship and how big is it?
Step 6: Communicate the findings. Explain the methodology and findings.
Main conclusion, a confidence level, description of the experiment, reasoning leading to the conclusion, and summary of the statistics used
‹#›
© 2019 McGraw-Hill Education.
Summaries of Scientific Method for Medicine and Business Examples
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method and Causal Inference
A Simple Treatment Framework
The basic goal when running an experiment is to measure a treatment effect
Potential outcomes framework:
Consider a group of subjects who will participate in an experiment. Index each with the letter i, so i = 1 refers to the first subject, i =2 refers to the second subject, etc.
Outcomeit is the outcome realized by the subject i if it receives the treatment t
OutcomeiNT is the outcome realized by that same person if it does not receive the treatment (NT), then:
Treatment Effecti = Outcomeit OutcomeiNT
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method and Causal Inference
The problem in trying to measure the treatment effect is that the subjects cannot be both untreated and treated at the same time
Hence, a single treatment status is chosen at the time of the experiment for any given subject
Two subjects are needed to observe the outcome of subject with treatment and the outcome of subject without treatment
The treatment effect on one subject may be different from the treatment effect on another subject.
‹#›
© 2019 McGraw-Hill Education.
The Scientific Method and Causal Inference
Since we are unable to measure treatment effects for individual subjects, we attempt to estimate the mean treatment effect for the entire population of subjects who may receive the treatment
Average treatment effect (ATE)
The average difference in the treated and untreated outcome across all subjects in a population
The expected value of the treatment effect for a randomly drawn subject from the population written as E[Treatment Effecti]:
ATE = E[Treatment Effecti] = E[OutcomeiT OutcomeiNT]
‹#›
