Elements of the scientific method

The Scientific Method: The Gold Standard for Establishing CausalityChapter 4© 2019 McGraw-Hill Education. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or distribution without the prior written consent of McGraw-Hill EducationLearning ObjectivesRecall the elements of the scientific method. Explain how experiments can be used to measure treatment effects. Execute a hypothesis test concerning a treatment effect using experimental data. Construct a confidence interval for a treatment effect using experimental data. Differentiate experimental from nonexperimental data. Explain why using nonexperimental data presents challenges when trying to measure treatment effects.

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

The scientific method is a process designed to generate knowledge through the collection and analysis of experimental data.

A classic application is in medicine, where researchers run clinical trial to learn the impact of a new drug on patient’s health outcomes.

Scientific method effectively establishes causality.

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

The scientific method consists of the following six parts:

Ask a question

Do background research

Formulate a hypothesis

Conduct an experiment to test the hypothesis

Analyze the data from the experiment and draw conclusions

Communicate the findings

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method Process

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

Step 1: Ask a question. Deciding which question to ask is often motivated by interest in a particular outcome

Step 2: Do background research involves learning more about the issue surrounding the posed question. The purpose is to find information that will help identify a possible answer to the question

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

Step 3: Formulate a hypothesis involves hypothesizing a possible answer to the question.

Hypothesis

A proposed idea based on limited evidence that leads to further investigation.

Typically grounded in the background research and involves a positive statement about causality

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

Step 4: Run an experiment

Experiment

A test within a controlled environment designed to examine the validity of a hypothesis

Experimental data

Data that result from an experiment

 

 

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

For hypothesis about causality, the experiment generally involves allocating a binary treatment, or treatment levels, across two or more groups

Treatment

Something that is administered to members of at least one participating group

Treatment effect

The change in the outcome resulting from variation in the treatment

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method

Step 5: Analyze the data and draw conclusions.

Compare the measured outcomes between the group receiving the treatment and those who didn’t

Build a confidence interval for the treatment effect

Is there a causal relationship and how big is it?

Step 6: Communicate the findings. Explain the methodology and findings.

Main conclusion, a confidence level, description of the experiment, reasoning leading to the conclusion, and summary of the statistics used

 

‹#›

 

© 2019 McGraw-Hill Education.

Summaries of Scientific Method for Medicine and Business Examples

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method and Causal Inference

A Simple Treatment Framework

The basic goal when running an experiment is to measure a treatment effect

Potential outcomes framework:

Consider a group of subjects who will participate in an experiment. Index each with the letter i, so i = 1 refers to the first subject, i =2 refers to the second subject, etc.

Outcomeit is the outcome realized by the subject i if it receives the treatment t

OutcomeiNT is the outcome realized by that same person if it does not receive the treatment (NT), then:

Treatment Effecti = Outcomeit OutcomeiNT

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method and Causal Inference

The problem in trying to measure the treatment effect is that the subjects cannot be both untreated and treated at the same time

Hence, a single treatment status is chosen at the time of the experiment for any given subject

Two subjects are needed to observe the outcome of subject with treatment and the outcome of subject without treatment

The treatment effect on one subject may be different from the treatment effect on another subject.

 

 

 

 

 

 

‹#›

 

© 2019 McGraw-Hill Education.

The Scientific Method and Causal Inference

Since we are unable to measure treatment effects for individual subjects, we attempt to estimate the mean treatment effect for the entire population of subjects who may receive the treatment

Average treatment effect (ATE)

The average difference in the treated and untreated outcome across all subjects in a population

The expected value of the treatment effect for a randomly drawn subject from the population written as E[Treatment Effecti]:

ATE = E[Treatment Effecti] = E[OutcomeiT OutcomeiNT]

 

 

‹#›