Difference in Differences Analysis

Using Data to Analyze the Effects of Policies

Policies are frequently implemented in numerous areas, including government, educational districts, and corporate sectors. Since policies often require significant funding, policymakers want to know if policies are working. There are a few methods generally used to estimate effects of current or new policies. A difference in differences analysis is one such method.

Difference in Differences

Brief Description: For analyses, observed outcomes of subjects for a treatment group and a control group are needed for both before and after a new policy was implemented. One reason to use this analysis instead of a simple first difference analysis is that this analysis accounts for variables that would otherwise be missed and likely lead to false conclusions. This analysis seems simple when reading the following examples, but it is statistically sophisticated and complex.

Example in Education:

In an effort to increase achievement on 3rd-grade end-of-year reading assessments, two Superintendents of Schools for local districts decide to collaborate to discover if a new reading program will aid students in learning to read.

1. Before the intervention, 3rd-grade assessment scores for each district are compared to find a pre-intervention difference.

2. After the intervention, 3rd-grade assessment scores for each district are compared to find a post-intervention difference.

3. The pre-intervention difference and post-intervention difference are compared, known as the difference in differences.

Example in Business:

Recently, there has been a lot of discussion about raising minimum wage and if that would result in a decrease in employment. Suppose that the minimum wage in Georgia rose from $7.25 to $15.00 while the minimum wage in South Carolina remained $7.25.

1. Before the intervention, the FTEs (full-time equivalent) for each state are compared to find a pre-intervention difference

2. After the intervention, the FTEs for each state are compared to find a post-intervention difference.

3. The pre-intervention difference and post-intervention difference are compared, known as the difference in differences

Note: Card and Krueger (1994) studied minimum wage’s impact on employment for New Jersey and Pennsylvania.