Recommendation to the Superintendent

The Sun Beach school system conducted a comparative educational experiment. A random sample of 200 third grade students was randomly selected from the district. The students were then randomly assigned to one of two groups. Each group received a different type of supplemental help designed to prepare them for the end of grade reading test. A pretest was given to all students in the study at the beginning of the third grade year. It is now the end of the school year and you have been assigned the task of analyzing the data. Just as you begin to examine the data using some basic descriptive statistics, the Superintendent calls and demands a fast answer on the spot about which treatment did better in the study.

Using your best diplomacy, you explain that you are planning to run additional descriptive statistics, and inferential tests to determine if there are statistically significant differences between the groups. You patiently explain that you plan on testing the difference between the two groups to examine two types of potential differences: 1.) the proportion of students who made gains from pretest to posttest, and 2.) the mean gain score from pretest to posttest. Not to be deterred, the Superintendent pushes you to make a judgment about which method is better because a decision has to be made immediately about which program to expand for next year.

The only analysis you have completed at the time of the call consists of displaying the gain scores for each group in the form of boxplots. The table below summarizes the information contained in the boxplots. Note that positive values indicate the student made a gain from pre to post, and negative values indicate that the student did worse on the posttest than on the pretest.

1. Create boxplots using the information in the table above.

2. Which method, A or B, would you recommend if the Superintendent’s goal is to select the method that is related to the greater proportion of children making gains?

3. Which method, A or B, would you recommend if the Superintendent’s goal is to select the method that is related to the greater mean gain score?

Solution

If the Superintendent’s goal is to select the method that is related to the greater proportion of children making gains, then the recommendation will be Method A. It is clear that Method A has more than 75% of students making gains while Method B has less than 75% of students making gains.

If the Superintendent’s goal is to select the method that is related to the greater mean gain score, then the recommendation will be Method B. Since the boxplot for Method A appears to be skewed left, the mean gain score will be less than the median gain score of 3. Since the boxplot for Method B appears to be skewed right, then the mean gain score will be greater than the median gain score of 1.

These are the recommendations using only boxplots due to no time to investigate further using inferential procedures. While it shows the importance of always completing exploratory data analyses before looking at more rigorous statistical procedures, it is definitely not ideal to make a decision based solely on this analysis. Also, this example also demonstrates the importance of designing a better study to ensure stronger conclusions.