Careful Analysis

Charles Murray has an editorial called Acid Tests. He makes the point that one needs to be very careful in interpreting the statistics that are generated in response to the NCLB act. He shows that detailed understanding of tests and test data is needed to determine whether purported gains in closing the racial achievement gap are real or artifacts. He borrows heavily from La Griffe du Lion, a website that looks at a several contemporary issues in great mathematical detail.

I am not nearly so negative on NCLB as Mr. Murray. I think some federal oversight is necessary to preserve the integrity of the data. Otherwise, as Dr. Cannell uncovered, the states will cheat, generating data that is even more worthless and deceptive than the data Mr. Murray is complaining about.

But he does make the following excellent point:

A pass percentage is a bad standard for educational progress. Conceptually, "proficiency" has no objective meaning that lends itself to a cutoff. Administratively, the NCLB penalties for failure to make adequate progress give the states powerful incentives to make progress as easy to show as possible. A pass percentage throws away valuable information, telling you whether someone got over a bar, but not how high the bar was set or by how much the bar was cleared.

The topic of NCLB and careful analysis of data is a subject I will be returning to. This time with an in-depth look at what being a "highly qualified" math teacher means. We will see that the "highly qualified" bars are set at drastically different heights depending on what state you happen to reside in. A teacher that just passes Arkansas' bar, would fall short of Colorado's bar by nearly 40 points on a 100 point scale. What sense does it make then to report on what fraction of your teachers meet the "highly qualified" requirements when the bars are set at such drastically different heights?