Standards So Low a Caveman Could Meet Them

If 100 cavemen wanted to become high school mathematics teachers, how many could pass the licensure test? The answer appears below.

Teachers in core subject areas are required by the No Child Left Behind act to prove they know the subject they are supposed to teach. NCLB gives broad guidelines as to what constitutes proof, but the details are left to the states. Most states require their new teachers to take a licensure test in the content area they plan to teach. Score above the state-defined cut-score on the appropriate licensure test and you have met your burden of proof.

How high to set these cut-scores is subject to debate. What is not debatable is that examinees with zero relevant content knowledge should not be able to pass. No matter how good your teaching skills — “You can’t teach what you don’t know, anymore than you can come back from where you ain’t been.” [Will Rogers]

For secondary mathematics teachers, the Praxis II (10061) test is currently used by a majority of states for the purpose of proving sufficient mathematics content knowledge. The cut-scores vary widely.

I showed in a previous post that Colorado’s requirement was approximately equivalent to an examinee knowing 63% of the content on this high school level mathematics exam, whereas Arkansas’ standard is approximately equivalent to knowing just 20% of the content. Such extreme variation is already an indication that something is very wrong with how these state standards are set.

I say “approximately equivalent” because this equivalency assumes that the examinee takes the test only one time and has just average luck guessing on those questions he doesn’t know how to solve. However, in the real world, examinees who miss their state’s cut-off score can take the test an unlimited number of times. They are also encouraged to guess by a test format that does not penalize for incorrect answers. This situation makes it possible for examinees of much lower ability to (eventually) pass.

We can calculate the probability that an examinee with a certain true ability level will pass in one or more attempts. The examinee’s true ability level gives the percentage of questions they know how to solve. This is the score they would get on a constructed response exam, that is an exam with no answer choices. On an exam with four answer choices per problem, like the Praxis II, an examinee will correctly answer this percentage of questions plus, with just average luck at guessing, a fourth of the remaining questions. However, some examinees will have above average luck as seen in the table below.

Probability of Passing the Praxis II in Arkansas

True Ability Level	Probability of Passing in One Attempt	Probability of Passing in Ten Attempts
0%	1.4%	13%
4%	3.7%	32%
8%	9.0%	61%
12%	19.0%	89%
16%	35.1%	99%
20%	56.0%	≈100%
24%	76.9%	≈100%
40%	100.0%	100%
Table 1. Probability of passing the mathematics licensure test in Arkansas for various true ability levels.

An examinee with a true ability level of 20% has a better than even chance of passing on the first attempt and is all but certain to pass in a few attempts. In this sense, the Arkansas standard is approximately equivalent to knowing 20% of the material (red row). This is an extraordinarily low standard given the content of this exam. (It is sometimes misreported as 40% because this standard requires correctly answering about 20 of the 50 questions. However an examinee that knew how to solve just 10 problems would average another 10 correct by guessing on the remaining 40. He answered 40% correctly, but only knew how to solve 20%).

However, with a some luck, examinees with absolutely no relevant content knowledge can pass (blue row). If 100 cavemen were to take this exam, up to ten times each, about 13 would pass. We are not talking about the brutish-looking, but culturally sophisticated cavemen of the Geico commercial. We are talking about cavemen whose relevant content knowledge is limited to the ability to fill in exactly one circle per test question.

Now presumably such zero-content-knowledge examinees would never have graduated college. Yet the fact that the standards are set this low says that some people of very low ability must be managing to satisfy all the additional requirements and enter teaching.

Such extraordinarily low standards make a joke of NCLB’s highly qualified teacher requirements. They also make a joke out of teaching as a profession and are a slap in the face to all those teachers who could meet much higher standards.

Only teaching shows this enormous variation and objectively low standards. (Even Colorado’s 63% would still be an ‘F’ or at best a ‘D’ if the Praxis II were graded like a final exam.) In contrast, architects, certified public accountants, professional engineers, land surveyors, physical therapists, and registered nurses are held to the same relatively high passing standards regardless of what state they are in.

How is it that these other professions can set arguably objective standards, while teachers cannot? The standards in other professions are set by professional societies. Their decisions are moderated by several concerns including the possibility of having members sued for malpractice.

For teachers, the standards are first set by a group of professionals, but their recommendations can be overridden by state educrats. The educrats are concerned largely with having an adequate supply of teachers. The entire process lacks any transparency, so we cannot tell the extent to which the educrats substituted their concerns for the professionals’ judgment about standards for teaching.

In shortage areas, like mathematics, low standards guarantee adequate supply. It’s a lot less trouble for the educrats to simply lower standards than to pro-actively change incentives so that more academically able people might consider teaching math.

Something like NCLB’s requirement that teachers prove they have sufficient subject matter content knowledge is clearly needed to prevent cavemen from teaching our kids math, but the Feds trust in the states to set these standards is not justified. Under NCLB, the perfect scorer and the lucky caveman are both indistinguishably “highly qualified.” Setting higher standards would force states to begin to face the elephant in the room: Not enough is being done to attract mathematically talented people into teaching.