Friday, August 04, 2006

Save the Data

In my previous posts I’ve presented evidence for how much (or really how little) mathematics our secondary math teachers need to know to be annointed “highly qualfied”. The Reader’s Digest version: On the Praxis II test, a test whose content is at the advanced high school level, teachers can gain “highly qualified” status even if they miss 50% of the questions. In some states the miss rate climbs above 70% all the way to 80%. If this test were graded like a typical high school exam, about 4 out of 5 of the prospective teachers would fail.

In this post I will look at a related question: “How Much Math Should Math Teachers Know?”; that is, what evidence is there for a correlation between teacher math knowledge and student math achievement? I touched on this topic briefly in my previous post. Let’s look at some details.

The bottom line here is that we don't know. The research is largely uninformative. In a 2001 review of research entitled &ldquoTeacher Preparation Research: Current Knowledge, Gaps and Recommendations”, Wilson et. al. state:
We reviewed no research that directly assessed prospective teachers’ subject matter knowledge and then evaluated the relationship between teacher subject matter preparation and student learning.
They reviewed no such studies, because no large-scale studies of this type existed. An opportunity was missed with the TIMSS study. In a previous post, I wondered why the TIMSS study didn’t also test the teachers. Such a study could have been quite informative. If it showed a significant difference in subject matter knowledge between U.S. teachers and teachers from countries with superior student results, then teacher preparation should get more attention. If not, then we can primarily look elsewhere for solutions. Both the magnitude of any differences in teacher knowledge and its possible correlation with student achievement would be of interest. When a very small study was done of Chinese versus U.S. elementary teachers, huge differences were found.

Studies of the effect of teachers’ math knowledge use indirect proxies for teachers’ math knowledge. The typical proxies used in these studies are based on the teachers exposure to college level math. For example, did they have a major or minor?; or simply how many college math courses did they take. It was plausible that math majors would be better at high school level math than others. If so this would be a reasonable proxy.

The data says something different. My analysis of teacher testing results revealed the surprising fact that math and math education majors do not exhibit mastery of high school level math. Nor do they do any better than other technical majors on the Praxis II. That means the proxies are poor. The minimal or non-existent correlation shown by the studies Wilson reviewed is therefore entirely consistent with my teacher testing data, even if a strong correlation exists between teacher math mastery and student achievement.

Wilson makes similar observations:
The research that does exist is limited and, in some cases, the results are contradictory. The conclusions of these few studies are provocative because they undermine the certainty often expressed about the strong link between college study of a subject matter area and teacher quality. ...

But, contrary to the popular belief that more study of subject matter (e.g., through an academic major) is always better, there is some indication from research that teachers do acquire subject matter knowledge from various sources, including subject-specific academic coursework (some kinds of subject-specific methods courses accomplish the goal). There is little definitive research on this question. Much more research needs to be done before strong conclusions can be drawn on the kinds or amount of subject matter preparation that best equip prospective teachers for classroom practice.

Some researchers have found serious problems with the typical subject matter knowledge of preservice teachers, even of those who have completed majors in academic disciplines. In mathematics, for example, while preservice teachers’ knowledge of procedures and rules may be sound, their reasoning skills and knowledge of concepts is often weak. Lacking full understanding of fundamental aspects of the subject matter impedes good teaching, especially given the high standards called for in current reforms. Research suggests that changes in teachers’ subject matter preparation may be needed, and that the solution is more complicated than simply requiring a major or more subject matter courses. [emphasis added]
Requiring a math or math education major, as some states do, is no guarantee of mathematical mastery. There is no control over the quality of the courses, or the reliability of the grades. There is no quantitative measure of how much was learned. Even if there was, it is debatable to what extent exposure to college level course work correlates with mastery of high school level math. (In my study, math majors had a mean score that was essentially at the minimal ability level. This level is almost 40 points, on a 100 point scale, below what I would call mastery.) Teacher licensure tests could provide a more reliable direct measurement of that mastery.

Without clear and convincing evidence, the interpretation of studies is subject to confirmation bias
Confirmation bias refers to a type of selective thinking whereby one tends to notice and to look for what confirms one's beliefs, and to ignore, not look for, or undervalue the relevance of what contradicts one's beliefs.
Every human being operates with both knowledge and beliefs. However, sometimes they confuse their beliefs for knowledge.

I believe that a deep, grade relevant, understanding of mathematics is essential to great mathematics teaching. I don’t think you need a math major. I do believe you also need some knowledge of how to teach, of how to control a class, of how to manage a classroom, of how to assess a student, and of how to deal with parents and administrators. I believe it takes years to acquire the necessary math skill. I believe it would take only weeks to aquire the other skill set, at least that part that can be taught in a classroom, if it were efficiently organized and if you already have decent people skills. There were great math teachers before there were schools of education, but I have yet to meet a great math teacher who doesn't know math. It also helps to have good textbooks and a rational curriculum.

As scientist I am willing to change my beliefs when presented with data. The relevant experiments are becoming easier to do, if only the data was preserved and made publicly accessible. A lot of educational research reminds me of the man that’s looking for his lost keys by the lamp post because the light is better there. Education researchers use the data that is convenient without sufficient attention to the relevancy of that data to the questions they are trying to answer. I have some sympathy for both the man and the researchers. I would probably first look where the light was good. After all, maybe the keys are there. But when you cannot find them, after a thorough search, it is time to look elsewhere.

Some 36 states now use the Praxis II to test prospective mathematics teachers. The questions on this exam go through an elaborate vetting process (see here, 90 page PDF). Unfortunately, most of the richness of this data set is discarded. What is preserved is a pass/fail decision, the criteria for which varies enormously from state to state. That’s not good enough.

Save the data!

6 comments:

Darren said...

As a math teacher myself, and one with a blog to boot, I'm definitely linking to this post.

Euclid said...

I agree Darren; I’m linking to this also.

One comment I will leave is a couple of years ago we hired a young teacher that did not get the highest grades in her undergraduate course work. We knew however that she was motivated to become a math teacher. We gave her a chance teaching 9th and 10th grade courses and she just couldn’t do it. Granted the classroom management got in her way but I think the most troubling fault was lack of daily preparation. She would admit she couldn’t do the work she assigned and the students where merciless (rightly so). I worked with her as a mentor constantly but she would not do her HW. Because she couldn’t? Getting B’s and C’ and an occasional D in college level mathematics limited her background knowledge so she wasn’t flexible enough when kids where thinking about the math differently (but correctly).

Is it all about Jamie Escalante’s “Ganas”? I think she could have done it with more desire. I know my first year teaching I reached a point that I had to decide whether I was going to do what was needed to stay in the classroom. Returning to lumberyard work (or worse flipping burgers) just wasn’t an option.

I’ve always said we learn a lot of math teaching it.

Rambling Teacher said...

Hi Dr. P.,
This is quite an interesting post. In Australia, we have only recently had such a process instituted by the AAMT (equivalent of the NCTM). Becoming a Highly Accomplished Mathematics Teacher carries no rewards, financial or otherwise. It is available to experienced teachers who can submit a portfolio showing their capabilities as educators as well as mathematicians.
The idea that research is lacking on what constitutes high qualification (or accomplishment as the AAMT puts it) is quite intriguing. One would have thought that such questions would be paramount in the minds of researchers and of funding bodies.

Elias.

Dr. P. said...

Unclemath - you touch on an important point. It is also important for teachers to know how to teach math concepts, especially to students who struggle with math.

On this topic Liping Ma's book"Knowing and Teaching Elementary Mathematics" might be of help

jonathan said...

It almost seems that you are taking the lack of evidence to be evidence in and of itself. We don't think that taking an extra grad course in math makes someone a worse teacher, right? I do believe that my college coursework has strengthened my teaching in both direct and indirect ways.

I also looked at your writing about Praxis cut scores. You hint at the force driving this stuff, but I didn't see it mentioned explicitly: there are not enough math teachers and potential math teachers out there with good math backgrounds.

I offer these comments in context: you are hitting important issues related to teaching math. I am linking to this blog.

jd2718

Krishna said...

HI,

There is a debate about how much mathematics is sufficient to a high school teacher. Some people argue that the advance matheamtics course taken in college has nothing to do with teaching high school matheamtics. Some argue that advance level mathemathics is very helpful in may ways, in instruction, in checking assignments ( quickly), in testing etc. I wonder does any one in this group has some examples that the mathematics that you have done in your college actually is helping you to enhance your teaching in any way? I would appreaciate your posting.

punamdhital@yahoo.com