At the last board meeting on Thursday, November 3rd, Dr. Robinson defended her support of the superintendent’s recommendation to sign what was at the time a $1.7 million dollar contract with Teach for America by claiming that she had spent the week reviewing studies on TFA and the effectiveness of the teachers placed in a system by TFA.
She claimed that there were both “good” and “poor” studies concerning the effectiveness of the teachers that TFA hires. In short, she was convinced that TFA’ers were more effective than traditionally certified teachers. In an email from Dr. Wardynski that Dr. Robinson shared with me, he claimed, “We are making this investment because evidence has shown that the achievement gains provided by TFA exceed the gains afforded by a similar investment in alternative strategies” (November 9, 2011).
Thus, the extra $10,000 spent training a TFA’er will be money well spent because TFA’ers will provide a “very large gain in school culture and student achievement” (November 9, 2011).
He offers no explanation of how the gains in “school culture” are to be evaluated, but the “student achievement” standard is fairly clear.
To evaluate that, we’ll need to look at the studies available to us.
“Good” and “Poor” Studies
Assuming that the Heilig and Jez study that I shared with her on November 1, 2011, was one of the “poor” studies, I approached Dr. Robinson after the meeting to ask for her thoughts about this study.
In summary, while she didn’t call Heilig and Jez’s study “poor,” she did claim that this study did not take into account significant changes that TFA made to their training program in 2009. As the study was published in June of 2010, she is correct that it does not take into consideration data after 2009. She did not offer an explanation of why TFA needed to make significant changes to their training program in 2009, nor did she offer any details concerning what changes were made to their training program in 2009.
She also claimed that the Heilig and Jez study was “supported” by NEA. On this “criticism” she is correct. The National Education Association is a supporter of the National Educational Policy Center. They are also funded by private donations to the University of Colorado Foundation, the Ford Foundation, and the Great Lakes Center for Education Research and Practice.
While it is important to know that the NEA has provided some funding for the NEPC, that does not mean that the NEA influenced the Heilig and Jez study to produce a negative report on TFA any more than The Ford Foundation influenced them to produce a positive one. (The Ford Foundation has made countless donations to Teach For America.)
Finally she claimed that the TFAers hired under this contract would be placed at secondary schools rather than at elementary schools. As such, she claimed that what we really needed to consider were TFAs performance at the secondary level.
When I asked for copies of the “good” studies that proved that TFA was putting teachers into schools who were just as effective as traditionally trained and certified teachers, she offered me two studies. In her defense, Dr. Robinson, unlike Dr. Wardynski, did proffer the links to the studies that she considered to be “good.”
Dr. Wardynski merely claims that there is “evidence [that] has shown” TFA’s achievement. For this, I thank Dr. Robinson.
“The Effect of Teach for American on Students: Findings from a National Evaluation”
The first was a 2004 study sponsored by Mathematica Policy Research, Inc. entitled, “The Effect of Teach for America on Students: Findings from a National Evaluation” by Decker, Mayer, and Glazerman. This study offered an evaluation of TFA for teachers teaching at the elementary level.
Dr. Robinson did not offer a reason why this study evaluating elementary schools in 2004 was a “good” study while the Heilig and Jez study of elementary schools in 2010 was not.
The Decker, Mayer, and Glazerman study claims that TFA’ers “produce higher student test scores than the other teachers in their schools” (16). Figure 2 (xiv) supports this claim concerning Math scores, but it does not support this claim concerning Reading scores.
In short, while TFA’ers out performed the control teachers by “10 percent of a grade equivalent” (xiv) in Math, the effect on the reading scores was negligible. In other words, the math students were almost one month ahead of their peers with control teachers.
But who exactly were these control teachers with whom the TFA’ers were competing in this “good” study? They are described as including, “traditionally certified, alternatively certified, and uncertified teachers” (xii).
So, Dr. Robinson’s first “good” study proving that TFA’ers are better teachers than traditionally certified teachers compares TFA’ers to both “alternatively certified” and even “uncertified teachers.”
So some of the best evidence available showing that TFA’ers are better show only that they are marginally better (in math) than uncertified teachers.
As Hibpshman’s State Board review of the Decker, Mayer, and Glazerman report states, “it is conceivable that the effect size [TFA’s superior performance] would altogether disappear or be reversed if they were compared to better teachers” (5).
In February and April of 2011, the Huntsville City School board of education’s reduction in force plan laid off 154 certified teachers. When Mrs. Morrison asked Mrs. Belinda Williams, HR Director, for evidence that these 154 teachers were being re-hired, Mrs. Williams replied during the November 3rd meeting that she did not have data to support that claim at that time.
“Making a Difference?: The Effects of Teach for America in High School”
The second “good” study that Dr. Robinson shared with me was by Xu, Hannaway and Taylor of the CALDER Urban Institute entitled, “Making a Difference?: The Effects of Teach for America in High School.”
This “Working Paper” from April of 2007 argues that, “The findings show that TFA teachers are more effective, as measured by student exam performance, than traditional teachers” (3). This is referred to as a “working paper” by CALDER because it has not been peer-reviewed unlike the “poor” Heilig and Jez study that I had shared with her on November 1st. CALDER themselves state that “CALDER Working Papers have not gone through final formal review and should be cited as working papers. They are intended to encourage discussion and suggestions for revision before final publication.”
While discussion is a hallmark of the educational process, it’s troubling when nearly $2 million dollar decisions, when decisions that will effect at least a third of a child’s primary and secondary education are being based on studies that have not even been reviewed.
In addition, one of the report’s primary authors, Jane Hannaway, discloses on the title page of the report that her “daughter is employed by Teach for America” (1). She does not report in what capacity her daughter is employed. Also, she does not reveal that she is “engaged in a major conceptual effort for the Gates Foundation on the design of human resource management strategies in education.”
Despite the lack of peer review, I was able to find one independent assessment of this study by the Department of Education who provide CALDER with their primary funding. The Institute of Education Sciences (IES) offer the following assessment of this study:
What did the study authors report? TFA improved student performance on standardized end-of-course tests in math and science–by about one-tenth of a standard deviation. This is equivalent to moving a student from the 50th to the 54th percentile. (1)
So again we find that the best evidence supporting spending an additional $5,000 per teacher, per year is that it increases student performance in math and science (but not in the humanities subjects) by one-tenth of a standard deviation. The best that can be said is that math students are about a month ahead of their peers.
IES went on to offer the following assessment of the “Making a Difference?” report:
Students may be placed in a course taught by a TFA teacher because of their ability in that subject – and not solely because of their general math and science ability. If so, differences in performance in TFA and non-TFA classes may be influenced by differences in student ability in specific subjects. As a result, the study may not accurately measure the effect of having a TFA teacher. In addition, the data did not link students directly to the teacher who taught their course. Instead, the study matches students to teachers based on test proctor and classroom demographics. This method is somewhat imprecise, and matching errors could lead to misleading results. (1)
So again, the best studies that those who support hiring TFA’ers at, at least, a $1.7 million dollar premium can offer are studies that “may not accurately measure the effect of having a TFA teacher.”
This is what passes for evidence with the school board. But, in their defense, this is more evidence than they usually require before they rubber stamp the superintendent’s recommendations.
Dr. Wardynski often claims that our children only get one shot at their education. If he really believes this, why is he so willing to take a $1.7 million dollar chance on unqualified teachers? At best, TFA represents a huge investment for minimal gains in math. At worst, it represents a huge investment for significant losses in student achievement.
It represents a corporate experiment where TFA’ers learn to teach on other people’s kids.
We should cancel this contract immediately.