This blog post is listed on the NASSP blog “Principal Difference” which is created by Mel Riddle, a former Virginia high school principal and VASSP board member. It was written by Stuart Singer, author of The Algebra Miracle
If there is wisdom in the belief that important lessons can be learned from the mistakes of others, then for education the evaluation process of the Washington D.C. school system is a potential treasure trove from which to gain valuable insights. A recent Washington Post article relating the career odyssey of elementary teacher Sarah Wysocki could fill a book of “What not to do in teacher assessment”.
A quick overview:
The details of Wysocki’s truncated career at a D.C. elementary school presents a road map of the missteps possible when districts incorporate student standardized test scores in teacher evaluations. As reported in the Post:
“By the end of her second year at MacFarland Middle School, fifth-grade teacher Sarah Wysocki was coming into her own. ‘It is a pleasure to visit a classroom in which the elements of sound teaching, motivated students and a positive learning environment are so effectively combined,’ Assistant Principal Kennard Branch wrote in her May 2011 evaluation.”
The article also stated that Mr. Branch was so impressed with the work of Ms. Wysocki he told her that she should be sharing her teaching strategies with other staff members at the school. These positive remarks were not unique for the second-year teacher. Several other classroom observations resulted in equally favorable comments and the president of the school’s PTSA described her as “One of the best teachers I’ve ever come in contact with.” And yet despite these glowing reviews, at the end of her second year at the school Ms. Wysocki was fired.
Her removal from the D.C. Public School system was the direct result of the performance of her students on English and math standardized tests. The dismissal was not based on the precise scores but rather on their failure to improve to a level mandated by a complicated “value-added” algorithm utilized by the district in its “IMPACT” teacher evaluation program.
The concept of value-added did not begin in the classroom. It was originally developed to study the productivity of turning strawberries into jam. Just as comparing apples to oranges is rarely valid, using an analysis designed for strawberries to also measure effective teaching can be equally flawed. A biology teacher once lamented to me that her advanced students always complained that their results on labs were never as accurate as those in physics. Her explanation was simple—living organisms just are not as predictable as mechanical ones. And there are few living organisms as complex and unpredictable as adolescents.
Sometimes math does not add up.
This approach to assessing teacher performance has its detractors. The IMPACT evaluation system is complicated at best. The Post article adds:
“Yet even researchers and educators who support value-added caution that it can, in essence, be overvalued. Test results are too vulnerable to conditions outside a teacher’s control, some experts say, to count so heavily in a high-stakes evaluation. Poverty, learning disabilities and random testing day incidents such as illness, crime or a family emergency can skew scores.”
Unfortunately the educational decision makers in D.C. do not agree. For Ms. Wysocki these exam results constituted 50% of her evaluation. Glowing classroom observations were worth 35% and a category described as collaboration with the school community was the remaining 15%.
One misstep followed by many others.
While placing that much emphasis on test scores may be excessive, there are other fundamental flaws in the evaluation and termination process in Washington that should serve as lessons for other districts. In order to ensure the validity of a data analysis it is critical to utilize as large a group of statistics as possible. When a sampling is small too many faulty conclusions can be formed. For example, if one flipped a coin three times and it always landed on heads, it would be erroneous to surmise that a discernible pattern could be established. In Washington DC the value-added evaluation process is currently being employed on the work of only 12% of the teachers. Like every other school district this is the result of the limited number of classes that are actually administered standardized tests. Consequently seven out of every eight educators in the D.C. system are not being subjected to this process. In real numbers that percentage would translate into a faculty of fifty having only six individuals using the value added component of the system. Determining the accuracy of this process within such a limited scope should be considered highly suspect.
An inadequate sampling was only the beginning of the nightmarish assessment of Ms. Wysocki’s work. Currently 41 schools in Washington are being investigated for testing fraud in 2010. These inquiries resulted from the contention that student scores were inflated as a result of teacher and administration malfeasance. Fourteen of the 25 students in Ms. Wysocki’s class came from a school in that questionable group. Consequently, there is the strong possibility that measuring many of her students’ “improvement” would be nearly impossible. While the system acknowledges this situation, it decided that since there had been no resolution to those complaints, it was not relevant to an evaluation. It was interesting to note that the other fifth-grade teacher at Wysocki’s school, also highly regarded by the administration and community, was fired as well.
Adding insult to injury are Wysocki’s classroom evaluations. In a four-point system where one is unsatisfactory and four represents exceeding expectations, Wysocki improved from a mediocre first year to a stellar 3.2 average in her second. In most classrooms such a score would translate into a solid “B”. This significant improvement was deemed insufficient to warrant a third year.
A better method:
There is a gnawing sense that evaluation and termination systems like that employed by the Washington D.C. Public Schools are actually designed to overly simplify the process for administrators. Secure a group of numbers—previous test scores, current test scores, a few demographics—hire someone to create an equation, plug in the data and firing decisions are almost exclusively based on the two-digit result. An evaluation generated by machines is not an appropriate method to assess 50% of the work of human beings interacting with other human beings.
The education leaders in D.C. dismiss the value of school-based evaluations because they may reflect personal bias on the part of the observers. This is a valid point that can be eliminated by hiring and training a cadre of individuals whose sole responsibility would be to perform assessments of educators system-wide. These evaluators would assume the duties currently being undertaken by administrators within a building.
The incorporation of student test results in teacher evaluations should continue. As long as these exams are utilized as a measure of academic achievement and eventually to graduation they need to be a factor in judging the work an educator. But giving these numbers the dominate role in firing is a risky proposition as demonstrated by the story of Ms Wysocki. A more rational approach would be to give this responsibility to people who have been trained in classroom observations, interpeting data, and who are not distracted by other activities in a school building.
The ability to accurately evaluate education and to terminate the contracts of those who are underperforming are two critical issues facing education. The right approach will not be found in a number computed by a complicated equation. It will evolve from a commitment to provide the financial and human resources necessary to create full-time professional evaluators.
