Letters to the Editors 25.1

PACT: Intrusion or Opportunity to Learn?

I just read Ann Berlak’s piece, “Coming Soon to Your Favorite Credential Program: National Exit Exams”(Summer 2010). Professor Berlak makes substantive arguments about the dangers of standardized assessment—including the narrowing of the curriculum and the reduction of the immense complexity of teaching and learning to dimensions that can most readily be measured. And if you are one of the dwindling number of Americans who are happy with the way we currently prepare teachers, you may agree with Berlak and needn’t read on.

But I am not happy with how we prepare teachers. I say this as a well-scarred veteran, like Berlak, of several decades of work as a teacher educator. I love the work—and hold deep respect for the people who do it. My dissatisfaction is not personal, it’s systemic. It derives from our collective inability to respond to issues we know a good deal about, if only because we have been (not) dealing with them for decades. For readers skeptical about this assertion, I recommend the decades-long succession of reports on the “state of teacher education” from Conant (1963), Sarason (1993), Goodlad, Soder, and Sirotnik (1990), or most recently Levine (2006).

I have had three significant experiences with the Performance Assessment for California Teachers (PACT). Two involve using the instrument in teacher education programs I have directed—at the University of California, Santa Barbara and here at the University of Washington. The third is an interview study Morva McDonald and I did with 24 of the 32 PACT programs in California (McDonald and Peck, 2009). Based on these experiences I offer the following observations:

• PACT data provide a measure of what preservice teachers actually can do with the students they are teaching in the public schools. Although the instrument is not perfect (as its authors readily and regularly admit) it is hugely more useful than the “scantron” assessments of teacher knowledge and skill that dominate the policy landscape of teacher preparation.
• In my experience, PACT data have pushed back hard on faculty assumptions about what candidates are actually taking from courses and implementing in the classroom. To their enormous credit, faculty have been vigorous and proactive in changing what they are doing in response to this feedback. Nowhere has the feedback been more powerful than in reflecting our (in)effectiveness in preparing new teachers to understand and respond to the needs of students who are learning English, who have disabilities, and/or are members of cultural communities that are unfamiliar to many of our candidates.
• Although some candidates do indeed lament the work involved in this demanding assessment of their teaching practice, others value the experience highly as a learning opportunity: “The assessment portion as well as the teaching portion allowed me to see my own teaching and the results of my teaching. It was great to have hands-on material to work with and to use to improve my teaching.”
• PACT training (“calibration”) is not about disrespecting faculty expertise, but about getting them to apply a consistent standard in making professional judgments that have tremendous consequences in the lives of teacher candidates. In my experience, this has not displaced faculty prerogatives about evaluation in coursework, nor has it displaced the sophisticated clinical judgment often exercised by university field supervisors.

In the brutal context of contemporary public disrespect for the work of teaching and teacher education, PACT represents the most significant attempt to date by teacher educators to take control over the evaluation of our own profession. Our collective abdication of this responsibility will set a rich table for profit-oriented test companies, whose contracts are made with policy makers and bureaucrats rather than teacher educators. Although concerns Berlak and others have raised about PACT are not trivial, they misjudge the importance of the opportunity that PACT represents to move preservice teacher assessment out of its trajectory toward fill-in-the-bubble testing and into the classroom.

—Charles “Cap” PeckProfessor of Special Education University of Washington Seattle

Parsing PACT

Although it is true that the state-mandated PACT has had a profound impact on credential programs, the aim of this response is to address the claims made about the multiple subject credential (MSC) program at San Francisco State in Ann Berlak’s article on PACT.

• In reference to individuals who score the PACT teaching events, the claim that “there is no requirement that they have any particular background in teaching or expertise in the area they are assessing” is not true. Contrary to this, we select individuals with a teaching background. The author was invited to attend a training to become a scorer and, by choice, did not complete the training.
• Scoring a PACT teaching event is a long, involved process. Thus, the claim that individual scorers “have an incentive to score as rapidly as possible” due to the compensation is not true. In fact, we limit scorers to a certain number of events.
• “Education faculty members are not considered qualified to do PACT assessments unless they have been approved by PACT trainers” is also not true. PACT has an established training and calibration protocol whereby individuals become qualified scorers. Due to their expertise, education faculty members are encouraged to become scorers.
• Although it is true that the PACT is an unfunded, state-mandated assessment, and that our MSC program struggles to meet the requirements of this legislation, it is not true that our program is funding the PACT by “reducing student teacher supervision.” All teacher candidates are supervised over the course of two semesters. The high cost of supervision in the field is a challenge for credential programs in general.

I cannot allow unfounded claims about our MSC program at SF State to be published without response. High-stakes testing places a burden on teacher preparation programs that is similar to its impact in the public schools. However, the present reality is that our candidates need to pass the PACT, in addition to other evaluative components, to apply for a credential. Our mission is to help them reach their goal.

—Debra Luna Chair, Department of Elementary Education San Francisco State University
San Francisco

Ann Berlak responds:

Like many of us in the field, Cap Peck would like to protect teachers and teacher education from public disrespect. He seems to think that PACT will help us accomplish this feat. If only that were so. However, if the plague of standardized K-12 testing has taught us anything, it’s that standardized testing is far more likely to be used to control and degrade educational institutions and teachers. As the Race to the Top so clearly demonstrates, it’s likely that under a PACT regime teacher education programs would be ranked, rewarded, and punished in terms of their students’ scores on PACT.

The key question is whether PACT scores accurately and objectively measure quality teaching. That PACT assessments are neither reliable nor valid is certain to become widely apparent in the next decade.

Peck claims PACT has not displaced clinical judgment of university field supervisors. It’s hard to reconcile this with the fact that too-low PACT scores prevent candidates from receiving credentials, regardless of their supervisors’, teachers’, and mentors’ evaluations.

Contrary to Peck’s assumption, I am not happy with the state of teacher education, in part because of the profession’s lukewarm commitment to promoting critical thinking, social justice, and empowerment. These goals are peripheral to PACT. But I do not advocate making these goals the focus of a high-stakes exit exam based on rubrics constructed by experts. Instead, we need an assessment process that promotes democratic empowerment for students, teachers, and diverse communities.

Debra Luna, who is the chair of the department where I teach, read my article as a critique of the credential program at San Francisco State. Nothing could be further from the truth. I have great respect for the program. I wrote the article because I feared that PACT would be going national and I wanted to share my perspective on how it has been experienced by a number of credential candidates and faculty across the state. As a recent article about PACT in Education Week (“State Group Piloting Teacher Prelicensing Exam,” Sept. 1) attests, my fear was not unwarranted.

It is true, as Luna claims, that all assessors have backgrounds in teaching, but they do not necessarily have particular expertise in areas they will be assessing (e.g., second language acquisition or teaching mathematics). The question of whether having a teaching background is sufficient expertise was raised by colleagues at another California university.

Luna says there is no trade-off between paying for PACT and paying for supervision. The fact that there has been no reduction in resources devoted to supervision in our program as a direct result of PACT is irrelevant. Money that could be spent on supervision—and on stipends to co-operating teachers—is being spent on administering PACT and on paying scorers to use an unreliable and invalid assessment instrument.

Once again, I want to be clear that my article was not a criticism of any individual or program. Departments of education do PACT because it, or an equally questionable instrument, is required by law.

Value Added?

Ann Berlak’s article is an important wake-up call to teacher educators, but I fear it may be too late. Here in New York, the Board of Regents has just announced a plan for entities such as Teach for America and KIPP Academy to “train” teachers who will then be eligible for state certification and master’s degrees that will be awarded directly by the state—with no university preparation. The Board of Regents simultaneously announced the need to adopt a system like PACT to serve as a high-stakes assessment required for credentials.

Seeing PACT on the horizon, I attended a session at this year’s American Association of Colleges for Teacher Education conference on “What we have learned from implementing PACT.” After a long, detailed, and exhausting account of the problems and limitations of PACT, including how it had consumed incredible amounts of time and technological resources, a question came from the audience about validity and reliability. The presenters stated that the outcomes correlated almost identically with field-based performance evaluations.

I raised my hand and asked: “So you’ve explained how expensive this has been, how much time has been invested, and that it correlates very strongly with field supervisors’ judgments. What, then, is the value added?”

The reply: “Yes, well, there is that.”

There is certainly a critical need for teacher education programs to engage in systematic assessment of candidates’ readiness to teach. Teacher educators are gatekeepers: We decide who is ready to teach. Having engaged in clinically rich, field-based, inclusive, and critical multicultural urban teacher education for more than 17 years, I do not want a high-stakes assessment system to replace the collaborative decision-making that is at the center of our practice. Each year there are candidates we cannot recommend for certification; these decisions are not made lightly and are based on extensive field observations and sophisticated criteria. How can any reductionist system replace collaborative deliberations with cooperating teachers, school principals, field supervisors, and university-based faculty on a case-by-case basis?

—Celia Oyler Director, Elementary and Secondary Inclusive Learning Programs Teachers College, Columbia University
New York City