Coming Soon to Your Favorite Credential Program

In an Oct. 22, 2009 speech to teacher educators at Columbia University’s Teachers College, Secretary of Education Arne Duncan proclaimed that revolutionary change in the education of teachers is essential if we are to solve the problems facing U.S. society and U.S. schools. Toward the end of his speech, he declared that a standardized performance-based system for assessing teachers was key to bringing about this change.

Punctuating his speech with military and corporate metaphors, Duncan declared that the supreme purpose of public schooling is to keep America competitive, and that the decline in our standing in the world order can be attributed to the fact that the institutions that prepare teachers “are doing a mediocre job.” Duncan’s conclusion: “Teacher preparation programs need revolutionary change, not evolutionary tinkering.”

Duncan’s plan to reform teacher education takes as a given the need for corporate, top-down management, a view he shares with virtually every major corporate think tank, the Heritage Foundation, the Business Roundtable, and the Broad, Gates, and Walton Family foundations. According to Duncan, the “core mission” of teacher education programs is to bring about “substantial increases in student achievement.” He proposes to accomplish this by requiring credential candidates to pass a “performance-based” standardized exit exam that will measure both their competence to teach and the quality of their credential program (just as pupils’ scores on standardized tests are said to measure the quality of their teachers and schools). Teacher credential programs whose candidates score well on the exam would be rewarded, while those that do not would be subject to punitive sanctions. Part of the $4.3 billion in Race to the Top funds would provide resources and incentives to construct and promote these assessments.

Near the end of his Teachers College speech, he named the particular exam he had in mind—PACT (Performance Assessment for California Teachers). Few listening to Duncan’s speech had heard of PACT, but it has been a fixture in California for several years, and many teacher educators and teacher credential candidates in California are, unfortunately, quite familiar with it. This article was written in the hope that the rest of the nation might learn from and not replicate our experience.

The Lesser of Two Evils?

In 1998 the California legislature passed a law requiring teacher credential candidates to pass a state-approved high-stakes exit exam. Many credential programs in California, including San Francisco State University (where I teach in the Collegeof Education), chose PACT, devised by members of the education faculties at Stanford University, the University of California-Berkeley, and several other institutions. The alternative to PACT was CalTPA, a system designed by the state in consultation with the Educational Testing Service. Our College of Education elected to use PACT because it purported to be qualitative, not quantitative, and to assess “authentic” teaching performance in “real-world” contexts.

PACT’s creators and chief advocates claim that PACT provides the crucial link in the chain of evidence connecting the classroom performance of individuals at the end of their credential programs to achievement of their pupils in their first year of teaching, as measured by standardized tests.1 In academic lingo this is called “predictive validity.” The assumption, then, is that PACT will identify how well various credential programs promote teaching practices and learning outcomes valued by federal and state authorities.

Like all other states, California already had an elaborate set of state-mandated entrance and exit assessments for teacher education programs. In California this included the CBEST (a test of basic literacy), CSET (a standardized test of content knowledge), RICA (a test of knowledge about teaching reading), student teaching supervision, and GPA requirements. Our programs were also assessed by an increasingly prescriptive NCATE, the national accrediting body that imposes an entirely separate and largely redundant assessment system. PACT adds another layer to what is already a complex system of assessment. Each additional test throws up yet another hurdle to becoming a teacher—hurdles that disproportionately affect people of color and those for whom English is not their first language.

How Does PACT Work?

PACT assesses two types of performance: the Teaching Event (TE) and embedded signature assignments. The signature assignments are course assignments that faculty are required to implement and score according to PACT-designated and state-approved criteria, a clear intrusion into institutions’ and faculty members’ prerogative to set their own criteria for assessing students.

I will focus on the TE, a 15-minute videotaped lesson taught by a candidate during the final semester of student teaching that must be accompanied by a portfolio of approximately 50 pages of teaching plans, teaching artifacts, student work samples, written reflections, and commentaries related to the TE. The guidelines and the more than 50 prompts specifying the elements of the portfolio are laid out in detail in a student handbook. Students enter the TE data (including the video recording) into a computer program. To say that the preparation for the Teaching Event consumes an inordinate amount of time and psychic energy is a serious understatement.

Scorers, trained by official state PACT trainers in two daylong sessions and then “calibrated” annually to assure scorer consistency, assess the TE video and accompanying documentation using a series of standardized rubrics. After the rubric scores for the signature assignments and the TE are submitted, a computer program transforms them into numbers that purport to represent the effectiveness of individual credential candidates, and these are forwarded to the state. Thus, in the end, this is a quantitative, not a qualitative, assessment. If the submitted numbers are at or above the cut score, the candidates will have fulfilled the PACT criteria for earning a credential.

The scorers are anonymous and must not know the candidates they are scoring. Though the scorers may be faculty members, there is no requirement that they have any particular background in teaching or expertise in the area they are assessing. (Any trained and calibrated person is qualified for the job.) Since remuneration is a central motive for engaging in the tedious, highly regulated scoring task, and scorers are paid per head, they have an incentive to score as rapidly as possible.

Education faculty members are not considered qualified to do PACT assessments unless they have been approved by PACT trainers. One professor at a state university in Southern California, who is recognized as a national and international expert in second language acquisition, was appalled by the notion that, after only two days of training, a calibrated scorer’s judgments about candidates’ competency to teach English language learners was considered more valid than hers. She told me that, with her 30 years of teaching and research in this discipline, she resented being calibrated to rate candidates on a questionable rubric alongside scorers who may have no knowledge or experience teaching English language learners. “PACT is a death knell for the longstanding respect for academic scholarship and expertise,” she wrote.

A faculty member at another California state university wrote:

Our faculty do not want to be involved in scoring the PACT teaching events, as it is very time consuming and tedious. We may be moving to a process that is almost exclusively scored by persons who know little about and who do not teach in our program. This is very troubling. I see institutions “farming out” the assessment of PACT to regional centers to cut costs and to score the huge numbers of events that will need to be scored. In short, PACT seems to violate everything we know about designing assessment. What kind of assessment have we created that faculty who teach in the program do not want to score? 

Does PACT Assess Good Teaching?

The PACT brochure proclaims: “A candidate who passes this assessment has shown he or she is a better prepared teacher who can help students succeed.” What do the PACT authors mean by “succeed” and by “better prepared”? Better prepared than whom, and for what?

In his Teachers College address, Duncan proposed to “reward states that publicly report and link student achievement data to the institutions and programs where teachers and principals were credentialed,” and advocated “longitudinal data systems that enable states to track and compare the impact of new teachers from teacher preparation programs on student achievement over a period of years.” Since the student achievement data and longitudinal data systems Duncan refers to are inconceivable apart from students’ standardized test scores, good teachers apparently are those whose students score well on standardized tests.

The rubrics or criteria used to assign PACT scores are based on state standards and are supposed to identify elements of teaching that scientific research has shown enable students to reach these standards. Teaching patterns valued by PACT are primarily aspects of explicit, systematic, direct instruction that will instill in students knowledge specified by the state-mandated standards and measured by standardized tests. PACT cannot and does not aspire to assess teachers’ perseverance or their ability to think on their feet, hear and respond to feedback, learn from experience, listen to students, or promote student self-confidence and critical thinking.

Each rubric is intended to represent one aspect of a concise checklist designed to assess a component of teaching (e.g., instruction, planning, assessment). To achieve test-scoring consistency PACT assessors are trained (calibrated) to make similar if not identical inferences from their observations as the basis for assigning scores on each of the rubrics. Thus,those who construct the rubrics and train the calibrators hold the power over how good teaching is defined and identified.

Many of the attributes tested by the rubrics are important components of good teaching. However, if we know anything for certain about the effects of standardized testing, it’s that what’s not tested is unlikely to be taught. So what is not assessed by PACT? PACT does not assess most of the qualities that Duncan himself in his Teachers College speech lists as attributes of great teachers—namely, the ability to “literally change the course of a student’s life . . . light a lifelong curiosity, promote a desire to participate in a democracy, instill a thirst for knowledge, reduce inequality, and fight daily for social justice.”

What’s more, no one who has been specifically prohibited from interacting with a candidate personally can, on the basis of observing a videoed 15-minute teaching segment and reading supporting evidence, identify with any certainty many of the teaching attributes most of us value highly. Nor can assessors acknowledge teaching attributes and actions they might deem as valuable but have been “calibrated” to ignore.

There’s also the obvious question of whether a videotape of a small segment of instruction can actually capture authentic teaching. (“Not to mention,” a teacher educator writes, “that the quality of the videos is often poor, unless we spend hours training students’ video skills.”) Many credential candidates themselves argue that the video-recorded Teaching Event is artificial and contrived, and does not represent their real teaching. Several reported that during the TE they were preoccupied with keeping in mind all the rubrics the assessor would be bringing to bear on the 15-minute performance, and with whether the camera was picking up their students’ voices. Many credential candidates elected to plan the simplest and most technically unchallenging lessons they could think of. One reported to a faculty member that he was encouraged to teach his PACT lesson for practice the day before the videotaping.

Because of PACT timeline logistics, students usually complete lesson plans for the TE weeks before they teach it, thus discouraging contingent and learner-centered instruction. One student told me:

I spent about 15 hours writing about the lesson I was going to videotape, citing resources, spouting theory, explaining my best practices. Then the first stormy day of the year threw everything off track. It started hailing, the kids ran to the windows as branches came crashing down and garbage whipped the windows. I had never planned so hard for a lesson in all my life, and I had never had one go that badly. It went badly because, no matter how much you plan, a lot of teaching happens in the moment. A huge part of teaching, especially for beginning teachers, is finding that feeling in your gut that says ‘Step in now!’ or ‘Sudden change of plans!’

PACT’s Effect on Teacher Education

Because PACT is an unfunded mandate, credentialing institutions have to pay for the video equipment and the costs of administering and scoring the assessments. Our institution did so, in part, by reducing student teacher supervision. (In his Teachers College speech, Duncan actually claimed that America’s taxpayers already “generously support” teacher preparation programs, but this is certainly not true of ours.) Since supervision has been reduced, and students receive no feedback from PACT assessments unless they do not pass, the PACT regimen severely limits candidates’ opportunities to engage in the reflective dialogue with experienced practitioners about their teaching that is central to learning to teach.

Most of the credential candidates’ responses to PACT cited on the official PACT website are positive. However, almost all of the students at San Francisco State and at least four other California state universities found PACT a serious and significant distraction from their course work and student teaching, creating unnecessary anxiety and exhaustion as they tried to satisfy the requirements of what they perceived as repetitive, bureaucratized tasks. The first half of student teaching in the final semester has come to focus on preparing for the Teaching Event, to the consternation of many student teaching supervisors, master teachers, and the candidates themselves. Some students were absent from student teaching in order to complete the PACT documentation. One student told me, “I wasn’t really able to take student teaching seriously until I’d completed the Teaching Event.”

Many students engaged in what has been called “bureaucratic ventriloquism: an inauthentic response so markedly detached from the individual’s own beliefs, that the utterances themselves appear to be projected from elsewhere.”2 One talented candidate felt that “the teaching for PACT wasn’t coming from me.” A systematic interview study of responses from credential candidates at the University of California-Davis3 confirmed my experience that the vast majority of credential candidates had overwhelmingly negative feelings about PACT, citing excessive writing demands, the stress of assembling portfolios at the same time as student teaching, and the toll it took on their health and personal relationships. They found PACT minimally helpful regarding classroom management and instructional strategies.

At our school, preparing students for the TE has become the focus of student teaching seminars that had formerly been devoted to examining the interface of theory and practice. Lisa, who graduated in December, reported:

We sacrificed 90 percent of our third semester practicum class working on PACT, when we could have been discussing how to handle situations we were facing in student teaching. We sacrificed a lot to prove to the legislators in California, and everyone they answer to, that we were ready to teach. And of course PACT didn’t prove that at all.

Like the frog in slowly heating water that offers no resistance and eventually is boiled, it wasn’t until our department had been subjected to PACT for several years that our faculty began to recognize the effect it was having on our program and morale. We were more stressed and were working harder than ever before, not out of internal motivation, but because of the requirements PACT thrust upon us. In exchange, PACT added no information that could help us improve our credential programs, but instead diverted time and resources from programmatic commitments to prepare teachers to educate the “whole” student, promote democratic citizenship, and reduce the opportunity gap. It resulted in curriculum changes that had no justification beyond adapting to the exigencies of PACT, which was as demoralizing and disempowering to many faculty members as standardized testing has been to K-12 teachers.

Technical and logistical issues about PACT came to dominate our faculty meetings and substantive discussions, such as those about how we as a faculty should respond to the increasing focus on high-stakes testing in the schools, have virtually disappeared. One author of PACT was quoted on the official PACT website as follows: “[PACT provides a great] opportunity to talk with other faculty about expectations for candidates and what we value. A real treat to engage with faculty over substance.” Perhaps prior to PACT there had been no such discussions in this woman’s department. But before PACT our faculty did have discussions about issues of substance on a regular basis.

A student who graduated in December wrote:

I tried not to let myself question PACT’s usefulness while going through it because it would just make me angry and, no matter what, I would still have to do it. It was easier to just put the question of PACT’s usefulness out of my mind and write, and write, and write. . . . PACT was just another hoop I had to jump through in order to get my teaching credential. Viewed in the most positive light: Jumping through hoops is a useful life skill, especially if I plan on working for the state for the next 35 years.

Evidently Duncan’s ideal is that every certification program in the nation should conclude with a PACT-like assessment. If it comes to this, other certification programs will have to make accommodations similar to the ones we made.

The quality of teacher education programs certainly varies, and most teacher educators are well aware of the need for continuing improvement. However, imposing PACT, a system with no record of success, will surely have the same withering effects upon teacher education that high-stakes standardized testing has had on K-12 teaching.

If Duncan has his way there will soon be no “old-timers” who can remember when teachers and teacher educators were respected as competent professionals whose role was to promote curricula and pedagogies that are relevant to time and place, and to contribute to the construction of a more just and joyful world. Perhaps as the consequences of the “accountability movement” in teacher education become even more onerous, active resistance among teacher educators will grow. We cannot, however, go it alone. We need the help of K-12 teachers and university teachers across the disciplines, and popular support as well.

Endnotes:

1 Pecheone, Raymond l., and Ruth R. Chung. “Evidence in Teacher Education: The Performance Assessment for California Teachers (PACT).” Journal of Teacher Education 57.1 (2006) 22-36.

2 Rennert-Ariev, Peter. “The Hidden Curriculum of Performance-Based Teacher Education.” Teachers College Record 110.1 (2008) 105–38.

3 Okhremtchouk, Irina, et al. “Voices of Pre-Service Teachers: Perspectives on the Performance Assessment for California Teachers (PACT).” Issues in Teacher Education 28.1(2009) 39-62.