Tests and Standards: Will the Carrot or Stick Win Out?

Following is an interview with Monty Neill, Acting Executive Director of Fair Test. He was interviewed by Leon Lynn.

The FairTest report on state assessment systems (see article, page 27) paints a pretty bleak picture. How did things get to be so bad?

A whole series of factors contributed. First, for most of this century, starting with I.Q. tests and early achievement tests in the 1920s, the belief in the standardization and quantification of education has been real prominent. People believe you can administer standardized testing instruments to everyone, use them to gather a lot of information, and make good decisions based on that information. This is a limited and flawed view of how people learn, and the content that students should learn.

Also, I think that policymakers, and the upper class in general, have been interested in using tests as a control tool and as a sorting mechanism, when dealing with immigrants, the poor, and people of color. There is some truth to the idea that since World War II, the elites in the United States have opened themselves up to southern and eastern Europeans and Jews in ways that had not been open before, based in part on their performance on standardized achievement tests. But that has never been a role for those tests for African Americans, Latinos, Native Americans, and, to some extent, poor kids and kids from rural schools. Instead of opening the gates of opportunity, what we see instead is those tests still being used to keep those gates shut.

How so?

Because their schools are increasingly teaching to the tests, we see the tests being used to define education for those children. Even where tests are of better quality, they can’t stand in for a whole curriculum. You can’t test public speaking, for example, with a pencil and paper. And you can’t do a multiple-choice or short-answer assessment that measures deep knowledge or critical thinking skills. So if you teach to the test, you end up with a curriculum that’s a mile wide and an inch deep, which stresses memorization, recognition, and rote application. If you go out to the well-to-do suburbs, teaching is not organized around those tests. Those kids are expected to do other things.

If all students have had is a curriculum organized around those tests, they are unlikely to learn the knowledge and skills they need to do well at four-year colleges and universities. And it’s pretty much guaranteed that if they don’t go to those schools, those students are not prepared for the kinds of work that gets you into more prestigious, higher paid, socially powerful jobs and positions.

In his recent State of the Union address, President Clinton spoke highly of Chicago’s new policy of prohibiting social promotion. In Wisconsin, Gov. Tommy Thompson recently recommended the same policy. Should kids get passed on if they don’t achieve at grade level?

All the evidence shows very clearly that retention doesn’t work. Kids who are retained in a grade because they’re at the bottom of the class stay at the bottom of their new class, and then it’s much more likely they’re going to drop out. So retention doesn’t solve the problem. Simply passing kids along who don’t learn much doesn’t do them any favors either. The real solution is to fix the education program, so that the students are learning more.

Chicago is using the Iowa Tests of Basic Skills (ITBS) to determine if students should be promoted. Those tests shouldn’t be used that way. There’s too much likelihood of error to make decisions like that based on one test. What we’ve heard about in Chicago is that in many cases, they’re turning schools into test-coaching programs, coaching students to pass the ITBS. I’m not saying that’s true for every school, but it’s been a predominant response. That’s a really lousy education. And schools end up with a sort of triage mentality. They concentrate heavily on the kids who are already close to passing the test and end up doing little for the kids who aren’t close.

What’s your opinion of President Clinton’s plan for voluntary national tests in fourth-grade reading and eighth-grade math? Don’t parents have a right to know how well their child and their child’s school are doing?

First, the tests Clinton is proposing are quite redundant. Almost every state in the country has, or soon will have, state-wide tests in mathematics and reading at the elementary, middle school, and high school level. The only exception is Iowa, and almost every district in Iowa uses the Iowa Tests of Basic Skills. So kids are already being tested in these subject areas. You might not be able to equate these state tests to each other but, at the very least, they’re often comparable. Also, more than 40 states are already using the National Assessment of Educational Progress (NAEP), which means you can compare states, but not individual students, on the same test.

Second, Clinton is proposing tests that are 75% multiple choice and very short answer. The kinds of questions that might challenge a student’s ability to think deeply in a particular subject aren’t going to be asked.

Third, Clinton is assuming that everyone can agree on definitions of basic reading and math, and that’s just not true. The conservatives are afraid that the definitions will include whole language, the progressives fear the definitions will include too much rote memorization and drill-and-kill type work, and so on. There is not an agreement on what these so-called basic skills actually are.

Finally, there are no safeguards on how the tests are going to be used. Right now Clinton says the tests are just voluntary and only in two subjects. But ever since he first ran for president, he’s said that he wants full-fledged national testing. We think this is the camel’s nose under the tent. We think Clinton’s team saw that President Bush’s proposal for full-fledged national testing was defeated and decided it’d be wiser to just start with reading and math in fourth and eighth grade. And the tests may be voluntary initially, but we don’t believe these will stay voluntary very long. It could well be, for example, that in eight years the government says that to get Title I money you need to participate in the testing program.

How can teachers do performance-based assessment when they have large classes, very little planning time (especially at the elementary level), and at times students who may be disruptive?

It’s very difficult, no doubt about it. Performance assessment has to be seen as part of the whole reform picture. If a teacher’s class is too big, and all they can do, because of its size, is drill and practice, that’s a problem, it’s not quality education. Teachers, parents, and people in the community have got to organize for decent education.

Would such assessment systems be considerably more expensive than current paper and pencil tests? Given the budget crunches in public school budgets, how will that ever be achieved politically?

It’s difficult, but I think it can be done. If the procedure is to administer comprehensive assessments to all kids or score portfolios for all kids, yes, that would be horrendously expensive and never happen. But FairTest is arguing for a different model where assessment is taking place mostly in the classroom, the teacher is making the assessments based on the day-to-day work that kids do. You can do sampling for accountability, but for accountability purposes you don’t need to test every kid. It’s the same as saying that you can do a blood test with a small sample instead of draining the whole body.

In this model the teacher should be saying, “Here’s how I judge this particular student in light of the district’s standards,” and the district should sample, say, five student portfolios to make sure the teacher’s assessments are accurate.

With this model, the expense isn’t so much for testing, but instead for making the kind of classrooms we’re talking about. It’ll cost more than a straight multiple choice test, but not horrendously more. You could probably do the bulk of it for about $30 per student annually.

Vermont is beginning to do this with its portfolio program. In the early years they’ve had low reliability, and that’s a problem, no question. Scorers and teachers weren’t fully trained, they weren’t always sure what was supposed to go into portfolios, the scoring guides were flawed. Hopefully they’re improving, though certainly it takes time. Vermont wants to build a good system over time, instead of jumping quickly through hoops. They have no intention of making high-stakes decisions based on portfolios. They’re using them instead as a way to provide information for people.

Isn’t there a lot of resistance to giving teachers that much control over assessment?

Absolutely. In a lot of communities, teachers are essentially being treated as technicians who deliver instruction. Also, I think there’s a complex concern in communities of color, where people believe, and rightly so, that their kids have not been served well by schools. Sometimes this means they want more control over teachers, and that’s something teachers will need to approach with some care and thought. Teachers need to be socially responsive and responsible, negotiating with the community being served, but also functioning as professionals.

Part of the problem is that teachers’ professional development on assessment is inadequate. Most states, for example, have no required assessment training. So teachers do what had been done to them, and that isn’t always good. We also know that most schools offer very little time for teachers to work with each other around curriculum and assessment. There’s been some movement to improve professional development on assessment, but it is nowhere near extensive or deep enough.

Ironically, the standards push is making it even harder to work on professional development. Adding more demands on teaching, and putting more mandates on instructional time, has virtually eliminated recess and lunch in a lot of schools, leaving much less time for teachers to collaborate.

What advice do you have for a teacher who finds herself in a school where many staff members have bought into the ideology that for survival sake, they must “teach to the test” and, if necessary, “even teach the test?”

Where you have high-stakes tests, you simply can’t ignore them. But teaching to the test is going to deny kids the education they deserve and need in the long run. It’s like eating a candy bar before a race to get a boost of energy. A diet of candy bars won’t work in the long run. You might get a slight gain in scores, but that gain plateaus out quickly. And then the gap between those kids and the ones getting a good education, usually in the suburbs, gets wider.

But teachers are concerned with this year. They have to be. We think teachers should get students to understand the test, engage in a critical analysis of how the particular test is constructed, what the test makers want, and the best ways to give it to them. You can actually do constructive learning this way, with a focus on higher order thinking.

The solution varies from subject to subject. In some ways, the easiest place to deal with testing pressures may be reading and writing. There should be a strong reading and writing program for all kids, one that encourages students to read a lot, read well, discuss their reading in ways not reduced to short answer and multiple-choice responses, get kids to think about reading and what they read. That needs to be a foundation even in a test-driven program.

That will work somewhat less well for math, especially if all the tests are computation and traditional word problems. That really crowds out time for problem-solving, abstract thinking, mathematical communication. But getting students to understand the “why” of math is likely to help them do better on their rote work as well. The Third International Math and Science Study presents some evidence that students in Japan are doing better in math because the curriculum helps them get the “why” of math better.

Science and social studies may be the worst areas, because standardized tests often reduce learning in these subjects to vocabulary drills, memorization of facts, and dates.

You’ve been quoted elsewhere as saying that the right wing has “hijacked the standards movement in California.” What do you mean?

In California, all members of the state board of education were appointed by conservative governors. The right-wingers in the state jumped up and down and convinced the state board to toss out the standards being developed for English language arts and math, and create more conservative versions of the standards instead. It demonstrates that standards are often decided in a very political process.

The result was that the standards have more emphasis on memorization of facts and procedures, instead of communication. The standards reflect the notion, for example, that it’s important for students to be able to calculate square roots by hand. What’s the point of this? No one does this in the real world. If you don’t have access to a calculator, you’re not working in this field. But there’s this notion that we did it 40 years ago so it must be good for kids today.

In any curriculum, there’s a limited amount of time. So when you devote time to things like this, you’re taking time away from kids learning to apply math to real world problems, figuring out the why of math instead of just what to do.