March 9, 2002

LOOK BENEATH SURFACE OF VOUCHER POLL RESULTS

Phi Delta Kappa, a professional organization for teachers, sponsors an annual public-opinion poll on a broad range of educational issues. The poll, which is conducted for PDK by the Gallup organization, is highly regarded and widely reported.

Comes now Terry Moe, a professor of political science at Stanford University and a fellow of the Hoover Institution, whose work on education is also highly regarded and widely reported. In an article in the magazine Education Next, published by Hoover, he is harshly critical of the poll's questions relating to vouchers, and in all but so many words accuses the sponsors of deliberately manipulating the questions to get the result they want and then reporting the results they do get so as to emphasize public support for their side.

As to which sides these are, PDK generally takes the view that vouchers would damage the public schools; Moe, that vouchers would improve the education children get in the public schools. I think Moe's right, but there are reasonable arguments on both sides.

PDK's report on its 33rd annual poll, written by its executive director emeritus Lowell Rose and Alec Gallup of the polling organization, is available at pdkintl.org on the Internet; Moe's article is at educationnext.org.

Until 1991, the voucher question read, ``In some nations, the government allots a certain amount of money for each child's education. The parents can then send the child to any public, parochial or private school they choose. This is called the `voucher system.' Would you like to see such an idea adopted in this country?''

By that year, support had reached 50 percent, with 39 percent opposed.

In 1993, PDK dropped that question and replaced it with this one: ``Do you favor or oppose allowing students and parents to choose a private school to attend at public expense?''

Obviously that's a loaded question, and support predictably dropped by half, to 24 percent.

The following year, PDK added another question: ``A proposal has been made that would allow parents to send their schoool-age children to any public, private, or church-related school they choose. For those parents choosing nonpublic schools, the government would pay all or part of the tuition. Would you favor or oppose this proposal in your state?'' This less tendentious wording polled 45 percent in support (and consistently higher for people who have children in public schools).

When I spoke with Rose this week, he first told me that PDK chooses the form and order of the questions but Gallup chooses the final wording. Later he acknowledged they could rewrite the question, but said that Gallup was reluctant to do so because it was a `trend'' question.

Paul Talmey, whose firm Talmey-Drake Research & Strategy does polling for the Rocky Mountain News, said the wording of the question is less important than changes to the same question over time.

``No matter how the question pushes people,'' Talmey said, ``it ought to do it consistently.''

Over time, the ``public expense'' question peaked at 44 percent support in 1997 and 1998, and fell back to 34 percent in 2001. The second question peaked at 51 percent in 1998 and 1999, and was at 44 percent in 2001.

``When support (for vouchers) was rising,'' Rose said, ``no one complained about the questions.''

Fair point.

Moe also observes that the PDK press release announcing poll results describes only the trend in answers to the less favorable ``public expense'' question. I don't see anything reprehensible about an organization highlighting poll results most congenial to its own views; everybody does that, and the full report includes historical data for both questions.

The problem is that reporters don't always read the full report, and therefore readers may never hear about it.

Finally, Moe notes, other polls asking similar but neutrally worded questions show no decline comparable to the one PDK describes. He believes that's because of other prejudicial questions asked in the poll. The decline, he says, ``is an artificial phenomenon of Phi Delta Kappa's own making.''

Whether Moe is right about why PDK's poll is the way it is, I have no idea. But I'm sure he's right about what the organization could do to make it better.