The 10 GOP candidates for the 2016 presidential election who will participate in Fox News' debate. (Credit: Fox News)

Fox News’ criteria for acceptance into its GOP presidential debate on Thursday was, as anyone who pays even scant attention to the news, based on how much support candidates enjoyed in the five most recent national polls of GOP voters – as of Tuesday afternoon, August 4.

Now there are many reasons to object to this approach, not the least of which is that the Republican Party has subcontracted its screening process to a media network. Not only that, the poll results are based on a national primary electorate that does not exist in practice (the process is a series of statewide caucuses and primaries, not a national vote).

And, of course, at this stage of the process, most voters don’t know much about the candidates and their policies, so it seems counterintuitive to confer on those same voters the responsibility of screening out some candidates and favoring others.

Nevertheless, the objections of some pollsters to this process are at best misleading, and perhaps even a bit disingenuous.

The most vocal among these is Lee Miringoff of the Marist Poll, who wrote a piece last May, presenting the “Top Ten Reasons Why Polls Should Not Be Used to Determine Eligibility for Debates.”

And then, as the Fox News announcement was approaching, he refused to poll about the standing of presidential candidates, so that his results would not be used by the network as one of the five to identify the top 10 candidates. He objected to the fact that such polls “are affecting the process they’re supposed to be measuring.”

He argued that to use polls to determine who’s ahead in the nomination contest “is a bad use of public polls. It asks public polls to have a precision that ignores margin of error.”

That latter point was supported by Patrick Murray of the Monmouth University Poll, who calculated the margin of error for each candidate’s percentage in his most recent poll, and concluded that only five candidates would definitely be in the top ten, while two would certainly not make the list. “The poll results for the remaining 10 candidates overlap when the margin of error is considered, making it clear who is truly in the top 10.” He added that “based on current polling, there’s no good rationale for arbitrarily selecting a top 10.”

Polls and the Election Process

First, let’s get rid of Miringoff’s objection that polls shouldn’t influence the “process they’re supposed to be measuring.” The literature and practical experience over the past half century is pretty clear: Polls influence the ability of candidates to raise money, attract staff, and gain media attention. This use by FOX to include candidates in a debate is simply a more transparent example of how the media pay attention to candidate standings when they cover elections.

Does Miringoff really believe that his horse race polls (along with all the other such media polls) are ignored by financial donors, potential campaign workers, and the media? If not, if he thinks the media pay attention to his polls (and why else would he publish them?), then – given his concern about the way the media use the polls – he should stop asking horserace questions not just leading up to Thursday’s debate, but permanently.

The Margin of Error Misunderstanding

In their concern that the percentages of the lower-tier candidates are all within margins of error of each other, both Miringoff and Murray overlook the sample size that Fox News said it would use. The network promised to use five quality polls, and by any measure of quality, they did (polls by FOX, Monmouth University, CBS, Quinnipiac, and Bloomberg). All use telephone interviewing (both landline and cell phone), and the organizations all report margins of error for their polls.* Given the polls they used, it is not unreasonable to treat all five as though they have a collective sample size, which FOX announced on Tuesday evening as “more than 2,400.”

The 10th place candidate was John Kasich, who received 3.2% support. (For all the percentages, see here.) Given the collective sample size, the margin of error for that figure would be 0.7%, which means his support could be as low as 2.5%.

The 11th place candidate was Rick Perry, with 1.8% support. His margin of error would be 0.5%, which means his support could be as high as 2.3% — close to Kasich’s support, but still not higher, not “overlapping” as Miringoff and Murray feared.

Conclusion: All candidates in 10th place and above have support that is greater than the 11th place candidate. There is a clear statistical distinction, with a 95% level of confidence, between candidates in the top ten and those who are not.

Problems with Margin of Error Calculations

The calculation of margin of error by Murray and by me, does not take into account any additional design features of the survey that could slightly increase the final number. But most media pollsters do not add a significant design factor, and in any case doing so wouldn’t change the principal argument significantly.

There are many in the polling field who question the calculation of a margin of error at all. The measure is based on the assumption of a good random sample, but with response rates so low, it’s difficult to argue that any telephone sample these days actually constitutes a random sample.

Nevertheless, there is a significant part of the polling industry that continues to calculate margins of error, because their samples are scientifically chosen, and because the evidence so far suggests that low response rates do not necessarily detract from the representative character of a sample.

In any case, that’s an issue for a different discussion. My concern has been with pollsters who do use margins of error, and who have (mistakenly, in my view) suggested that the FOX use of five polls to identify the top ten GOP candidates is flawed, because of the margin of error problem.

The process is flawed (again, in my view) for many other reasons, mostly of a political nature. But the two arguments discussed here – that polls shouldn’t be used to influence the electoral process, and that the average of five polls cannot reasonably be used to differentiate the top ten candidates from the rest – are hardly persuasive.

*The formula typically used to calculate a margin of error (MOE) at the 95% confidence level is:

MOE=1.96 X the square root of [(p) (1-p)]/sample size, where p = the percent support, expressed in decimal terms (p= 3.4% = .034), where 1-p = .966).