Robert Mueller in 2012 (Credit: White House/Wikipedia)

A recent Navigator Research poll finds that most Americans don’t know that the Mueller investigation has uncovered crimes. Robert Mueller and his team have secured 17 indictments and five guilty pleas thus far, but the poll claims 59% of Americans believe the investigation has not uncovered any crimes, while 41% say it has. (This poll was conducted before the latest news that Mueller has accused Paul Manafort of attempted witness tampering.)

That’s at least how the media have portrayed the results (for other media reports on the same poll, see NBC News, the Hill and Daily Kos, among others).

What these reports did not include was another finding in the poll suggesting an even split among Americans on this same issue: 51% saying the investigation has not uncovered any crimes, with 49% saying it has.

How could the same poll come up with two different findings?

Split Sample Experiment

The pollsters conducted a split sample experiment, with half the sample asked this question: “To the best of your knowledge, has the investigation uncovered any crimes?” This question wording produced the 59% no to 41% yes results.

It could be that many people (correctly) concluded that no crimes have thus far been uncovered (or at least reported by the Mueller team) involving Donald Trump himself. Though the question did not specify who might have committed the crimes, the investigation had earlier been described as one about “Donald Trump’s 2016 presidential campaign.” This may have led some respondents to think the question was about uncovering crimes committed by Trump.

In anticipating such a problem, the pollsters asked the other half sample of respondents a more precise version: “To the best of your knowledge, has the investigation uncovered any crimes by associates and advisors to Donald Trump?” This question found the 51% no to 49% yes results.

It’s worthwhile noting that while the media could have chosen to report either set of results, or even both of them, all the media outlets cited above chose to go with the more sensational finding that a clear majority of Americans believe no crime has been uncovered. It simply wasn’t as interesting, despite being equally valid, that the public was split down the middle.

Yet, the second finding is probably the more plausible. When that second question wording made it clear it was asking not about Trump per se, but about his associates, more people said crimes had been uncovered.

For Democrats, the second question found 76% saying yes, compared with the first question that found 64% — a 12 point increase. For independents, the comparable figures are 48% compared with 38% — a 10 point increase. Republicans, however, showed little change – 17% when the question focused on Trump’s associates, not much different from the 15% more general question.

Even these results from the second question, however, suggest that a lot of people – at least half of the public – are unaware of the indictments and guilty pleas already obtained during the investigation.

But maybe not! The wording is so vague, we can’t be sure how everyone interpreted what the question was really asking.

Fuzzy Question Wording

First of all, it’s important to remember that indictments do not necessarily mean crimes have been committed, only that people have been charged with crimes. So, some respondents in the poll could have dismissed the 17 indictments as evidence of crime.

Second, some respondents may have reasoned that guilty pleas do not necessarily mean there is any crime. It could be the people who were charged with a crime were destitute and could not afford to defend themselves in court, and thus pleaded to lesser charges just to avoid the expense of a trial.

Moreover, in the view of some respondents, people charged with a crime could even have been falsely charged, as part of the “witch hunt” hoax that Trump has alleged and that many of his supporters believe. So, even though some associates pleaded guilty, they were – according to this view – railroaded by a corrupt FBI.

On the other side of the debate, among respondents who are critical of Trump, many could believe that, yes, crimes – even by Trump – have been uncovered by the Mueller investigation, although Mueller has yet to reveal all his findings to the public.

As these speculations make clear, the problem with the poll question is that it refers to “crime” rather than to actual court findings.

To avoid misinterpretation of what the poll is asking, the question could have been worded more precisely. Here is one such version: Do you believe that the Mueller investigation has – or has not – produced any official guilty pleas, or court findings of guilt, involving Donald Trump’s associates and advisors?

That formulation is very specific. It doesn’t require the respondents to make some interpretation of whether any guilty pleas or indictments mean there is “crime,” or whether there is uncovered crime that has yet to be made public.

The question is much more factual, avoiding the word “crime” and instead asking specifically about court findings. People who are aware of what is happening would have to acknowledge that there have been guilty pleas, even if they believe no crime was committed.

Uncertainty Among the Public

Finally, neither of the questions in the split sample experiment provided a “don’t know” option, thus pressuring respondents to come up with an opinion – or wild guess – regardless of how closely they had been following the investigation.

An earlier question in the poll reveals that overall, about one in five respondents admitted having heard little to nothing about the investigation. Yet, somehow, the poll showed 100% of respondents expressing a view – either that crimes had been uncovered, or they had not. Weirdly, the poll reported no one saying “I don’t know.”

By pressuring respondents to come up with an opinion, the poll no doubt underestimates the actual number of people who really don’t know about the indictments and guilty pleas.

When forced to come up with an opinion about uncovered crimes, even though they may have had no clue one way or the other, many respondents probably just relied on their party affiliation. Democrats would say yes, Republicans no, with independents perhaps randomly falling equally between yes and no.

In any case, the poll results don’t help us much in determining how many people are unaware of what the Mueller investigation has actually produced.

Fuzzy wording and forced-choice question formats are hardly the best approach to measuring public opinion.