How polls structure questions can drastically affect the way people respond – and that matters

A story in The Hill, entitled “New polling confirms Democrats' left-leaning policies are out of touch,” concludes this way:

The electorate is increasingly pessimistic about the direction in which President Biden and Democrats are steering the country and feel that the party's priorities do not align with their own.

I get it.

There’s a sizable population of once-and-future Democrats who think the preferences of the left don’t fit theirs. I think a focus on racial, gender, and sexuality issues (“identity politics”) and poorly said narratives about police reform (“defund the police”) and drug treatment (harm reduction) do not connect with a lot of voters.

My intuition is based on a well-established relationship between poor or working-class people, and what they want from their leaders. Generally, those populations are concerned with economic security and strong institutions (good schools and effective policing).

But I've learned to be skeptical of polls and news reports about them.

On the one hand, polls help us understand the opinions of people who respond to pollsters’ questions. To the extent that the answers given are true reflections of people's attitudes, they are indispensable.

On the other hand, polls can sometimes give people attitudes and opinions based on how they are written. Then the interpretation of those opinions by news outlets affects the political discourse.

Let me explain.

Do polls mirror or make reality?

Suppose we have these two polling questions:

1. Agree or disagree: "we are not spending enough money on assistance to the poor."
2. Agree or disagree: "we are not spending enough money on welfare."

These are the same questions worded differently.

What do you think you are going to get?

People will agree with 1 more than 2, because it’s worded more positively. "Assistance to the poor" is positive. "Welfare" is negative.

(Indeed, research from 1989, a period in which welfare reform was an important topic, showed that about 63 percent of poll respondents said the government was spending too little on “assisting the poor,” but only 23 percent said it was spending too little on “welfare.”)

Now imagine the welfare question being picked up by a national news outlet. The story leads with “New polling confirms Democrat focus on welfare is out of touch.” Maybe a congressional aide sees the story and informs her boss that constituents reject assistance to the poor.

Has the poll mirrored reality or made reality?

The poll in The Hill story was done by Schoen Cooperman Research. In its totality, it indicates the American electorate's disapproval of Joe Biden. The electorate disapproves of the job Joe Biden is doing as president and his ability to lead an economic recovery. I think this is a rather consistent finding across many polls over the past year.

But the main takeaway from The Hill was that Biden and the Democrats need to tack to the center. Here is one of the key questions from that poll that are evidence of that conclusion:

Joe Biden and Democrats in Congress are out of touch with hard working Americans. They have been so focused on catering to the far-left wing of the party that they're ignoring Americans' day-to-day concerns, such as addressing the rising prices for goods and gasoline and combating violent crime.

Forty-three percent “strongly agreed” with this. Eighteen percent “somewhat agreed” (61 percent total). Only 23 percent “strongly disagree.” Eleven percent “somewhat disagree” (34 percent total). Five percent were not sure. Six in 10 voters believe Biden is out of touch.

But look at the question carefully.

There are at least four different claims being made.

In effect, a story is being told to the respondent:

1. "The Democrats are out of touch."
2. "The Democrats are focused on catering to the party's far-left wing."
3. "The Democrats are ignoring Americans' day-to-day concerns."
4. “The Democrats are ignoring rising prices for goods and gasoline and combating violent crime."

The poll is “explaining” why the Democrats are out of touch and the “real issues” Democrats are ignoring. The use of "hard working" Americans is also a nice touch, helping to paint the picture of Biden as disconnected from the average blue-collar worker.

Then the next question:

Do you think President Biden and the Democratic Party should move more to the left and embrace more liberal policies, move more to the center and embrace more moderate policies, or do you think President Biden and the Democratic Party should stay where it is right now?

Well, after reading an excellent story about how and why the Biden administration is catering to out of touch leftists, here is what respondents say about the question of whether Biden should pivot.

Eighteen percent of respondents said Biden should move to the left, 54 percent said he should move to the center, 13 percent said he should stay where he is and 15 percent said they were not sure.

So there you have it.

It’s hard to tell whether these questions mirror or make reality.

The questions, as they are worded and the sequence they are in, may push many voters to say they think Biden should “move to the center.” Then The Hill’s reporting pushes that narrative into public discourse.

There is a there there, but where?

Surveys show a lot about the electorate.

Over time, polls asking questions about similar topics in slightly different ways coalesce around a general conclusion we can trust.

We can be confident that many in the electorate disapprove of Biden's job as president and whether he can lead an economic recovery.

But when it comes to the claim that Democrat narratives do not align with the needs of many working-class voters, it is more complicated.

As I mentioned, I do believe there’s a there there. But we need to be skeptical of how polls are written. What would pollsters have gotten had they asked respondents to agree or disagree with this:

Joe Biden and the Democrats were stopped from providing help to hardworking Americans because of opposition from Joe Manchin, Kyrsten Sinema and all Republicans in the Senate?

My guess is you would get a different breakdown and narrative.

Indeed, within the same poll, a question brings us closer to the reality of what people really think. “Which of these issues do you feel President Joe Biden is actually most focused on? Select up to three."

Here are the top five:

1. Russia's invasion of Ukraine (34 percent)
2. The coronavirus pandemic (33 percent)
3. Infrastructure; improving roads and mass transit (18 percent)
4. The economy; creating jobs (16 percent)
5. Inflation/rising prices (16 percent)

Wait a minute!

Where’s the identity politics? The kooky lefty policies?

There’s a there there, but where?

Many voters likely believe Democrats are worried about identity politics more than Republicans. A not insignificant number are dissuaded from voting Democrat for those reasons. But saying, as The Hill claims, Democrats are seen as "out of touch" is off-base.

Polls can be tricky.

We should be skeptical of what they say and how they say it.