Putting a right wing poll of Washington State under the microscope: An evaluation of Echelon/Future42’s spring 2023 survey

This week, the right wing group Future42 — a “capacity group” of the right wing political infrastructure initiative Project 42 formed not long ago by former Washington Policy Center CEO Dann Mead Smith — released the results of a statewide poll it commissioned from Echelon Insights on a range of current issues.

Right wing commentators have been eagerly sharing screenshots from this survey on their Twitter accounts, while former Q13 Fox personality Brandi Kruse devoted a special episode of her podcast to highlighting a number of the findings.

Oftentimes, when a survey like this comes out from a right wing outfit, the sponsor simply doesn’t publish enough information for our team to conduct an in-depth evaluation of the survey. However, to their credit, Future42 has published what appears to be their full script and complete topline results, along with a description of their methodology. That impressed our team, so we decided to take the time to go through the survey and evaluate their work.

Here are our impressions of their survey.

How we evaluate polls

First, a few words about how we evaluate polls.

We believe that subjective organizations are perfectly capable of conducting objective research, so we won’t reflexively dismiss data from a survey merely because it comes from right wing sponsor. However, we are sticklers for the scientific method, so we will look to see if that sponsor and their pollster adhered to best practices for obtaining sound data, or whether they cut corners.

The key to accurate, credible polling is neutral questions asked of a representative sample. You’ve got to have both, or your data is going to be skewed. You can’t find out what people think if you tell them what to think first, which is why neutral questions are important. And your sample needs to properly resemble the electorate or the population of the area you’re surveying, or the findings simply aren’t going to accurately reflect what public opinion is.

It’s extremely difficult to write neutral questions and it can also be tough to build representative samples. But it can be done, as we’ve demonstrated over the past decade in our work with Public Policy Polling of North Carolina, a well-regarded pollster, and more recently also with Change Research of California.

About the Echelon/Future42 survey

According to Echelon Insights, this was a survey of 600 registered voters in the “Washington likely electorate” (meaning, it appears to be a poll of likely voters, perhaps likely 2024 general election voters, though this is not stated). 70% participated online (text-to-web) and 30% by landline (live interviews). The poll fielded from June 5th — 7th, 2023, and it has a margin of error of +/- 5.0%.

Echelon Insights has not released enough public polling to receive a letter grade from FiveThirtyEight, one of the few entities that rates pollsters. Instead, it has a provisional A/B rating. You can see its profile on FiveThirtyEight here.

The firm is based in Virginia, and is Republican-aligned.

(As you can probably guess, a Republican-aligned firm is partisan and works mainly with right wing clients. A Democratic-aligned firm like either of our pollsters is the equivalent, also partisan and works with progressive clients.)

A copy of the results file is below.

Evaluating the sample

On several fronts, Echelon Insights and Future42 did alright with their statewide sample. However, in crucial areas, we think their sample is skewed.

Here’s a scorecard of their demographics:

Let’s dive into the problematic areas.

Echelon reports that 24% of their respondents initially identified as Republican, 33% Democratic, 35% independent, 6% “something else,” and 2% don’t know / refused / unsure (Question 27). The initial percentages for Republicans and independents seem alright, but the percentage of Democratic voters is too low.

Looking at the answers to the very next question (Question 28), we can see that the “Something else / don’t know / and refused” voters are more Republican than Democratic, which means that the sample skews Republican:

QUESTION: If independent, member of another party, or unsure, ask: Which of the following statements best describes you?

[Randomize scale]

ANSWERS:

Total votes for Republicans: 38%
Total votes for Democrats: 27%

1. I almost always vote for Republican candidates: 15%
2. I vote for Republicans more often than I vote for Democrats: 23%
3. I vote for Democrats more often than I vote for Republicans: 17%
4. I almost always vote for Democratic candidates: 10%
5. I vote equally for Democrats and Republicans: 28%
6. Don’t know/Refused/Unsure: 7%

CONSOLIDATED ANSWERS:

QParty + QPartyLean

1. Total Republican+Republican-leaning Independents: 40%
2. Total Democratic+Democratic-leaning Independents: 46%
3. Total Independent: 13%

Then, in the answers to the very next question, we get confirmation of the skew: 34% say they are very/somewhat conservative and 35% say very/somewhat liberal. The number who are conservative are statistically the same as the number who are liberal? That simply doesn’t reflect Washington State’s electoral profile.

As anyone familiar with our state’s politics knows, Washington is a predominantly Democratic state. We haven’t voted for a Republican for president since 1984, a Republican for governor since 1980, or a Republican for U.S. Senate since 1994… and so on. Democrats currently hold every statewide office, have large majorities in both chambers of the Legislature, and have eight of ten U.S. House seats.

Most voters here are Democratic or lean Democratic. Underrepresenting them in a survey is a cardinal sin if you’re doing public opinion research in Washington.

In the education crosstabs, there’s also a skew.

Those with only a high school education or less are slightly over-represented, while college graduates are slightly underrepresented. Here’s a breakdown:

QUESTION: What is the highest level of education you have completed?

ANSWERS:

1. High school or less: 25% [Percentage is too high]
2. Some college: 22% [A little low]
3. Associate’s degree: 12% [A little low]
4. Bachelor’s degree: 25% [A little low]
5. Graduate degree: 15% [Percentage OK]

Education nowadays has a strong correlation with party — the more educated a voter is, the more likely they are to be a Democratic voter versus a Republican voter — so this is additional confirmation that their sample skews Republican.

On age, we see more younger voters than we’d expect to see and fewer middle-aged voters. However, this is more of a minor quibble compared to the other two categories and probably doesn’t negatively impact the toplines as much.

The bottom line? This unfortunately isn’t a properly representative sample of the Washington State electorate. It’s just too right wing.

The questions

The survey contains twenty-two topical or issue questions, grouped under several categories: Landscape, The Legislature, Crime/Policing, Cost of Living and Taxes, Energy Costs, Housing/Homelessness, and Parental Rights.

Critiquing every question would result in an extremely lengthy post, so rather than do that, I’ll discuss a few selected questions. First, let’s consider a question that gets a passing grade with respect to the neutrality of its wording:

QUESTION: If the election for the Washington state legislature in your district were held today, would you vote for the [ROTATE: Republican/Democratic] candidate or the [ROTATE: Democratic/Republican] candidate?

[PHONE: If Republican/Democratic: And would you DEFINITELY vote for the (Republican/Democratic) candidate or only PROBABLY vote for the (Republican/Democratic) candidate?]

ANSWERS:

The Republican candidate, definitely/probably: 44%
The Democratic candidate, definitely/probably: 45%

1. The Republican candidate, definitely: 29%
2. The Republican candidate, probably: 15%
3. The Democratic candidate, probably: 18%
4. The Democratic candidate, definitely: 27%
5. Don’t know/Refused/Unsure: 11%

The wording here works because no attempt is being made to influence people’s responses one way or the other, which is extremely important.

They also rotated their answer options, which they didn’t have to do, but which is very defensible. And unless something was omitted from the script, they avoided trying to stack the deck by putting negative messaging into previous questions (e.g. by saying Democratic legislators have been passing bad policies).

However, because their sample skews Republican, the responses to this question don’t reflect the state’s actual electoral dynamics. We’d expect to see somewhere in the range of like a five to ten point lead for the Democratic Party in a generic ballot question like this. Instead, the Republicans are even with the Democrats.

This is an unsupportable finding.

Similarly, in an adjacent question, immediately prior to this one, respondents were asked whether they have a favorable or unfavorable view of three people and two groups: Commissioner of Public Lands Hilary Franz, Democrats in the Washington State Legislature, Attorney General Bob Ferguson, Governor Jay Inslee, and Republicans in the Washington State Legislature.

The question wording looks just fine.

But because of the sample’s Republican skew, the favorability ratings don’t line up with what we’d see in a credible public opinion research survey.

For example, consider Governor Inslee’s favorability rating:

QUESTION: Do you have a favorable or unfavorable view of the following groups or individuals in the news lately? If you haven’t heard of this person, just say so. [RANDOMIZE OPTIONS] [PHONE: If favorable/unfavorable: And would you say you have a VERY (favorable/unfavorable) view or only a SOMEWHAT (favorable/unfavorable) view?

ANSWERS FOR GOVERNOR JAY INSLEE:

• Favorable (Very/somewhat): 40%
  • Very favorable: 20%
  • Somewhat favorable: 20%
• Unfavorable (Very/somewhat): 56%
  • Somewhat unfavorable: 12%
  • Very unfavorable: 44%
• Heard of / no opinion: 3%
• Haven’t heard of: 1%

We periodically ask about Governor Jay Inslee’s job performance, including as recently as last month. That’s not the same thing as favorability, but it is a similar line of inquiry. Typically, we see close to half our sample giving Governor Inslee positive marks and close to half giving him negative marks.

That was true again about a month ago:

QUESTION (FROM NPI’S RESEARCH): Do you approve or disapprove of Governor Jay Inslee’s job performance?

ANSWERS (JUNE OF 2023):

• Approve: 48%
• Disapprove: 47%
• Not sure: 5%

Our survey of 773 likely 2024 Washington State voters was in the field from Wednesday, June 7th through Thursday, June 8th, 2023.

The poll utilizes a blended methodology, with automated phone calls to landlines (41%) and online answers from cell phone only respondents (59%).

It was conducted by Public Policy Polling (PPP) for the Northwest Progressive Institute, and has a margin of error of +/- 3.5% at the 95% confidence interval.

NPI and PPP have worked together for a decade and have a track record of excellence, as detailed in this 2022 electoral polling recap and this 2020 one.

You can see from other public opinion research since the 2020 presidential and gubernatorial election that our finding is a typical one, whereas Echelon/Future42’s is an outlier. It is not surprising that a poll that over-samples Republicans would yield more negative numbers for Governor Inslee, as well as Attorney General Bob Ferguson, who’s now running for governor.

Now let’s look at a question that utterly fails the neutrality test.

QUESTION: Under Federal law, capital gains are considered income, and the state constitution of Washington prohibits an income tax. Recently, the State Supreme Court ruled that a capital gains tax is permissible if it is called an “excise tax”. Was the Supreme Court right or wrong to revise the definition of income to allow for this new tax?

(Answer options were randomized)

ANSWERS:

• Wrong: 62%
• Right: 24%
• Don’t know / refused / unsure: 14%

This is definitely a contender for the worst question in this poll: it’s prejudicially worded and it contains inaccurate information. Let’s correct the record:

1. What federal law says about capital gains is not relevant because people are being asked about a state law which levied a capital gains tax only on the wealthiest Washington State families and households.
2. The Washington State Constitution does not prohibit an income tax. This is a myth. Go ahead and read our plan of government for yourself — nowhere will you find a provision saying that the state may not impose an income tax. There is an old Supreme Court decision from the 1930s which held that income is considered property, and there is a clause in the Constitution (the “uniformity clause”) that says taxes on property must be uniform, but that’s not a prohibition against having an income tax.
3. The state Supreme Court’s recent ruling in Quinn is egregiously mischaracterized here. The Court did not rule that a capital gains tax is permissible if it is called an excise tax. Rather, it ruled the capital gains tax is permissible because it is an excise tax. The Court examined the tax in-depth and the law that created it, and emphatically rejected the right wing’s argument that it is not really an excise tax.
4. And finally, the Supreme Court did not “revise the definition of income to allow for this new tax.” Rather, the Court concluded that the Legislature had acted within its lawful authority to create an excise tax on capital gains. See for yourself by reading the majority opinion.

At no point in this question are respondents given any positive information about the capital gains tax on the wealthy. At no point are they told who pays it and what’s exempt. Instead, they’re told the tax is bad and then invited to agree with the authors of the question that the Supreme Court erred by upholding the tax.

As I mentioned earlier, you can’t find out what people think if you tell them what to think first. Future42 did not write this question in a way that would allow the respondents to bring their own opinions to the table. Instead, they wrote a loaded question which reads like a Tim Eyman style push poll. Bad questions and a skewed sample combine here to yield utterly worthless figures.

Here is what a balanced, neutrally worded question looks like. This is one we’ve asked for several consecutive years now, including last month:

QUESTION (FROM NPI’S RESEARCH): Proponents say that Washington State’s new state capital gains tax on the wealthy will raise about $500 million a year in crucial funding for education in Washington State, including early learning and childcare, and will help balance our upside-down tax code by requiring the wealthiest 8,000 individuals to step up and pay their fair share in dues to our state. Opponents say that the new state capital gains tax on the wealthy is an unconstitutional and illegal income tax that will hurt job creation and put the state at a competitive disadvantage, hurting the whole economy while failing to address regressivity. Both sides agree that the text of the capital gains tax law fully exempts retirement accounts, family farms, and all real estate. Having heard the arguments for and against, do you strongly support, somewhat support, somewhat oppose, or strongly oppose Washington’s new state capital gains tax on the wealthy?

ANSWERS (JUNE OF 2023):

• Support: 57%
  • Strongly support: 44%
  • Somewhat support: 13%
• Oppose: 37%
  • Somewhat oppose: 9%
  • Strongly oppose: 28%
• Not sure: 6%

Support is about the same as when we asked last year (57% now versus 56% last June.) Opposition is the same, at 37%, with a tiny, statistically insignificant bit of softening. The number of not sure voters has also likewise barely changed.

You’ll notice that in our question, the unconstitutional and illegal income tax framing is present, but it’s appropriately attributed to opponents of the tax rather than being passed off as a fact. (If you’re curious, the “con” phrasing used came directly from a group led by Rob McKenna — Opportunity Washington.)

The quality of the non-demographic questions really varied in this poll.

Generally speaking, the electoral questions and favorability questions had acceptable wording, while the questions about issues utilized right wing framing and lacked neutrality to varying degrees. Interestingly, Echelon generally asked respondents whether something was “a good idea” or “a bad idea” rather than asking if they supported or opposed it, which is the dichotomy we generally use. Their results file says the answer options were randomized as a general practice.

Concluding thoughts

Our assessment is that this survey skews Republican by as much as ten points, which is coincidentally twice its stated margin of error. Accordingly, even the topline responses to the neutrally worded questions are flawed.

That’s disappointing, because this survey might otherwise have yielded some useful findings and added to our state’s body of credible public opinion research.

For their next survey, we urge Echelon/Future42 to (1) fix their sampling to include a sufficient number of Democratic and college-educated voters and (2) simplify their party identification question, which also could help with (1). Those moves might at least yield usable data from their electoral questions, even if their issue-related questions remain poorly and prejudicially written.

Putting a right wing poll of Washington State under the microscope: An evaluation of Echelon/Future42’s spring 2023 survey is a post from NPI's Cascadia Advocate, the journal of the Northwest Progressive Institute. Published continuously since March of 2004, NPI's Cascadia Advocate provides thoughtful commentary and analysis on regional, national, and world politics. Keep The Cascadia Advocate going by making a contribution to sustain NPI's research and advocacy here.