BREAKING: Majority of Utahns Support Gay Marriage, New Poll Confirms

Last week I released a poll about support for marriage equality in Utah to update a two-year-old study from BYU that has frequently been used as a proof point countering Judge Shelby’s ruling on Amendment 3 in Utah. The first poll found that in two year’s time, 13% of Utahns switched their opinions from only supporting civil unions to supporting full civil marriage equality.

One criticism about my methodology has raised some questions. Some groups said by including civil unions as a choice, the poll may have under-represented support for full civil marriage equality. For context, I included civil unions as an option in order to compare directly against the BYU study; however, when the same observation was made by pollsters from the Williams Institute, I decided to run a new poll. Here are the results.

Utah Marriage Equality Poll 1-11-14

Given a choice between civil marriage equality and no legal recognition for LGBT couples, 51.3% of Utahns support full civil marriage equality and 43.7% oppose. The new scientifically accurate poll was conducted between January 8th to January 11th, 2014.

Which groups have changed their mind the most in the last ten years? Breaking down the data, women support civil marriage equality the most in Utah at a whopping 58 to 37%. Women ages 18-34 AND women 45-64 all have support of at least 52%. Supporters of marriage equality also come from both urban and suburban regions, and typically households making more than $50,000 a year.

This is the first time a poll has found that a majority of Utahns support marriage equality. It represents a major and rapid opinion change (+17) since Amendment 3 passed with 65% of the vote a decade ago. In Utah, the tipping point has been crossed for marriage equality.

  • Soul Glencoe

    I feel heartened at this revelation.

  • http://www.insidegaymormon.blogspot.com/ Trevor

    Why isn’t this making news?!

    • http://blog.davidbbaker.com David B Baker

      I just posted it. Give some reporters a bit of time but I’ve sent it to several.

  • Bill Roberts

    How were the data collected? If it was done online then it excluded a healthy portion of the population who oppose SSM. Many old people are still stuck in their ways. Including not being able to computer.

  • R Jay

    What were your calculations to power the study in order to determine statistical significance? Also, don’t you think the fact that nearly 30% of results were obtained from arts and entertainment sites gives you a young, liberal bias? Additionally, your response choices are very limiting, all or nothing. Many people who support civil unions but not marriage may have just not responded, dramatically skewing your data.

    • http://blog.davidbbaker.com David B Baker

      Hi R Jay, the statistical significance is provided by the tool not by me. If you look back at the poll from last week there was an option for civil unions. I don’t think the lack of an answer biases the response rate as the two are comprable at 22% there also is an option for other that many took advantage of.
      The weight of a hypothetical “young and liberal” audience can be seen in the raw age data and the fact that news sites skew more conservative in Utah.

  • B. K.

    Was the poll ever announced when it was going to take place? I didn’t hear anything about another poll being conducted. I think if all of Utah was to vote the polls might be different.

    • Aaron

      I can actually imagine an announcement would create response bias. An attempted random sample representative of the general population is different than a popular vote in this regard.

  • SE

    Looking at the actual data, your sample biases toward males and young people, precisely the segments of the Utah population that we’d expect ex-ante to favor legal same-sex marriage. In other words, your “sample” can hardly be considered random with respect to the question you’re trying to answer. It’s a convenience sample. I realize that you’re probably limited by resources and scientific training, but if you’d really like to answer this question you’d need to use more sophisticated sampling techniques to acquire a representative sample.

    • http://blog.davidbbaker.com David B Baker

      First I challenge your ex-ante assumption that males are expected to favor civil marriage equality as nationally there is an 8-pt spread between Men and Women with Women favoring same-sex marriage at 56% to Males 48% (per Gallup – http://www.gallup.com/poll/163730/back-law-legalize-gay-marriage-states.aspx)

      Second, the sample also biases older with the 55-64 range having almost a double sample size than the census data indicates for that age range which would counter-balance your youth argument as every other age is sampled along-side the census data +/- a couple of points (see below) EDIT: the spreadsheet didn’t format right in the comments below I apologize but I think you can recreate.

      Sample
      Total
      18-24
      25-34
      35-44
      45-54
      55-64
      65+

      Utah Census Data
      2,763,885
      12%
      16%
      12%
      11%
      9%
      9%

      GCS Poll
      500
      18%
      17%
      15%
      10%
      16%
      9%

      Varience
      -
      6%
      1%
      3%
      -1%
      7%
      0%

      • SE

        Notice I said “segments of the Utah population” that we’d expect to favor same-sex marriage, not segments of the national population. It’s well known that Utah women are much more likely than women nationally to be mothers and to hold traditional views on womanhood and families. Thus, your gallup poll doesn’t do much to rebut my challenge.

        Regardless, our back and forth arguments only emphasize the weak empirical design of your survey. Even if your sample controlled for age (which it doesn’t, as the raw data demonstrate), your sample doesn’t control for other well known determinants of ideology such as internet usage (see Bill Roberts’ comment in this thread).

        The point is this: When a survey uses weak sampling techniques, it’s very easy for you (or me) to tell stories about why the data are not (or are) biased. A strong empirical design rules out alternative explanations, and limits the ex-post “storytelling” in which you and I are engaging.

        All I’m saying is that if you truly wanted to convince people, you’d need to adopt a stronger empirical design. Stronger empirical designs (such as the gallup poll you cited above) can make statements such as this one: “Samples are weighted to correct for unequal selection probability, nonresponse, and double coverage of landline and cell users in the two sampling frames. They are also weighted to match the national demographics of gender, age, race, Hispanic ethnicity, education, region, population density, and phone status (cellphone only/landline only/both, and cellphone mostly).”

        Your survey cannot.

        • Ignorance is not bliss

          Do you have a brain or do you just not use it? To suggest that Utah men are more progressive then women is a clear misrepresentation of facts. Since your Majesty appears to be a data king, why don’t you run a survey that proves him wrong?

          • SE

            …More storytelling… which “facts” am I misrepresenting? Can you link to said “facts”? Better yet, I’d prefer you just cut to the chase and re-read the 3rd paragraph of my comment above.

            Apparently one can run a crappy survey, and then the burden of proof falls on skeptics to “run a survey that proves the crappy survey wrong”. Welcome to the real world, where if you want to convince a skeptic, you need to carefully form a hypothesis, design a valid empirical instrument, carefully execute the instrument, and carefully analyze the data. Doing all this takes training, time, and resources. That’s why large organizations such as Gallup exist–to do this type of work right.

            When some random blogger named David Baker decides he’ll sidestep this difficult process, he (and others) shouldn’t be surprised when skeptics engage in “storytelling”.

            I don’t claim to be a data king, but I am trained in social science research at an Ivy-league doctoral program. Social scientists work years to publish a single paper, because it takes a lot of work to publish valid conclusions about a topic. So you can imagine why we get annoyed when we see people drawing sweeping conclusions from a crappy survey.

        • http://blog.davidbbaker.com David B Baker

          Hey SE,

          1st. I think that you should provide at least something to indicate the veracity of your claim that Utah women support same-sex marriage less than Utah men because the raw data indicates otherwise.

          2nd. While my poll isn’t perfect, the fact that we are having back & forth doesn’t mean that I am wrong and you are right. (see Climate change scientific data vs. deniers) a random commenter on a blog can easily attack any data-set and even other scientists do so all the time. Climate Change data is held up because of multiple repeatable experiments. This is the first and will hopefully spur others. If they prove my data wrong then we can have that discussion but this is the best available data at present, until there is other data that polling groups should be doing, then we will can use this data point as a spur for conversation on the shifting attitudes within Utah.

          • SE

            In response to 1):
            Surveys show that women outnumber men in the LDS Church, especially in Utah (http://www.sltrib.com/sltrib/news/53117548-78/mormons-utah-percent-mormon.html.csp). Given the LDS Church’s stance on same-sex marriage, it wouldn’t be surprising if women in Utah are less supportive of same-sex marriage. Does the gender story not convince you? Let me tell two more stories:

            Story #2: Given that you use an internet survey to collect your convenience sample (yet don’t adjust demographic weightings), you likely underweight
            Latinos (see http://www.pewhispanic.org/2013/03/07/closing-the-digital-divide-latinos-and-technology-adoption/
            ). Latinos are less likely than the national population to support same-sex
            marriage (see http://firstread.nbcnews.com/_news/2013/04/12/17721816-poll-latinos-move-in-favor-of-gay-marriage?lite)
            . Thus, your survey likely biases toward a more favorable view of same-sex
            marriage.

            Story #3: There is some evidence that conservatives are less active on the internet than liberals (see http://pewinternet.org/Reports/2012/Political-engagement/Additional-Analysis/Social-Media-and-Political-Engagement.aspx). Given that you use an internet survey to collect your sample (yet don’t adjust for political ideology), your survey may overweight liberals, who are more likely to support same-sex marriage.

            I could think of more stories if you’d like me to. Some of these stories may hold true, others may not. I’m just trying to reinforce the point: Poor research design leaves skeptics telling stories.

            In response to 2)
            I agree that we can use this data point as a spur for further conversation, and I sincerely hope that polling groups do run more scientific polls on this
            topic. I’m mainly annoyed with how you present this “data point”. If, for example, you were to change the title of the blog to: “Does a Majority of Utahns support Gay Marriage?” and then at the beginning of the post say something like this: “Disclaimer: I ran this poll using a convenience sample collected from an internet survey. My sample is not representative of the larger Utah population. The confidence intervals and significance tests I provide do not adjust for demographic and other biases in my sample. In all likelihood, if I were to adjust for these biases, the significance test provided in this blog post would not be statistically significant.” If you were to do all that, I wouldn’t object.

          • SE

            New poll out conducted by Dan Jones showing that majority of Utahns still support amendment defining marriage as between a man and a woman: http://www.deseretnews.com/article/865594458/Poll-Majority-of-Utahns-against-same-sex-marriage-and-say-states-have-the-right-to-decide.html?pg=all.

            It appears that the flaws in your poll did have quite an impact on the results.

          • http://blog.davidbbaker.com David B Baker

            Actually it doesn’t highlight anything about my poll. We have no clue on the actual questions asked let alone the sample data. And the Tribune’s poll contradicts Dan Jones and supports mine.

            Let’s look at who is funding the Dan Jones poll and take that into consideration with the results.

            Additionally do you admit that you were wrong on the women aspect? Women in Utah for all 3 polls trend towards supporting marriage equality aligned with national data, contrary to your unsupported theory that just because they are LDS moms they are against same-sex marriage.

          • SE

            The question asked was: “In 2004, the following amendment was on the ballot in Utah: ‘Marriage consists only of the legal union between a man and a woman. No other domestic union, however denominated, may be recognized as marriage or given the same or substantially equivalent legal effect.’ If that amendment to Utah’s constitution were on the ballot today, would you vote for it or against it?”

            The fact that the majority of Utahns still support the amendment clearly contradicts your claim that the majority of Utahns believe “same-sex weddings should be legal”.

            Dan Jones is the most respected pollster in Utah. It’s laughable that you would call into question his credibility given how you ran your poll and represented your data.

            The Tribune’s poll does not support yours. That poll found no difference–which certainly doesn’t support the conclusion that the “majority of Utahns support gay marriage” as your headline claims.

            I’m happy to admit I was wrong on my gender story. As I mentioned earlier: “some of these stories may hold true, others may not”. Apparently at least one of my stories held true, because both polls reached a different conclusion than yours did.

          • http://blog.davidbbaker.com David B Baker

            Notice I didn’t critique Dan Jones who himself has admitted that reaching younger voters is difficult given cell phone usage. I critiqued and questioned Deseret News.

            Additionally that was a single question in the poll not the main one quoted at the top of the article.There is a difference between asking people belief questions and “were it on the ballot” questions.