Moderation for understanding: Tone matters for online comments, study says

This is a good reason for the comment policy on this site. The results of this rather large study is not surprising. Tone matters.

Online comments hurt science understanding, study finds

 

A new obstacle to scientific literacy may be emerging, according to a paper in the journal Science by two University of Wisconsin-Madison researchers.

The new study reports that not only are just 12% of Americans turning to newspaper and magazine websites for science news, but when they do they may be influenced as much by the comments at the end of the story as they are by the report itself.

In an experiment mentioned in the Science paper and soon to be published elsewhere in greater detail, about 2,000 people were asked to read a balanced news report about nanotechnology followed by a group of invented comments. All saw the same report but some read a group of comments that were uncivil, including name-calling. Others saw more civil comments.

“Disturbingly, readers’ interpretations of potential risks associated with the technology described in the news article differed significantly depending only on the tone of the manipulated reader comments posted with the story,” wrote authors Dominique Brossard and Dietram A. Scheufele.

“the tone of the comments . . . can significantly alter how audiences think about the technology itself”

Did you get that? It’s IMPORTANT.

It’s also a reason why I moderate the comments here rather heavily. Please read our comment policy if some of your comments never make it onto the post or suddenly disappear. It may be because I don’t agree your tone is helpful to the discussion. I have judged a comment trash if it’s pointless, may derail discussion, or present a distorted view of the story. The goal of Doubtful News is present a certain viewpoint. We are BIASED towards reason and scientific consensus. There are plenty of other places you can go where science is debased and disregarded in discussing these topics.  Or, where ad hominem attacks are allowed or encouraged. Thus, I do not allow derogatory comments unless there is a clear response that can illustrate an issue. Otherwise, those types of comments are a platform for those who aren’t serious about understanding the issue..

I can’t help but wonder if this tone argument also applies to other perception besides understanding science. I can’t see that it wouldn’t. If you represent your website with a certain keyword – whether that be religious, atheist, freethinking, parenting, fashion, medical, skeptic, feminist, whatever – and your comment threads are full of sniping, name-calling, and nastiness, what kind of image are you presenting? It’s not serious or professional. It’s certainly not discussion worth reading for understanding the issue. You want nastiness, there are plenty of places like that. Not here. For good reason.

Tip: CFI’s Morning Heresy

  11 comments for “Moderation for understanding: Tone matters for online comments, study says

  1. D.Walker
    January 8, 2013 at 2:26 PM

    That’s an excellent policy to have, and I will do my utmost to adhere to it, even though some of the things I read really get my goat. Civility is the best policy.

  2. January 8, 2013 at 2:37 PM

    I completely agree with moderating comments and it clearly matters. For some reason, people do give credence to comments, even when they have no idea who the person is, in most cases because it fits with what they already think, feeds a fear, or seems logical with limited information.

    However, I think there are a few caveats and some context to consider. The press release (or is it an article? It’s hard to tell) is very misleading, particularly the title.

    1) The study did not find a difference in “science understanding”. The measure which was affected by tone was perceived potential risk of the technology discussed in the piece. In fact, a very interesting finding was that science knowledge did not moderate the relationship between tone and perceived potential risk, so while people high in science knowledge may have rated risk lower than those low in science knowledge, the types of comments they read still affected their ratings. I’d like to see the numbers, though, because I’m baffled about why that finding wasn’t the focus of the article.

    2) People who are given something to read for a study will read all of it. However, what proportion of the people who get their news online read all (or any) of the comments?

    3) Since the article is not available yet (something that REALLY bothers me; they should not be reporting on something we can’t evaluate), I can’t see the details or effect sizes, which matter. I would also like to see exactly what the differences were between the two conditions. If ANY content of the comments has changed (anecdotes or opinions), the study is fairly worthless when it comes to measuring the effects of tone.

    • January 8, 2013 at 6:33 PM

      I rely on Barb to point out to me things I saw but didn’t NOTICE and didn’t emphasize their importance. There IS more to consider here. I didn’t take the time to do it.

      I realized I am talking about two different things here, though they may relate: tone in relation to understanding the content and tone in regards to giving an impression about the reliability of the information. In fact, it’s way more complicated than this, as are my reasons for the comment policy. But to stick with what the article mentions – scientific literacy – I’m not at all clear what they are actually saying.

      I also noticed that it describes “interpretation of potential risks” as a criteria. I was confused about that. Thus, “scientific” is not in the title of my framing of it though it is in the initial piece. How many people were influenced by the comments? How did they measure this? Don’t know. This was a crappy writeup, I agree, because there isn’t a stat from it except for (sort of) the N value.

      I chose to go for another angle with this story, the comment moderation in order to keep focus on the content, not get derailed or influenced by commentators. I suspect that’s what the study is pointing too, how commentators can skew your ideas too much.

      I’m glad the methodology flaws were brought out. I can’t assume the study will suck until we see what it was but it rings true for me in another sense – tone influencing my impression – since they emphasize the civility aspect.

      • January 8, 2013 at 7:06 PM

        There really isn’t enough info to judge if there are methodological flaws, exactly. The things I mentioned were limitations more than flaws. But it is clear that the piece misinterpreted/misrepresented the findings.

        The finding that “potential risk” is affected by comments is one of trust in technology, which isn’t actually science, but the application of it. So, it requires trust in the people who created the technology. But the comment that science knowledge didn’t mediate the effects is a dead giveaway that what the title says is false.

  3. Chris Howard
    January 8, 2013 at 2:46 PM

    I struggle with this a lot. I understand that tone matters, I’m just not certain thatit should?

    I think there are valid points to be made from both the DBAD and (for lack of a better phrase) anti-accommodationist crowds.

    Having worked with addicts, adjudicated juvenile sex-offenders, and gang members I can attest to the necessity of the “cold hard truth” method. This is especially true if the patient was attempting to harm themselves or others.

    The “CHT” method is often misinterpreted (especially by the patient) as being “mean, cruel, and nasty.”

    On the other hand an argument could be made that those who responded well to a “reality check” were already at a place psychologically where they were ready to hear the truth.

    I guess what I’m saying is that even though you get more flies with honey it may be necessary, from time to time, and depending upon the situation, to shock one into awareness. I’m not certain as to wether offended sensibilities should make a person deny the facts, ignore evidence, and advance a comforting lie at expense of truth?

    Of course that point, while philosophical ideal, has nothing to do with “the real world” and we must operate in the world as it is.

    I also wonder wether the rise in narcissism and sociopathic behavior isn’t leading to a rise in people getting offended so easily, and an increase in science denial. It’s difficult to learn, and accept anything when ones personality disorder lends themselves to believing that they are an expert at everything, and therefore never wrong.

    • January 8, 2013 at 4:42 PM

      The “cold, hard truth” can be stated without unnecessary personal insults. Humans have a natural tendency to dichotomize things, so it shouldn’t be surprising that arguments against “DBAD” seem to assume that the opposite of it is “being nice”. It’s really not.

      That said, there are people who take any criticism whatsoever as “mean” and they’re growing in number (those people you noted in your last paragraph). There isn’t much you can do about how those people perceive what you say/write.

    • January 8, 2013 at 6:36 PM

      I think we are talking specifically about comments. Therefore, one is not the writer of the piece. In that case, you can pick your own tone as it suits. As a commentator I try to respect the writer and site and other commentators. So, your point of it takes all approaches is true. That’s useful when used strategically. Some people’s strategy is to BE the asshole, all the time. If they are willing to deal with any fall out from that, it works out well. Problems occur when they are misinterpreted or unwilling to accept the consequences.

      • January 8, 2013 at 6:49 PM

        “So, your point of it takes all approaches is true.”

        I haven’t actually seen any good evidence of that, just claims. But, again, not being an asshole doesn’t mean sugar and cream or lying/withholding criticism to spare people’s feelings. It’s simply not adding unnecessary personal insults.

  4. S. Madison
    January 8, 2013 at 5:06 PM

    Excellent post and excellent discussion in the comment section. Thanks Sharon, Barbara, Chris, and D. Walker.

  5. January 8, 2013 at 8:12 PM

    Over the years, I’ve been interested in what it means to be an ‘effective activist.’ Certain skeptical activists, for instance, will take approaches which widely differ featuring ridicule, name-calling, poking fun, using sarcasm….to only focusing on the arguments and not the person making them in a very ‘serious’ manner. I’ve drifted between the two at times (although I largely favor an approach which sticks to the arguments as much as possible in most situations) and participate in communities or otherwise engage with others who seem to use ridicule as a main tool. Both approaches, it seems, can be effective in their own ways and will appeal to different audiences.

    I don’t say there is “one way” or a “best way” concerning tone (or more specifically activism), but rather suggest for others to do what works for them and mostly stay away from ridicule on my end.

    • January 8, 2013 at 10:07 PM

      “I don’t say there is “one way” or a “best way” concerning tone (or more specifically activism), but rather suggest for others to do what works for them and mostly stay away from ridicule on my end.”

      I have heard this argument a LOT over the years and it makes little sense to me. All approaches are not equal or equally valid any more than all theories about speciation. There may be more than one good approach, but there are also many bad ones. And there is a TON of research on persuasion and learning–research that is pooh-poohed by people who just don’t want to give up being assholes. BTW, I consider that a form of science denial.

      We should advocate for best practices and, as I noted in another comment, I’ve seen a LOT of claims that ridicule “works” (whatever that is defined as), but very little evidence. Humor (e.g., South Park, Bullshit!) works because they allow people to save face and to fool themselves into thinking that they always knew that X was BS. That’s not the same thing as ridiculing others. Ridicule may appeal to an audience, but it’s not the audience we’re trying to persuade and educate. It’s the masses of people who enjoy feeling superior to others and watching those people get beaten up. Bullies and hate-mongers.

      Also, we shouldn’t just walk away from this and let people do “what works for them” because it only works FOR THEM. In addition to making the job of activism a lot harder for the rest of us (and ensuring that the label “skeptic” comes with assumptions that one is an asshole), they cause harm to the people they ridicule. Aren’t we trying to reduce harm?

Comments are closed.