The Righteous Mind Challenge

Posted by in moral philosophy

In a recent essay in the evolutionary magazine This View of Life, I analyzed Sam Harris’s Moral Landscape Challenge, in which he offers to pay $10,000 to anyone who can  convince him to change his mind and renounce his views.  From the perspective of The Righteous Mind, it seems unlikely that anyone who is heavily invested in an idea, and who writes about it with high levels of certainty, can be persuaded to change merely by the force of an essay words written by a stranger. So I offered to pay Harris $10,000 if he changes his mind.

Please read that essay first. This blog entry just gives lots of additional details, particularly about the word-count analyses. I’ll update this entry as people ask me additional questions.

I) How did I choose books to analyze?

I started by analyzing the three big New Atheist books: The God Delusion, The End of Faith, and Breaking the Spell. I then chose 3 recent books that were written by scientists who seem focused on explaining religion, not condemning it. I chose my own book, plus the two books that I knew about which came out shortly after mine, on the origins and psychology of religion: Jesse Bering’s The Belief Instinct, and Ara Norenzayan’s Big Gods. (I also examined some older books, to be confident that I wasn’t cherry picking, and they too were almost all below 1.6, so I just picked the 3 most recent of all the books I analyzed.) To pick the right-wing writers, I chose three of the most prominent, and then went onto Amazon to see which of their books was the most reviewed, which I took as a proxy for most read, or more influential. That led me to Glenn Beck’s Common Sense, Sean Hannity’s Deliver us from Evil, and Ann Coulter’s Treason.  But here too, I analyzed a bunch of other books by these and other authors, and found that they typically fell in the middle range, between 1.4 and 1.7. [see below, in section III-B]

 

II) How did I do the analyses?

I obtained plain text files of all the books  (I also bought Kindle versions of all the books, to ensure that the authors would get the royalties they deserve). I checked the text files carefully to make sure there were no issues that would skew the word count, such as headers that repeated on every page. I stripped out all the front-matter before the first word of the main text, and all the end-matter after the last chapter. I kept in introductions and epilogues, but cut out acknowledgments, notes, appendices, and references. I then ran each text file through LIWC, using the built-in 2007 dictionary, outputting the Certain category. Here are all the words that LIWC scores when computing its “certain” score:

absolute, absolutely, accura*, all, altogether, always, apparent, assur*, blatant*, certain*, clear, clearly, commit, commitment*, commits, committ*, complete, completed, completely, completes, confidence, confident, confidently, correct*, defined, definite, definitely, definitive*, directly, distinct*, entire*, essential, ever, every, everybod*, everything*, evident*, exact*, explicit*, extremely, fact, facts, factual*, forever, frankly, fundamental, fundamentalis*, fundamentally, fundamentals, guarant*, implicit*, indeed, inevitab*, infallib*, invariab*, irrefu*, must, mustnt, must’nt, mustn’t, mustve, must’ve, necessar*, never, obvious*, perfect*, positiv*, precis*, proof, prove*, pure*, sure*, total, totally, true, truest, truly, truth*, unambigu*, undeniab*, undoubt*, unquestion*, wholly

 

III) What are the actual mean scores?

A) Here are the original results I posted.

Author

Title

LIWC Certain score

Harris The End of Faith 2.24
Dennett Breaking the Spell 1.77
Dawkins The God Delusion 1.7
Beck Common Sense 1.56
Hannity Deliver us from Evil 1.49
Coulter Treason 1.49
Bering Belief Instinct 1.56
Norenzayan Big Gods 1.2
Haidt Righteous Mind 1.15

Note that there are no standard deviations, and no error bars on the graph, because these numbers are not samples from a larger population. They are the exact measurement done on the total population of words in each book.

B) Analyses of additional books [added on 3/3/14]:

Rahul in the comments asked me to post analyses of some other random works, e.g., from Gutenberg.org, to give us more context within which to interpret the certainty scores I posted. I agreed that this was a good idea, and asked him to pick some texts from Gutenberg, ideally works of relatively modern non-fiction. Rahul obliged, and provided this list. I show each work with its LIWC certainty score in parentheses: Darwin: On the Origin of Species (2.04); United States Presidents’ Inaugural Speeches: From Washington to George W. Bush (2.18); Gandhi: My Experiments with Truth (1.68); Speeches & Letters of Abraham Lincoln (2.05); Albert Einstein:  The Meaning of Relativity  (1.81); Works of Martin Luther (2.26); G. K. Chesterton “Orthodoxy” (2.85).  I was surprised to find a relatively high score for The Origin of Species, which did not fit with my general sense of Darwin’s careful writing. So I added in all of his other books, which turned out to score quite low on certainty: Darwin: Descent of Man (1.43);  Darwin:  Expression of emotions (1.32);  Darwin: Voyage of the Beagle (1.31), so Darwin’s average certainty score across his four books was 1.52 .  I do grant that there is variation within each author, depending on the writing task at hand. Ideally, one would analyze multiple books from each author.

 

To put this all together, I provide below the LIWC certainty scores of all the books I have now analyzed, including the 9 I presented in my original essay; the 7 suggested by Rahul, plus Darwin’s others; the  right wing authors I did not show in my original analysis (e.g., Rush Limbaugh, Bill O’Reilly, Michael Savage, and Mark Levin); and two older books on religion by non-new-atheists (David Sloan Wilson and Scott Atran). I sort the table by declining avg score for each author, given that I have more than 1 work for 6 of the authors.

Work analyzed in LIWC Avg certainty for each author Certainty score for each book
chesterton.orthodoxy 2.85 2.85
HARRIS AVG 2.3
   harris.end of faith 2.24
   harris.The Moral Landscape 2.37
luther.works of martin luther 2.26 2.26
us.presidents.inaugural.speeches 2.18 2.18
lincoln.speeches and letters 2.05 2.05
einstein.the meaning of relativity 1.81 1.81
dennett.breaking the spell 1.77 1.77
dawkins.god delusion 1.7 1.7
gandhi.my experiments with truth 1.68 1.68
beck.common sense [right wing] 1.66 1.66
wilson.darwins cathedral [not a new atheist] 1.65 1.65
limbaugh.the way things ought to be [right wing] 1.64 1.64
O’REILLY AVG [right wing] 1.59
   oreilly.culture-warrior 1.59
   OReilly.whos_looking_out_for_you 1.6
bering.the belief instinct 1.56 1.56
COULTER AVG [right wing] 1.56
   coulter.godless 1.63
   coulter.treason 1.49
HANNITY AVG [right wing] 1.53
   hannity.deliver us from evil 1.49
   hannity.let freedom ring 1.57
DARWIN AVG 1.52
   Darwin.descent_of_man 1.43
   Darwin.expression_of_emotion 1.32
   darwin.on the origin of species 2.04
   Darwin.voyage_of_the_beagle 1.31
atran.in gods we trust [not a new atheist] 1.43 1.43
savage.liberalism is a mental disorder [right wing] 1.43 1.43
levin.liberty and tyranny [right wing] 1.41 1.41
HAIDT AVG 1.25
   haidt.happiness-hypothesis 1.36
   Haidt.Righteous-Mind 1.15
Norenzayan.Big Gods 1.21 1.21

 

 

IV) Miscellaneous Methodological Issues

A) Some people on Twitter have pointed out a possible confound in the analyses: the LIWC certainty dictionary contains roots like funtamentalis* and fact. If the New Atheists are talking ABOUT religious fundamentalists, or if Harris’s book The Moral Landscape is about values as facts, its unfair to give them points for using those words. I agree, those are false positives. So I will re-run all analyses using a very restricted dictionary, which uses only the most unambiguously dogmatic words, such as “always” and “never.”  I propose using this subset of the LIWC dictionary: Absolute, absolutely, always, certainly, definitely, every, inevitab*, must, necessar*, never, obvious*, totally, undeniab*, undoubt*, unquestion*. Please comment below if you think I should cut any of those words, or add back any from the full list given in section 2.

[Text added 3/3/14]: Commenter Bianluca Barbetta, below, suggested cutting “absolute.” So I did that and then re-analyzed the original 9 books (plus Moral Landscape) using the restricted dictionary listed above. The resulting numbers are much lower, of course (because many fewer words are captured), but the basic picture that emerges is little changed. In declining order, the scores are: Harris-End of faith (.47); Beck-Common Sense (.47); Harris-Moral Landscape (.46); Dennett-Breaking The Spell (.39); Dawkins-God Delusion (.38); Hannity-Let Freedom Ring (.37); Coulter-Treason (.36); Bering-Belief instinct (.30); Haidt-Righteous Mind (.21); Norenzayan-Big Gods (.15).  These scores show the percentage of all words used in each book that were in the list of words coded by the dictionary. For Harris and Glenn Beck, it’s nearly a half percent of all words.

B) To get an independent check on whether the effects I report are real and robust, I hope somebody will create large text files for each author I chose, composed of, say, 20 blog post available on the internet, and then run LIWC on those files. If someone does that, i”ll post a link to the results here.

C) [added on 2/18/14]: Many of the critical commenters below note that there are many ways of using words like “certain” or “certainly” which don’t indicate anything about the mindset of the author. They are right. LIWC is a simple word count program; it does not analyze words in context, and it does not control for negation (e.g., “it is not certain that…”). So there are many false positives. A LIWC does not by itself prove that the new atheists are more dogmatic than other groups of authors. But it can test one’s subjective impressions; it can add or subtract confidence in one’s impressions. When I read the New Atheist books, Dawkins and Harris sounded angry, whereas Dennett did not. But all three authors seemed to to me to use certainty formulations to an unusual degree. I then ran LIWC to measure both of those categories, and it turned out that the New Atheists were high on both (except for Dennett, who scores low on anger, confirming the impressions of many readers). But then I examined the LIWC output to see the words in context, and it was clear to me that Harris scored high on anger in large part because he is talking about violence and killing related to religion. That’s his subject matter, not his emotion.  There were so many of these false positives that I decided it would not be fair or accurate to publish the anger findings. The certainty findings, however, hold up much better. Those word uses did seem to be capturing something about Harris’s prose style. In response to a commenter below, I opened up The End of Faith at random, to page 40, and found this passage:

“The basis of our spirituality surely consists in this: the range of possible human experience far exceeds the ordinary limits of our subjectivity. Clearly, some experiences can utterly transform a person’s vision of the world. Every spiritual tradition rests on the insight that how we use our attention, from moment to moment, largely determines the quality of our lives.”

Surely, clearly, and every? This is not the way most scientists write.

 

V) Additional Stuff:

A) Here is my entry in the Moral Landscape Challenge, where I say why I think Harris’s claims about morality are wrong.

B) [added on 2/18/14] What will it take for me to pay Harris the $10,000? I will send him a personal check (or donate to his foundation, whichever he prefers) if two conditions are met: 1) Harris pays someone the $10,000 of his own money for writing an essay that changed his views about morality, and 2) this payment is accompanied by an explicit acknowledgment that his “case for a scientific understanding of morality is mistaken,” as he puts it in the challenge itself. Harris and I both agree that a scientific understanding of how morality and moral judgment work, descriptively, is appropriate, and is proceeding well. (We both participated in an Edge.org project on The New Science of Morality). We agree that scientists can study morality just as they can study language, or sexuality, or color vision. The dispute between us arises over whether science can tell us what is in fact morally right and wrong, in the same way that science can tell us facts about the natural world. That is Harris’s most provocative and interesting claim in The Moral Landscape. If he renounces that claim, or in some other way says that his argument in The Moral Landscape was largely wrong, I’ll pay him the money. If he pays the money while admitting only a minor error, or conceding some peripheral points, that would not count as having changed his mind or accepted the refutation of his thesis.

C) [added 3/3/14] It is interesting that in Harris’s recent debate with Dennett over free will, Harris opens his rebuttal with a claim that seems on its face to support my basic argument:

The virtues of rational discourse are everywhere espoused, and yet witnessing someone relinquish a cherished opinion in real time is about as common as seeing a supernova explode overhead. The perpetual stalemate one encounters in public debates is annoying because it is so clearly the product of motivated reasoning, self-deception, and other failures of rationality—and yet we’ve grown to expect it on every topic, no matter how intelligent and well-intentioned the participants.

D) The full opening paragraph from Section I of Hume’s Enquiry, is worth reading:

DISPUTES with men, pertinaciously obstinate in their principles, are, of all others, the most irksome; except, perhaps, those with persons, entirely disingenuous, who really do not believe the opinions they defend, but engage in the controversy, from affectation, from a spirit of opposition, or from a desire of showing wit and ingenuity, superior to the rest of mankind. The same blind adherence to their own arguments is to be expected in both; the same contempt of their antagonists; and the same passionate vehemence, in inforcing sophistry and falsehood. And as reasoning is not the source, whence either disputant derives his tenets; it is in vain to expect, that any logic, which speaks not to the affections, will ever engage him to embrace sounder principles.

(Note that Hume is describing two different kinds of disputants – the pertinaciously obstinate kind, and the disingenuous kind. That is why he says “either disputant” in the last sentence. But since I’m only talking about one kind, I changed it to [such a].)

Note to commenters: You can be as critical as you want of my ideas, but any comments that use obscenity or insults will be deleted.

Read More

Look how far we’ve come… apart

Posted by in Civility

I have an essay in today’s NYT on America’s growing polarization. Marc Hetherington and I show what’s happened over the last 50 years, not just in Congress but among citizens, in 3 simple graphs.The first one is below, from McCarty, Poole, and Rosenthal. It shows that things were really polarized in the late 19th century, after the civil war, then they got much less polarized, but it’s been a rapid climb upwards since the 1960s. We talk about why.

 

The next 2 graphs were made by Hetherington, and they are just as depressing. Click here to read the essay.

But we end on an optimistic note. There really are so many changes we could make to roll things back, perhaps to the level of polarization in the 1990s, which was much lower than it is today. But only if we push our leaders to make the changes. They will not make the changes themselves, because each change will probably favor one side or the other, so the disadvantaged side will fight like hell. But extreme pressure from outside, for a comprehensive package of reforms — a kind of good-government-Simpson-Bowles — might do it.

—————

Post-script: see Pete Ditto’s call for “domestic realpolitik” to be realistic about how we’re going to get things done.

Read More

Moral psychology and the campaigns (on “The Cycle”)

Posted by in 2012 Campaign, Moral Foundations in Action, Politics, Videos

I was a guest today on a new MSNBC show, The Cycle, which is interesting because it features one conservative and three liberals discussing the days issues in a friendly way — I love it every time I see models of cross-partisan amity and constructive disagreement.

We talked about the role that moral values play in the campaigns. The hosts wanted to talk about what makes swing voters decide, but I preferred to talk about what energizes the bases. We haven’t had an election where both sides aimed for the middle since 2000, when George W. Bush ran as a moderate compassionate conservative.

[Forgive my big speech error of saying “right” once when I clearly meant “left.”]

 

Visit NBCNews.com for breaking news, world news, and news about the economy

Read More

Discovering that the other side is not really so loathesome

Posted by in Civility, Politics

Here’s a lovely essay from Michael Rubens, a former producer on The Daily Show with Jon Stewart. For years, Rubens job was to produce segments in which Daily Show correspondents interview and make fun of people with far-out views, mostly on the far right:

So imagine how irksome it was to have to deal with persons like that on a constant basis and discover that those persons, in person, generally weren’t loathsome persons after all. In fact, to my great consternation and disappointment, I often liked them.

Rubens describes a series of tea partiers, rapture-believers, and state representatives who push gun rights and squash gay rights. He hated what they stood for and wanted to find them despicable, but after working with them he found them to be as nice as anyone on his own side. Even the spokeswoman for the hate-spewing Westboro Baptist Church, turned out to be “warm and affable and lovely.” Here’s Rubens conclusion:

What I’m hoping the lesson is: People are complex and can hold different views and still be moral actors — essentially the message that Jon Stewart talked about during his Rally for Sanity. Maybe you already grasp that concept, because you have good friends or loving relatives with beliefs that are wildly divergent from your own. But I tend to think my experience is more typical: I lived in a little bubble surrounded by people who think more or less like me. And when I considered people with opposing viewpoints I would turn into a fabulist, concocting an entire narrative of who they were and what they were like — and what they were like was yucko. Because I was not really interacting with them. I just thought I was, because, hey, look, there they are on the TV, or there’s that guy’s post in the comments section. But that stuff doesn’t count. Meeting people counts. Talking counts.

So yes, I love to loathe people, but my “Daily Show” experience complicated all that and sort of spoiled my fun. When I’m exposed to views that I dislike, I try to remind myself of the human being behind those views and to cut that person some slack. I hope that they would do the same. I think we should all fight hard for what we believe in, but I’d like to put in a request for some general slack cutting – especially as we move deeper into what is sure to be a very heated campaign season.

 

Read More

The Largest Study Ever of Libertarian Psychology

Posted by in Moral Foundations in Action, Politics, Videos

We’ve been deluged in recent years with research on the psychology (and brain structure) of liberals and conservatives. But very little is known about libertarians — an extremely important group in American politics that is not at home in either political party.

At YourMorals.org we have now addressed the gap. Unlike most surveys, which force everyone to place themselves on a Left-Right scale, we have always allowed our visitors to choose “libertarian” as an option.  Given our unique web platform, where people register and then take multiple surveys, we have amassed what we believe is the largest and most detailed dataset in the world on the personality traits of libertarians (as well as of liberals and conservatives).

In a project led by Ravi Iyer, we analyzed data from nearly twelve thousand self-described libertarians, and compared their responses to those of 21,000 conservatives and 97,000 liberals. The paper was just published last week in PLoS ONE. The findings largely confirm what libertarians have long said about themselves, but they also shed light on why some people and not others end up finding libertarian ideas appealing. Here are three of the major findings:

1) On moral values: Libertarians match liberals in placing a relatively low value on the moral foundations of loyalty, authority, and sanctity (e.g., they’re not so concerned about sexual issues and flag burning), but they join conservatives in scoring lower than liberals on the care and fairness foundations (where fairness is mostly equality, not proportionality; e.g., they don’t want a welfare state and heavy handed measures to enforce equality). This is why libertarians can’t be placed on the spectrum from left to right: they have a unique pattern that is in no sense just somewhere in the middle. They really do put liberty above all other values.

2) On reasoning and emotions: Libertarians have the most “masculine” style, liberals the most “feminine.” We used Simon Baron-Cohen’s measures of “empathizing” (on which women tend to score higher) and “systemizing”, which refers to “the drive to analyze the variables in a system, and to derive the underlying rules that govern the behavior of the system.” Men tend to score higher on this variable. Libertarians score the lowest of the three groups on empathizing, and highest of the three groups on systemizing. (Note that we did this and all other analyses for males and females separately.) On this and other measures, libertarians consistently come out as the most cerebral, most rational, and least emotional. On a very crude problem solving measure related to IQ, they score the highest. Libertarians, more than liberals or conservatives, have the capacity to reason their way to their ideology.

3) On relationships: Libertarians are the most individualistic; they report the weakest ties to other people. They score lowest of the three groups on many traits related to sociability, including extroversion, agreeableness, and conscientiousness. They have a morality that matches their sociability – one that emphasizes independence, rather than altruism or patriotism.

In other words: Libertarians, liberals, and conservatives all differ from each on dozens of psychological traits, which help to explain why people – even siblings in the same family — gravitate to different ideological positions as they grow up. Understanding these psychological differences will be crucial for politicians and political movements that want to appeal to libertarians, who are often left out as so much attention is lavished on liberals and conservatives.

Here is the article itself.

Here is a press release on it

And here is Ravi Iyer giving a 12 minute presentation summarizing the article:

Read More

Rape doesn’t lead to pregnancy, in the mind of someone seeking “Moral Coherence”

Posted by in 2012 Campaign, Politics

In my earliest research I discovered that people sometimes invented the facts they needed to back up the moral judgments they had just made. When I asked people about whether it was wrong for a family to eat it’s dog, after the dog was killed by a car, people often said “yes, it’s wrong, because…. um… if you eat dog meat you’ll get sick.” This finding became the First Principle of Moral Psychology in The Righteous Mind: “Intuitions come first, strategic reasoning second.” Our gut tells us what’s true on moral questions, and our reasoning then kicks into high gear to justify that intuition.

So when Todd Akin said that “legitimate rape” rarely leads to pregnancy because, um… “the female body has ways to try to shut that whole thing down…” I was both horrified and delighted. Delighted only because it offered such a vivid demonstration of the First Principle in action.

By coincidence, my colleagues at Yourmorals.org, Brittany Liu and Pete Ditto, have just published an article showing how exactly the process works when people invent or inflate facts. Ditto and Liu talk about the need for “moral coherence,” which they describe as “the tendency for people to fit their factual beliefs to their moral world-view, so that what is right morally becomes what is right practically as well.” The apply this perspective to the Akin case in a blog post at the YourMorals blog. Here’s an excerpt:

We suggest that people’s desire for moral coherences initiates a motivated cost-benefit analysis in which the act that feels the best morally becomes that act that also leads to the best consequences. So, if a particular act feels morally wrong, moral coherence processes lead people to try to maximize the costs and minimize the benefits associated with that act. Likewise, if an act feels morally acceptable, people will minimize the costs and maximize the benefits associated with that act. By changing their factual beliefs about the costs and benefits of various actions, people emerge with a coherent moral picture in which their factual beliefs fit perfectly with their moral evaluations.

Applying this logic to the Akin case, strong opponents of abortion, like Akin, argue that abortion is fundamentally immoral and should be prohibited. But what if the pregnancy results from a rape? This creates a problem for a principled moral position on abortion. Isn’t abortion always wrong? But is it right to make a woman live with a baby conceived in from a violent, traumatic act she did not consent to? One way to resolve the conflict is to convince oneself that pregnancies from “legitimate” rapes are exceedingly rare. If this is true, then prohibiting abortion even in the case of rape really has relatively few costs because it occurs so infrequently. Thus, it is easy to see Rep. Akin’s views about rape and pregnancy (views that are held by many other anti-abortion activists as well) as emerging from his struggle to construct a coherent moral position on abortion that refuses to make exceptions for rape and incest.

Pat Moynihan is reported to have once said that we are each entitled to our own opinions, but not to our own facts. Unfortunately, partisans tend to create their own facts, most of which are not so outrageously and obviously wrong as Akin’s. Left and right in America today live in different moral “matrices,” and the search for moral coherence, satisfied by cable news networks, gossip, and the internet, allows us all to live in gated moral communities, each one grounded in its own set of facts.

 

 

 

Read More

How to understand one third of all political arguments

Posted by in 2012 Campaign, Civility, Politics

I came across this (tongue in cheek) lament about the hypocrisy of the other side, on Volokh Conspiracy (but it’s floating around the internet):

Why is the other side of the debates I’m on always so hypocritical? They always jump on what my side says, and yet they willfully ignore all the faults on their own side. Let’s be honest about the double standard: The other side gets away with stuff that my side would never get away with. It’s just like the other side to be so deceitful: They’re always looking to score any advantage they can. People like that drive me crazy, and it seems like most of the people on the other side are just like that.

It’s a perfect distillation of the main point of Ch. 4 of The Righteous Mind (and ch. 4 of The Happiness Hypothesis). But blogs being blogs, people then set out to debate it. One commenter offered the perfect summation of what happens in maybe a third of all arguments about things that Obama (or any president) does:

It isn’t just a matter of each side claiming that the other side is hypocritical, and you have to figure out which (or both). The following often happens:

1) Right criticizes Obama for doing X
2) Left (correctly) points out that Bush did X, and Right didn’t care then
3) Right (correctly) points out that Left cared when Bush did X, but don’t now.

 

Of course, the same happens in reverse with the Left initiating the first complaint. Essentially, both sides are actually admitting hypocrisy, but for some reason they only care that the other side is hypocritical. This is a truly horrible form of discussion, and a neutral observer does not need to think hard to figure out which side is “right,” because both sides are wrong.

Amen. It is indeed striking that the response to the charge of hypocrisy is rarely apology, it’s usually “but, but, but… you do it too.” That’s what you’d expect if we all carry around in our heads a little inner press secretary, or inner lawyer.

—————

Update: By amazing coincidence, Ramesh Ponnuru published yesterday a much more extensive and deeply insightful version of the “they’re all hypocrites” rant. His (tongue in cheek) rant should be required reading for all citizens. (hat tip to Independent Whig, below)

Update #2: By even more amazing coincidence, A. Barton Hinkle of the Richmond Times Dispatch, wrote an essay similar to Ponnuru’s the day before his was published. This one’s called “The Wrong Side Absolutely Must Not Win”. (Hat tip to Brian Keegan, below)

Read More