Orwellian Analytics: Christians, Atheists, and Bad Statistics

A recent Live Science press release, titled “Believers Leave Punishment to Powerful God,” opened with the memorable words:

"Believing in an involved, morally active God makes people less likely to punish others for rule-breaking, new research finds."

Which is equivalent to saying that non-believers are less forgiving, less compassionate, less merciful, and—oh, let’s just say it: they are worse people. Don’t get mad at me. This is research!

But then maybe this summary is too telegraphic. Because the very same research that proves atheists are more bloodthirsty than theists also proves “that religious belief in general makes people more likely to punish wrongdoers — probably because such punishment is a way to strengthen the community as a whole” (emphasis mine).

In other words, theists are less forgiving, less compassionate, less merciful, and just plain worse people than atheists. Except when they aren’t and when their roles are reversed.

The press release explains the conundrum thusly: “In other words, religion may introduce two conflicting impulses: Punish others for their transgressions, or leave it to the Lord.”

This is the power of statistics, a field of science which, given the routine ease with which two opposite conclusions are simultaneously proved, we may now officially dub Orwellian Analytics.

Research Shows...

 
The paper which this popular article was based on is titled “Outsourcing punishment to God: beliefs in divine control reduce earthly punishment” by Kristi Laurin and three others. It was published online in the Proceedings of the Royal Society B.

After a lengthy introduction arguing that all morality (except presumably the morals of the authors) can be reduced to urges induced by evolutionary “pressures,” and defining something called “altruistic punishment”, the authors describe how they gathered small pools of WEIRD young people (i.e. Western, Educated, Industrialized, Rich, and Democratic undergraduates) and had them play games. The results from these games told the authors all they needed to know about who enjoys punishment more. Incidentally, about the punishment, they said this:

"Prior to effective and reliable secular institutions for punishment, large-scale societies depended on individuals engaging in ‘altruistic punishment’—bearing the costs of punishment individually, for the benefit of society."

And did you know that “According to theory”—are you ready?—“Though administering punishment benefits society as a whole, it has immediate costs for punishers themselves.” Who knew?

Experiment one corralled “Twenty undergraduates” who “participated in exchange for course credit.” That’s one more than nineteen, friends. The supplementary data (which is mysteriously left out of the main article, but which is linked there) shows that these participants contained 8 whites and 9 Asians, with 1 black and 1 Arabic left over; 10 Christians, 1 Buddhist, 1 Hindu, 1 Muslim, 1 “Other”, and 6 Atheists. The authors claim to have “measured participants’ belief in powerful, intervening Gods, and their general religiosity.” Which makes you wonder how they classed the Buddhist and “Other.” No word on the breakdown of how participants answered the “religiosity” question.

But the next part is more fascinating: “We then employed the 3PPG–an economic game commonly used to measure altruistic punishment.” I'm struck by the words “commonly used.” It must be common, because there isn’t word one in the paper or supplementary material of what this creation is. But I can reveal to you it is the “Third-Party Punishment Game,” a frivolity invented by academics designed to flummox undergraduate participants in studies like this. More about that another day.

The “game” runs so (sorry for the length, but do read it):

"Player A receives 20 dollars, and must share that money between herself and player B in two-dollar increments, without input from player B. In the second stage, player C [who presumably knows what A did], who has received 10 dollars, can spend some or all of that money to reduce player A’s final payout: For every dollar that player C spends, player A loses three dollars. Player A’s behaviour does not affect player C, all players are anonymous and expect no further interactions, and punishing player A costs player C money. People treat sharing money evenly between players A and B as the (cooperative) norm; thus, player C’s willingness to punish player A for selfishly violating this norm can be taken as an index of altruistic punishment of non-cooperators."

In other words, Player C looks at how much A gave B. If C thinks this too low, C sacrifices some of his own money to reduce the amount A kept. But A and C got the money for free and since these are students we do not know if A actually knew B in real life, or if C knew either. For example, if I as A and Uncle Mike as B and Ye Olde Statistician as C were to play this game, I would split the money with Uncle Mike and Ye Olde Statistician would go along. This is because we were pals before the game commenced. But if we were enemies, something entirely different would occur. The authors never mention if they look for these kinds of effects in this or in any experiment. Leave finding flaws and contrary evidence for others.

But never mind, because C giving up some of his play money is scarcely the same thing as C desiring that a child rapist be tossed in jail to rot, even though C knows that the cost of the rapist’s cell will be taken from his wallet. But C in real life hardly knows even that. C knows that he pays taxes and that some of his taxes go to prison upkeep, but those taxes also go to pay for the fuel to ferry the president around on Air Force One from fund raiser to fund raiser. That is, most of us Cs don’t think that ponying up taxes is altogether altruistic.

The authors are mute on this objection, too.

Enter The P-value

 

"We regressed participants’ levels of altruistic punishment [amounts of money] on their God beliefs and their religiosity (both centered around 0) simultaneously…participants who believed more strongly in a powerful, intervening God reported less punishment of non-cooperators, β = -0.58, t(17) = 2.22, p = 0.04; whereas more religious participants showed a trend towards reporting greater punishment, β = 0.33, t(17) = 1.67, p = 0.11."

And there it is. Theists reported less punishment and more punishment. Except that the p-value for the “more punishment” isn’t small enough to excite. (And a linear regression is at best an approximation here.) The authors also discovered “more religious people tended to believe in powerful, controlling Gods.” The correlation wasn’t perfect, but neither should it be when you mix Buddhists and Christians. Let’s don’t forget that this regression model only included 6 atheists for its contrast.

The really good news is that “Given the strong correlation between religiosity and conservatism (r = 0.52), we conducted an additional analysis including conservatism in the regression. Results are reported in table 1; we found no evidence that conservatism explains the religion–punishment association.” Sorry, Chris Mooney.

The authors did four more studies, all similar to this one, but with increasingly complicated regression models (lots of interactions, strong hints of data snooping, etc.). The findings don’t change much. In their conclusion, however, they include these strange words: “In our research, we found it necessary to remind participants of their beliefs for these beliefs to influence their decisions.” This sounds like coaching, a way to induce results the authors expected.

So what's the real lesson here? That one of the largest science sites on the Internet has no problem squeezing a complex mass of data into a terse and misleading headline.
 
 
(Image credit: Steve Dease)

Note: Our goal is to cultivate serious and respectful dialogue. While it's OK to disagree—even encouraged!—any snarky, offensive, or off-topic comments will be deleted. Before commenting please read the Commenting Rules and Tips. If you're having trouble commenting, read the Commenting Instructions.