Psychologising social surveys: the case of religion
Social surveys—such as the World Values Survey—provide inestimably valuable cross-cultural information about people's attitudes on wide variety of topics, including religion. But brevity is the soul of such surveys, and this necessarily leaves psychologists wanting more. Perhaps more can be had.
Last year, Scientific Data published the International Death Survey (IDS), our first foray into cross-cultural psychology of religion, starting with the USA, Brazil, Russia, the Philippines, South Korea, and Japan. The IDS was inspired by our observation of a widespread intuition—among both theorists of religion as well as laypeople—that people might be religious because they fear death. We wanted to know if any evidence could be shored up for or against this idea.
The Negative Life Experiences and Religiosity project was similarly a response to a different common "theory", that people seek religion as a means of coping with adversity. We wanted to know if there was any discernible relationship between adverse experiences and religiosity, and if so, whether there are more complex interactions at play with other aspects of people's upbringing, specific theological beliefs, and psychological well-being. There are many more-or-less formal versions of this general idea that religion is a response to adversity: our dataset is sufficiently diverse to speak to a range of them.
Besides pivoting in terms of content, we also set ourselves the challenge of branching out to different cultures. The IDS sampled majority or plurality Christian countries, with Japan being the exception. Now, we wanted to see if our instruments worked across different religious traditions: we thus collected data from China, India, Indonesia, Thailand, and Turkey, as well as from three countries representing the three main branches of Christianity: the USA (Protestantism) , Brazil (Roman Catholicism), and Russia (Eastern Orthodoxy).
In our previous Behind the Paper post, we mentioned facing challenges with translations, especially of religious terms like "god" into Japanese. Translations into Chinese are similarly perilous. Indeed, even the Pew Research Center has had to retract findings based on their (mis)translation of "god", and Gallup has been criticised for their translation of "religious". Our awareness of these challenges certainly affected our translation process, but perhaps more importantly, it strengthened our commitment to transparency and open science practices.
Psychological measures like the ones we used in this project and the IDS are, or so psychologists claim, richer than single-item attitude measures typically found in social surveys. In this study, we measure not only "how religious" someone is, but also how they might describe their gods, and how involved they believe gods are in the world. In considering the relationship between religiosity and life experiences, these details might matter a great deal. But perhaps another benefit of psychological measures is that they are amenable to psychometric evaluation, including comparisons of each instrument across groups. Roughly speaking, most psychometric evaluations look at how items covary within measures and also how different measures correlate with one another. These tests provide a way—albeit an indirect one—to check the reliability of our translations. If a measure shows very similar psychometric properties across translations, we can be more confident, even if never entirely sure, that the difference versions of the measures capture the same sense. Techniques like multigroup confirmatory factor analysis can be used to model how similarly the measure behaves across translations and populations. Our data descriptor provides some initial psychometric evaluation, but we eagerly invite closer scrutiny.