Measuring Culture with Facebook

"Expanding the Measurement of Culture with a Sample of Two Billion Humans." Nick Obradovich, Ömer Özak, Ignacio Martín, et al. IZA Discussion Paper No. 13735, September 2020.

Economists often seek to determine whether culture affects broad macroeconomic outcomes or how it affects the decision-making of individuals. Gianmarco Ottaviano and Giovanni Peri, for example, sought in a 2006 journal article to explore the role of cultural diversity on productivity in U.S. cities. The measurement of culture itself, however, is complex and often limited in scope.

In an IZA working paper released in September 2020, researchers at the Max Planck Institute for Human Development, Southern Methodist University, the Spanish firm Nommon Solutions and Technologies, Universidad Carlos III, and the University of Exeter Business School tried a new approach: using social media to create a new measure of culture. They explored how this measure compares with existing metrics and how it might be used to answer complex research questions.

The authors noted that, with the onset of the information age, there are now vast amounts of data on human behaviors and interests being collected worldwide. In comparison, traditional methods for measuring culture are much more limited in their scope. Surveys, in particular, are subject to resource limitations (a limited number of people can be surveyed), measurement error (via self-reporting or researcher observation), and researcher bias (for example, researchers picking questions on the basis of their own cultural assumptions and experiences).

In response to these limitations, the authors turned to the Facebook Marketing Application Programming Interface and measured human culture using information on approximately two billion Facebook users across 225 countries and territories. Facebook analyzes user behavior to associate interests with each user; marketers can see how many users are associated with each interest given a geographic area and other demographic characteristics. For example, the researchers observed that, within the state of Virginia, a higher share of Facebook users was interested in "apple cider" than in any other state.

There are hundreds of thousands of possible interests on the Facebook platform. Of these, the authors chose to focus on the 60,000 interests with the largest number of users. For each geographic area, they measured the share of Facebook users who held a given interest. Using these data, the authors calculated the cultural distance between any two regions based on the attributes of the Facebook users in each region.

This allowed the authors to create a "bottom-up" measure of culture. Rather than imposing a narrow set of ideas of what makes culture, they observed naturally occurring similarities and differences across human groups.

To validate their measurement, they showed that the cultural distances they calculated between geographic areas using Facebook were positively correlated with distances calculated from existing measures of culture, such as the World Values Survey, a worldwide survey of cultural values. They also used a machine learning algorithm to test whether the data from Facebook could predict observed cultural values (for example, on gender roles) and found high correlations between the predictions and the observed values. They also showed that clustering — a type of machine learning — could generate country groupings that proved to be consistent with general knowledge of history and culture.

After validating their measure, the authors demonstrated its potential uses, paying special attention to the measurement of culture in subnational regions, which they argued is cost prohibitive using traditional means of measurement. They explored whether or not national borders matter for cultural dissimilarity, and they measured which countries have more dissimilarities among internal regions. Again using unsupervised clustering, the authors were able to create natural groupings of both national and subnational areas based on cultural similarity. They charted the grouping of all the countries in their sample and of U.S. states. They also explored cultural distance along other lines, looking at relationships among regional, gender, and age divisions.

Having explored various uses for their new measure of human culture, the authors acknowledged limitations to their approach. Not everyone uses Facebook. Moreover, groupings of interests that differ among regions might be difficult to interpret; unlike traditional cultural measures, which rely on everyday concepts like religion or art, clusters of Facebook interests may not have easily understood meanings. Thus, they saw their measure as complementary to existing traditional methods of understanding culture, rather than as a replacement. Even with these limitations, though, the authors argued that data culled from Facebook might be used to answer new research questions on topics as diverse as outcomes of immigration and issues of political stability.