Jeremy Wallace

We live in a complex world and tell all kinds of different stories about its machinations. Social science can be thought of as attempting to systematize and improve our knowledge of the way that the world works. Sometimes this involves creating new stories or finding connections between events in vastly different times or places. Often, however, social science is attempting to eliminate stories that clutter our understanding rather than add to it, a spring cleaning for our models of the world.

Two recent posts about getting to the micro-foundations of international relations or conflict research make many salient points about the need for better research designs and better data to help avoid the ecological inference problems. I agree with them but want to open the door a bit wider.

Thomas Zietzoff wrote a fine post for the Political Violence @ a Glance blog calling for more field experiments in international relations and conflict studies to identify the micro-foundations of their arguments. Jay Ulfelder wrote on his Dart-Throwing Chimp blog of his skepticism of field experiments on these topics. Instead, he points to big data’s ability to get at the micro-foundations by examining millions of bits of data that come from the digital trail that we leave behind. I would personally enjoy seeing more use of big data and experiments in political science scholarship, but it is not the case that these are the only tools that can make improvements to our understandings of the world.

IR has classically used countries, country-years, country dyads, and the dreaded dyad-year as the preferred units of analysis. Nation-states are big lumpy things and paying attention only to them when the action is actually taking place at below that level can be problematic. Comparative politics as a subfield has moved increasingly to examining sub-national variation. This trend has begun to pick up in international relations as well. As Zietzoff writes:

Acknowledging the limitations in using cross-national data, many recent, cutting-edge conflict studies (randomized artillery strikes in Chechnya, or civilian casualties in Iraq, etc.) make use of sub-national conflict data. However, like the previous cross-national studies, these recent studies have come to mixed conclusions about the effect of violence. Some find that violence leads to support for concessions, or reduction in insurgent attacks, and others show that it leads to an increase desire for retaliation.

Let me throw in my lot with the “cutting edge” here. Sub-national analysis can dramatically expand the amount of data that can be brought to bear on questions of interest to international relations scholars. To me, that these studies come to mixed conclusions speaks less to the problems of working with non-experimental data than of the complexities of violence and strategic interactions in varying contexts.

These ideas are on my mind because I am finishing (with Jessica Chen Weiss) an initial draft of a paper examining sub-national patterns of Chinese anti-Japanese protest to see how and to what extent the various arguments that are put forward fit with the data. From the research design section of the paper:

Using sub-national data, this paper seeks to evaluate different arguments that have been offered that claim to explain anti-foreign demonstrations. This first requires identifying the observable implications of these arguments at the sub-national level. For instance, venting arguments imply that the regime is interested in allowing angry populations the opportunity to release some of their anger at an external opponent rather than direct their ire at the regime itself. At the sub-national level, we make the claim that cities with higher levels of grievances (however measured) thus should be more likely to have such nationalist protests.

We have collected original data on the presence or absence of protests in all Chinese prefectural-level cities during the time period from mid-August to the end of September 2012. …

We do not have individual-level data on all Chinese urban citizens on their participation in protests and we certainly do not have information on why the protesters believed themselves to be protesting. While the lack of such individual-level information leaves us vulnerable to the claim that our analysis exhibits ecological inference problems (if we were to claim support for the idea that college students are more likely to protest based on city-level data showing that cities with more college students per capita are more likely to have protests), we believe that by examining local variation and comparing political, social, and economic patterns with the patterns of protest we improve on prior analyses that remain rooted at the national level.

Survey experiments about attitudes could certainly supplement this analysis. A “big data”-type analysis of the content of print and internet material could shed a lot of light perhaps as well. (A field experiment testing different causes of protests seems unlikely be workable at a number of levels.) But using “bigger” data can be enough to improve on what we know today.