Kevan Harris, UCLA and Daniel Tavana, Princeton University
This is part of the MENA Politics Newsletter, Volume 2, Issue 1, Spring 2019. Download the PDF of this piece here.
Polls in Iran, when conducted with sound methods, can inform us about a post-revolutionary state with semi-competitive elections and a rapidly changing society. Even government-backed organizations in Iran conduct surveys and relay the findings. The Iranian Student Polling Agency (ISPA), a polling organization founded in 2001 under a state academic supervisory body, the Supreme Council for Cultural Revolution, predicted the outcome of the July 2017 presidential election in Iran within two percentage points of the declared vote returns. The same prediction was also made by an independent polling firm in the days prior to the election.[1] Yet many social scientists who study Iran have been unaware of, or reluctant to use, survey methods, partly due to government limitations imposed on polling organizations in the past. Understandably, many Iranians tend to distrust survey data as well. Or, perhaps more accurately, they tend to distrust survey data unless they see a result which confirms their prior beliefs. Persian-language social media across the political spectrum is peppered with reports of highly-skewed polls, often conducted online with convenience samples, without much discussion of validity or reliability.
For the Iran Social Survey, a phone survey of over 5,000 individuals fielded in November to December 2016 using a nationally-representative probability sample of the population, we aimed to gather a dataset which laid empirical foundations under the abstract concepts often used to explain political and social trends in the Islamic Republic. Both scholarly and popular writings on the country, as with other states in the Middle East and North Africa, deploy terms such as “middle class,” “youth,” “urban,” and “educated” to analytically describe, or even theoretically explain, large-scale outcomes. These concepts do a lot of heavy lifting in social science on Iran and the rest of the MENA region in discussions of electoral behavior and social relations, yet such terms contain assumptions which need to be empirically scrutinized.
While reliance on household phone polls has become increasingly difficult in the United States, with low response rates and higher degrees of sampling error, conditions in Iran today are quite favorable for phone-based surveys. More than 95 percent of Iranian households have a fixed landline phone, and there is a low level of polling saturation among the population. Detailed census data is available for weighting samples as necessary. With a pilot-tested survey instrument, checks on enumerator error, and careful attention to word choice and question order, a survey can produce high response rates and relatively reliable data.
For example, we asked respondents whether they voted in the 2013 Iranian presidential election as well as the 2016 parliamentary election. Reported voter turnout rates in our sample were consistent with official turnout data produced by the Islamic Republic’s Ministry of Interior. In fact, the Iran Social Survey is one of the few studies in which official turnout rates reported after an election in Iran have been independently verified. Combined with respondent variables on demographics, income, education, and residence, we were able to probe long-asserted but rarely tested relationships between electoral behavior and individual characteristics in Iran. Through our data, we found many similarities within political life between Iran and its MENA neighbors. For instance, most Iranians follow political news closely, but few of them personally identify with a national political faction. This helps to explain why, in a country with high levels of voter turnout and widely mobilized electoral campaigns, a substantial degree of vote switching between factions occurs across elections.[2]
Other findings in the Iran Social Survey also align with the secondary literature on Iran. This is especially the case with survey questions which do not easily lend themselves to the possibility of preference falsification to enumerators. For instance, we asked respondents whether their father’s or mother’s families had previously owned any rural land. If so, we asked whether their parents’ families had received any of this land as a result of the pre-revolutionary land reform carried out by the Pahlavi Monarchy, one of the most sweeping land reforms in the MENA region during the postwar era. Of our survey respondents with one or both parents from rural backgrounds, 61 percent answered that their families had received land from the Shah’s redistributive land reform. This figure, albeit one dependent on respondent recollection of a historical process which took place more than four decades ago, before most respondents were born, conforms closely to estimates from the scholarly historiography on land reform in Iran. Given the survey data on family land ownership, we can investigate whether the descendants of families who benefited from pre-revolutionary land reform ended up better-off, the same, or worse-off after the 1979 revolution on a range of social indicators. In countries where rural land reform occurred in tandem with rapid urbanization and mass expansion in public education, the Iranian case can add insight into how families might have converted newly acquired small landholdings into human capital and social status for their offspring, even after moving to urban areas.[3]
As social scientists have increasingly focused on distributive politics across the MENA region, the Iranian case is also informative when unpacking the mechanisms linking political processes to social policy organizations and patronage systems. The Iran Social Survey included questions to test anecdotal claims that electoral behavior in Iran is associated with individual access to state benefits or other forms of social assistance. At least for the 2013 election, we found no evidence that individuals linked to welfare programs associated with conservative politicians or factions were voting differently on average than people linked to welfare programs associated with moderate politicians or factions. Additional data on the scope and breadth of various types of welfare organizations in Iran from the survey call into question existing paradigms which portray the country’s electoral trends as systematically dependent on clientelist mechanisms.[4]
As with
any survey, the quality of the data is partly determined by the design of the
instrument, the method of interview collection, and whether the survey is
attuned to the qualitative meanings attached to questions by respondents. If we
had not spent a good deal of time inside of Iran, conducting qualitative
fieldwork, learning the language, and refining a sense of how everyday Iranians
speak about politics and society, the outcome of a large-scale project like the
Iran Social Survey would have been worse off. As data scientists like to say:
garbage in, garbage out. Instead of simply plugging stock questions from
cross-national surveys into ears at the end of Iranian telephone lines, we adapted
questions from comparative surveys from the Middle East and other developing regions
for the Iranian context while also making sure the data would allow us to
compare Iran with cases in and beyond the Middle East. As most survey
methodologists know, a well-crafted polling instrument is a document thoroughly
informed by qualitative methods. As future surveys are fielded across the MENA
region, whether in person, over the phone, or online, the combination of deep regional
knowledge and theoretical acuity will produce the most innovative data for use
by those who aim to further integrate the region into ongoing debates in social
science.
Notes
[1] Ebrahim Mohseni, Nancy Gallagher, and Clay Ramsay, “The Ramifications of Rouhani’s Re-election: A Public Opinion Study.” (College Park, MD: Center for International and Security Studies at Maryland, July 2017).
[2] Kevan Harris and Daniel Tavana, Voter Behavior and Political Mobilization in Iran: Findings from the Iran Social Survey (Lund, Sweden: European Middle East Research Group, January 2018), https://doi.org/10.26369/RE.2018.001
[3] Kevan Harris and Zep Kalb, “Pen to the Tiller: Land Reform and Social Mobility across the 1979 Iranian Revolution,” forthcoming in the Journal of Agrarian Change.
[4] Kevan Harris, “Unpacking the Welfare-Politics Nexus in the Islamic Republic of Iran.” Project on Middle East Political Science, Briefing 28 (2017): 26-30. For the politics of Iran’s welfare system, see Kevan Harris, A Social Revolution: Politics and the Welfare State in Iran (Oakland, CA: University of California Press, 2017).