Aid in the 21st century is increasingly evidence-driven. Between 2000 and 2006, the World Bank spent a total of $630 million on research. By 2011 the World Bank was spending $606 million per year, or about a quarter of its country budgets. In September of this year, by signing up to the Sustainable Development Goals, the global community enshrined a commitment to “increase significantly” a range of high-quality data over the next 15 years, to facilitate qualitative as well as quantitative understandings of growth and progress.
As the international community seeks to tackle the “hard problems” of development—fragility, conflict, endemic poverty—qualitative research is ever-more important. These problems are not amenable to best-practice solutions but must be tackled through deep contextual understanding of their drivers. Or so the policy story goes.[1] As a result, conducting qualitative research today is different from the days when Geertz set out for Bali. Gone are the intrepid individuals setting off to explore and explain an untouched environment, unaware of the demands of policymakers.[2]
We argue that while practice has changed, the ideology of qualitative research has not. Qualitative research is generally understood as the individual exercise of research methods to produce knowledge about the world, knowledge that can then be taken up by governance actors of all stripes. By contrast, we believe that today we must understand research as a systemic intervention, within the broader context of globalization and international development. Therefore, we should start with the political economy of contemporary research—an iterative, professionalized and increasingly saturated practice—to rethink the political and ethical implications of the research that we do.
As a first step to this end, we contrast two stylized frameworks for understanding qualitative research in fragile contexts: The “fragility research” framework, which we argue dominates the current debate; and the “research supply chain” framework, which we offer as a new framework and a provocation to discussion. We discuss each in turn, first considering how fragility research frames knowledge production in fragile or conflicted-affected states, identifying some assumptions the fragility research framework rests on, and critiquing some of its key conclusions. We then discuss the research supply chain as an alternative framework to explore the relationship between knowledge generation and policy. Finally, we raise some questions based on the new framework’s implications.
Orthodox Understandings of Knowledge Production: The “Fragility Research” Framework
“Fragility research” refers to orthodox understandings of knowledge production and policy in fragile states. In this framework, research is a way to identify the problems of “failed” or “fragile” places and contribute to the “evidence-base” that instructs policymakers on where to intervene and how.[3] Development agencies have adopted this as a tool of global governance, using qualitative research along with quantitative data to explain fragile places in all their “deviance” and context-specificity.[4] Thus, research should be conducted to support activities in all spaces, from local level activities all the way up to the policies and projects of international development institutions, state governments, and private individuals or organizations;[5] it should be conducted at all times—from every step of policy processes through to program implementation, including contextual analysis, developing indicators, and monitoring and evaluation. Such intensive research is necessary to refine policy and keep on top of a febrile and constantly-changing context; at the same time, it justifies development interventions anywhere and anytime.[6]
The archetypal[7] fragility researcher is an intrepid individual who embeds himself[8] in local culture, collecting authentic life-stories of the conflict-affected and vulnerable in the most remote corners of the world. He treats these stories as a craftsman, shaping, polishing, and refining them as a corrective to generalized statements of policy and “big data”. These correctives guide and shape policy interventions, as well as the research that accompanies them. His responsibility is thus threefold. First and foremost, it is technical: to conduct methodologically rigorous research. Second, his ethical responsibility is to his research subjects: he must use his individual agency to evaluate real-time situations and the response they call for (whether to compensate research subjects; how to respond to racism, classicism, or sexism in the workplace; whether to report corruption; and so on). Third, his political responsibility is to share his findings, thereby building the evidence-base for policy makers and influencing political, legal, and social structures. He is not responsible, however, for governing the world—his ideas should be taken in light of all other evidence, and implemented by someone else. Fragility research, then, distinguishes between the individual craftsman and his role within a political system.
In sum, the fragility research framework can be described as follows: the global politics of fragility (structure) and the individual craftsmanship of producing research (agency) are linked through method (the processes and strategies a researcher uses to produce his or her product, which in turn defines its type and quality). Correspondingly, the fragility research framework assumes divisions between the political (structural), ethical (agent-based), and technical (methodological) roles and responsibilities of its actors. Critiques within the framework tend to focus on one role as the means to achieve the other two—arguing that the researcher must be more political when thinking about what method to use and when framing her engagement with individual research subjects;[9] or that she should be more ethical (considerate of participants time, for example) to get better data;[10] or that he should have more sophisticated method to tease out the truth and avoid problems of politics.[11] These critiques do not question the underlying logic of the fragility research framework.
Key Trends Shaping International Research: Saturation and Professionalization
Two recent trends in fragility research raise a series of political and ethical questions that the fragility research framework is not equipped to help us answer. The first trend, saturation of the research “field,” is the ever-increasing and iterated presence of researchers in the lives of people who we seek to research. Saturation frequently occurs in poor or marginalized communities that have experienced crisis (e.g. war, genocide, natural disaster) and that are geographically and politically accessible to researchers.[12] The second trend, professionalization of the conflict researcher, is the emergence of a cadre of people who assert a monopoly over a specific set of expertise to research conflict (as well as a group of institutions that hire people on that basis). Part of this process of professionalization of the conflict researcher is a set of research methods meant to help overcome biases inherent in research-saturated environments, such as triangulation and employing local research assistants who can translate local context.
There’s a lot at stake here. Saturation shapes how people respond to researchers. Over repeated experiences as research subjects, “respondents” might instrumentalize research projects and researchers for private ends or political ambition; regurgitate “correct” answers; become antagonistic or cynical towards researchers; and so on.[13] Professionalization shapes standards and benchmarks, the written and unwritten norms, by which researchers conduct and consume research. Together, saturation and professionalization mean that research has a cumulative—and increasingly systematic—impact with significant implications for method, ethics and politics. This is hard to show through the fragility research framework, which concentrates on individual acts of research, and fragments the questions we might ask about three areas of concern: first, how research subjects experience the act of being researched; second, how and to whom researchers view their roles and responsibilities; and third, how research—once produced—is used in the endeavor of global governance. However, the abundance of anecdotal support for issues associated with professionalization and saturation suggests the need for a new framing that places individual acts of research within a broader system or practice (see, for example, Chambers’ work, which pointed to this over two decades ago). We seek to provide initial steps to that end, drawing attention to the fragility research framework’s tripartite division, and offering an alternative framing that emphasizes the interconnections among politics, technique, and ethics.
Some authors have pointed out the importance of linking politics, ethics, and method in the context of saturation and professionalization. For example, Mosse and Lewis focus on the role of national and international brokers of knowledge between the global and the local.[14] They see brokers as entrepreneurs, navigating around the edges of global governance structures, finding a niche—and the jobs that go with it—by working the system, becoming middlemen for the transfer of knowledge. Others, like Jensen and Winthereik, discuss networks and assemblages, emphasizing the lightness of structure.[15] They see researchers as agents, reifying ideas like “fragility” both through research and “translation” for different policy audiences. While these approaches differ in their understanding of the weight of structures of global governance, they share a view of researchers as individuals rather than members of a professional cadre. Additionally, we argue that neither approach takes the impact of iterative research seriously enough.
A Proposal: The “Research Supply Chain” Framework
To provide a new perspective, we offer the research supply chain framework, where research might look something like this:
In a remote corner of the world, recently devastated by violent conflict, a subsistence farmer plucks an organic narrative fresh from the post-conflict field. He examines it, polishing it on his sleeve and testing how firm it is. It must conform to international quality standards, as do the stories cultivated by hundreds of others from the same fields. Just then, a local middleman arrives to collect and consolidate the harvest. He inspects each narrative and records those that meet muster in his notebook. The middleman makes the long journey to market, where an international consultant meets him. The consultant inspects the wares and selects an assortment for processing. At the processing center, the consultant and his team sort the narratives by quality. Some are destined for mass consumption by global policy-makers. The finest and juiciest will be bottled and labeled under the consultant’s own organic reserve brand, preserved for the highest impact academic journals.
This vignette draws attention to the role of the researcher in producing the system within which he works. It places agency in the hands of each person involved in knowledge production—from the producers of narratives, through researcher-farmers, to policymaker-consumers—while recognizing entrenched and institutionalized power disparities. This allows us to acknowledge individual agency while focusing on the systematic, underscoring the costs and benefits each actor accrues.
Five aspects of supply chains are essential to the research supply chain framework. First, a supply chain results from a series of transactional relationships. In the context of research, this might include immediate exchange, but also the promise of future benefit derived from policy beneficial to the research subject and founded on the stories she tells. Second, supply chains deal with alienable commodities (here, knowledge). Third, through the supply chain, purchasers or consumers have the power to set production standards along the chain, and thus to discipline the process of production. Fourth, the supply chain itself circulates, vernacularizes, distorts, and concretizes those market standards into different sticky habits, conventions and norms for actors along the chain.[16] These norms are thus the product of the interplay between the political economy of each step of the chain, and the political economy between the different steps. For the purposes of the research supply chain, we understand these as “professional” norms, as they relate to the control and use of research expertise. They are intersubjective, emerging through a series of transactions: a process of negotiation and bargaining over the substance, quality, and form of the product at each step of the chain. Fifth, chains for different varieties of the same good (organic and conventional tomatoes; policy papers and peer-reviewed journal articles) intersect and separate at different nodes, complicating the emergence of said norms.
To elaborate: in a produce supply chain, large distributors will produce quality standards to which actors all the way down the chain must adhere if they want to participate in that market. The risk of a poor quality product is priced into each transaction. In our analogy, development organizations—the World Bank, the United Nations, USAID, etc.—set standards for acceptable research. Actors who want to participate in this market must adhere to these standards at each step, from production of raw material (narratives and life experiences), to harvesting (field research), processing (analysis), and taking to market (writing and dissemination of findings). Those who do not adhere will be excluded from the market—consultants with poor methodological training won’t get contracts; villagers with “incoherent” life stories or the inability to communicate them in a manner consistent with the methodology du jour will be dropped out of the study; research assistants who fail to add value will be blacklisted or simply not recommended for future work. The obverse is also true: those who do adhere to the rules and regulations can fight for market share and capture their portion of the supply chain. Different researchers speak to the same, articulate villager, identified by a seasoned local research assistant; research assistants with good networks get hired repeatedly, further bolstering their social value as a patron and retrenching their connections.
The research supply chain relies on the fantasy of the “organic” and unadulterated data. Consider, for example, an anthropologist who distinguishes herself from policy-driven researchers. She uses craft to carefully engage with subjects, pulling out the “unvarnished truth” for its own sake, to give voice to the voiceless or to critique prevailing global governance orthodoxies. Our analogy suggests that authenticity is always already shaped by the market conditions that it creates. People narrate themselves with an eye towards what the researcher wants—which in turn is shaped by consumers, like development agencies and the academy. Further, researchers interpret individual narratives to be comprehensible to those consumers, thereby relating them to these pervasive narratives. These products are power—over policy, over the distribution of resources, over the discourses that frame the role of global governance in people’s lives.
The promise of the organic narrative paradoxically motivates the emergence of a research industry and of research saturation. It produces the notion that a pure product exists if we only spend enough time to get it. Thus, certain researchers articulate themselves as sources and suppliers of organic narratives (e.g. anthropologists), contrasting themselves with other researchers who work in mass production (e.g. drop-in consultants). The research supply chain suggests that these researchers are actors in the same market. All research is affected by the emergence and outcome of professional norms, not just development organizations or fly-in, fly-out consultants. Each researcher enables the other, creating a niche similar to market differentiation.
Researching Research Frameworks: Ethics Workshops in Gulu, Uganda
In December 2014, we held two workshops in Gulu, Uganda, to explore some of our intuitions and initial ideas that emerged from several years of working at the nexus of research and development. How widespread were the challenges posed by market conditions for research? Could they be expressed systematically, or were they still embedded in the fantasy of the organic narrative?
Over two days, we gathered approximately 20 Ugandan and foreign researchers to discuss some of the ethical questions we’ve posed here, focusing on the relationship between foreign researchers and local research assistants. This relationship is a key one in the research supply chain. It is explicitly contractual. The research assistant is paid to translate context. At times, the researcher further relies on that translation to make ethical choices (whether to pay research subjects for participation, how to respond to corruption in the research institution…).
International researchers energetically described ethical challenges, discussing the compromises they have to make on a day-to-day basis as part of the job. Several repeatedly raised the notion of “bringing research back” to communities they had studied as a key ethical move, a way of validating research when they could not promise direct benefits from the research itself. For some, “bringing research back” described an ongoing process of engagement with research subjects that began well before completion of the research and which continued long after. For others, “bringing research back” entailed presenting a final product, the last opportunity for subjects to participate in shaping the product, and symbolizing the end of field research, a form of closure of the relationships between researcher and researched, where ethical obligations similarly would cease. In the research supply chain, either approach to bringing research back can be understood as a move to depoliticizes the researcher: it circumscribes relationships and obligations by focusing on research findings, reified in a physical document, that become the site of political struggle. What relationships remain exist as a matter of individual engagement and ethical choice. From this perspective, bringing research back can distract us from the lingering and systematic effects that the research process might have on local power structures or individual narratives, which are instead a matter for future researchers to take into account when designing their methodology.
Ugandan researchers, by contrast, remain proximate to—and frequently part of—those lingering effects after the research is over. Working with internationals is a job. They assured us that they would tell their international employers if anything ethically problematic came up and established their bonafides with stories of the unscrupulous practices of other foreign researchers. At the end of the day, however, a number of participants pointed out that to raise questions about a potential employer’s project would be bad for business. Although we set ethical standards at the global level, and rely on local researchers to implement them in a context specific way, the research supply chain highlights the improbability of this. Local researchers have a doubled consciousness. They see international researchers as both individuals and as part of a broader system—sources of repeat work and possible connections to other employment opportunities.
| We asked our participants to “draw their research environment.” Top: a Ugandan researcher drew an image of the research process as a chain (with no prompting from us). Bottom: an international researcher illustrated the contradictions between submerging herself in the context while at the same time standing over and moving through it. |
Provocations, Questions, & Preliminary Conclusions
The notion of the research supply chain denaturalizes the relationships between qualitative research and policy that the fragility research framework assumes. We international researchers often view our work as giving voice to the voiceless, empowering the disempowered, or simply finding some underlying method to the madness of conflict and fragility. By contrast, a supply chain framework flattens the nature of human relationships along the chain to bargains and contracts, an idea that we think is increasingly apt. In doing so, the framework suggests that norms around research are not static and should not be taken for granted. Instead, these norms, and the distinction between political and ethical responsibilities, are dynamic, contested, and constantly produced.
The implications of the framework may be uncomfortable for conflict researchers—indeed, it’s partly this discomfort that bolsters our conviction that there are important questions. The descriptive aspects of the supply chain framework enable us to pose a set of empirical questions, while the metaphorical aspects allow us to pose a set of conceptual and theoretical questions that we cannot ask when viewing research as an individual endeavor. For example:
- Empirical: how do different levels of saturation impact which narratives are produced? How do norms around “professional” research emerge and concretize? How does the market for research influence funding streams, e.g. which projects get funded?
- Ethical: how should each individual in the market for research manage their relationship with those above and below them, in light of the supply chain? For example, to what extent is it okay to outsource aspects of production and quality control to research assistants; to what extent is there an ethical imperative to “bring research back,” and why?
- Political: how should accountability be apportioned in the market for research, on what grounds (legal, political, social, methodological) and through what institutions (courts, legislature, peer review, etc.)?
The relationship between the metaphorical and descriptive is not static. As we uncover more about how research functions, we can further explore some of the implications that follow from this framework and recognize its limitations. For example, what does it mean to think of knowledge as a commodity and research as simply transactional (ground that has been well-trodden in the context of research for industry but less so for global governance and fragility)?[17] Does this framework unfairly critique the possibility of genuine human interaction and motivations to “give voice” or pursue human engagement? How does a supply chain function in relation to a consumer that is a public authority? In exploring these questions, the nature of the framing may shift toward or away from the metaphorical.
We have proposed the research supply chain here as a means of stimulating discussion and building links with those who share similar concerns. Of course, different framings of the problem may similarly allow us to engage with how professional norms are set and hold ourselves—and the system—to account.[18] For example, what are the risks and rewards if we turn to law, such as in the O Case, where an Ethiopian national attempted to sue the British government for inadequate human rights due diligence during a development intervention? This symposium is, we hope, a step towards an agenda of research and debate that might lead to a change in the way we conceive of and conduct research in fragile contexts.
Notes
We are grateful to the Program on Negotiation and the Justice and Security Research Program for funding the workshops that contributed to this piece. We are also grateful to the six respondents for their careful and generous contributions; as well as to David Booth, Todd Foglesong, Julian Hopwood, Tim Kelsall, Hamish Nixon, Mareike Schomerus, and Susan Weinstein for their thoughts and comments. All errors remain our own.
[1] Matt Andrews, Lant Pritchett and Michael Woolcock (forthcoming), Building (Real) State Capability: The Implementation Imperative.
[2] Akhil Gupta and James Ferguson, “Discipline and Practice: ‘The Field’ as Site, Method, and Location in Anthropology,” in Anthropological Locations: Boundaries and Grounds of a Field Science, eds. Akhil Gupta and James Ferguson (Berkeley: University of California Press, 1997).
[3] Mariana Valverde and Michael Mopas, “Insecurity and the Dream of Targeted Governance,” in Global Governmentality: Governing International Spaces, eds. Wendy Larner and William Walters (Oxford: Routledge, 2004).
[4] Christopher Cramer and Jonathan Goodhand, “Hard science or waffly crap? Evidence-based policy versus policy-based evidence in the field of violent conflict,” in The Political Economy of Development: The World Bank, Neo-Liberalism and Development Research, eds. Kate Bayliss, Ben Fine, and Elsa van Waeyenberge (London: Pluto Press, 2011); Michael J Watts, “Economies of Violence: Reflections on the World Development Report 2011,” Humanity 3, no. 1 (2012): 115-30.
[5] David Mosse, “Global Governance and the Ethnography of International Aid,” in The Aid Effect: Giving and Governing in International Development, eds. David Mosse and David Lewis (London: Pluto Press, 2005).
[6] Casper Bruun Jensen and Brit Ross Winthereik, Monitoring Movements in Development Aid: Recursive Partnerships and Infrastructure (Cambridge, MA: MIT Press, 2013).
[7] Gupta and Ferguson, “Discipline and Practice.”
[8] When we opt to use the masculine pronoun, it is for simplicity. It is not a statement about gender.
[9] On politicizing method, see James C Scott, Weapons of the Weak: Everyday Forms of Peasant Resistance (New Haven: Yale University Press, 1985); Robert Chambers, Whose Reality Counts?: Putting the First Last (London: Intermediate Technology Publications, 1997). On politicizing ethical obligations to the research subject, see Paulo Freire, “Teachers as Cultural Workers: Letters to Those who Dare Teach,” in Handbook of Research on Teacher Education: Enduring Questions in Changing Contexts, 3rd ed., eds. Marilyn Cochran-Smith, Sharon Feiman-Nemser, D. John McIntyre and Kelly Demers (New York: Routledge, 2008).
[10] For example, Paul Rabinow, Reflections on Fieldwork in Morocco (Berkeley and Lost Angeles, CA: University of California Press, 1977); Gillian Rose, “Situating knowledges: positionality, reflexivities and other tactics,” Progress in Human Geography 21, no. 3 (1977): 305-20.
[11] https://d-lab.mit.edu/lean-research
[12] For example, Mayssoun Sukarieh & Stuart Tannock, “On the Problem of Over-researched Communities: The Case of the Shatila Palestinian Refugee Camp in Lebanon,” Sociology 47, no.3 (2013): 494-508 (writing about their experiences researching refugee camps in Lebanon).
[13] Ibid.
[14] David Mosse and David Lewis, eds., Development Brokers and Translators: The Ethnography of Aid and Agencies (Bloomfield, CT: Kumarian, 2006).
[15] Jensen and Winthereik, Monitoring Movements in Development Aid.
[16] Laura Raynolds, “The Globalization of Organic Agro-Food Networks,” World Development 32, no. 5 (2004): 735-43.
[17] Tomas Hellström and Sujatha Raman, “The commodification of knowledge about knowledge: Knowledge management and the reification of epistemology,” Social Epistemology 15, no. 3 (2001): 139-54.
[18] Nick J Fox and Pam Alldred, “New materialist social inquiry: designs, methods and the research-assemblage,” International Journal of Social Research Methodology 18, no. 4 (2015): 399-414.
