Maxwell School

Institute for Qualitative and Multi-Method Research – June 16-27, 2014

Schedule and Reading List  

(download pdf version)

There are three types of institute sessions: (1) Unified (whole institute) sessions; (2) research design discussion groups; and (3) elective modules. The unified sessions are on the first Monday (6/16), the first Wednesday (6/18), and the first Friday (6/20). 

The research design discussion groups will be held for two hours on most mornings of the institute. A separate schedule will be available.

There are 26 elective modules, of which participants will select eight; that is, they will choose one of the three modules that are offered as triples (e.g. modules 4, 5 or 6) or as quadruples (e.g. 13, 14, 15 or 16). 

Monday, June 16

Unified (whole institute) sessions on the foundations of qualitative and multi-method inquiry – Andrew Bennett, James Mahoney, Lisa Wedeen, Jason Seawright, and David Waldner

Tuesday, June 17

Module 1 Comparative Case Study Methods –  Andrew Bennett and James Mahoney

Module 2 Discourse Analysis - Lisa Wedeen and Jennifer Pitts

Module 3 Quantitative and Qualitative – Jason Seawright

Wednesday, June 18

Unified (whole institute) sessions on process tracing - Andrew Bennett and James Mahoney

Thursday, June 19

Module 4: Natural Experiments — Thad Dunning and Daniel Hidalgo

Module 5: Interpretive Methods for Archival and Historical Research —Thomas Dodman and Daragh Grant

Module 6 Designing and Conducting Fieldwork: Preparing and Operating in the Field – Diana Kapiszewski and Lauren MacLean

Friday, June 20

Module 7: Natural Experiments — Thad Dunning and Daniel Hidalgo

Module 8: Interpretivism, Discourse Analysis, National Identity, and IR Theory— Ted Hopf

Module 9 Designing and Conducting Fieldwork: Collecting and Analyzing Data – Diana Kapiszewski and Lauren MacLean

Monday, June 23

Module 10 – Set Theory I - Gary Goertz and Claudius Wagemann

Module 11 Ethnography I, Fred Schaffer and Timothy Pachirat

Module 12 Managing and Sharing Qualitative Data – Diana Kapiszewski and Dessislava Kirilova

Tuesday, June 24

Module 13 – Set Theory II- Gary Goertz and Claudius Wagemann

Module 14 Ethnographic Methods II, Frederic Schaffer and Timothy Pachirat

Module 15 Computer-Assisted Content Analysis I, Will Lowe and Sven Oliver Proksch

Module 16 Mixed-method research and causal mechanisms, part 1 – Nicholas Weller

Wednesday, June 25

Module 17 – Set Theory III - Gary Goertz and Claudius Wagemann

Module 18 Ethnographic Methods III, Frederic Schaffer and Timothy Pachirat

Module 19 Computer-Assisted Content Analysis II, Will Lowe and Sven Oliver Proksch

Module 20 Mixed-method research and causal mechanisms, part 2– Nicholas Weller

Thursday, June 26

Module 21  Challenges of Medium-N Analysis I – David Collier, Bear Braumoeller, and Christopher Krogslund

Module 22  Geographic Information Systems I: Introduction to GIS as a Qualitative Research Method – Jonnell Robinson

Module 23  Counterfactual Analysis: Jack Levy and Frank Harvey

Friday, June 27

Module 24  Challenges of Medium-N Analysis II – David Collier, Burt L. Monroe, Bear Braumoeller, and Christopher Krogslund

Module 25  Geographic Information Systems II: Exploring GIS Analytic Capabilities – Jonnell Robinson

Module 26  Archival Research and Elite Interviews - James Goldgeier, Andrew Moravcsik, and Elizabeth Saunders


Choosing Which Modules to Take

While many of the 26 modules can be taken as stand-alone units, there are some limitations on selections.

Modules with higher numbered suffixes (e.g. Content Analysis II) can usually only be taken with the first module in the sequence (e.g. Content Analysis I). [That is, while it is often fine to take I and not II in a sequence, it is usually not possible to take II and not I.] The three exceptions to this rule are Module 9 (Fieldwork II), Module 13 (Set Theoretic Approaches II), and Module 24 (Challenges of Medium-N Analysis II).

Modules 11, 14, and 18 (Ethnographic Methods I, II, and III) should be considered as a single unit, and accordingly can only be selected together (i.e. participants cannot take only Ethnographic Methods I, or I and II).

Apart from these formal limitations, we should also note that there are several modules which follow in a natural sequence and/or lend themselves to being taken as a group. For the avoidance of doubt, we outline these informal sequences simply to help you navigate the table above. Beyond the two limitations we mention above, you may take whichever modules you would find most helpful.

Module 3 (Quantitative and Qualitative), Modules 4 and 7 (Natural Experiments I and II), Modules 16 and 20 (Mixed-method research and causal mechanisms I and II), and Modules 21 and 24 (Medium N Research I and II).

Module 2 (Discourse Analysis), Module 5 (Interpretive Methods for Archival Research), Module 8 (Interpretivism, Discourse Analysis, National Identity, & IR Theory), and Modules 11, 14, and 18 (Ethnographic Methods I, II, and III).

Modules 6 and 9 (Designing and Conducting Fieldwork I and II), Module 12 (Managing and Sharing Qualitative Data) or Modules 11, 14, and 18 (Ethnographic Methods I,  II, and III), and Module 26 (Archival Research and Elite Interviews).

Books to Purchase or Otherwise Obtain

The reading for some unified sessions and modules includes a book or books that must be purchased, or borrowed from your university library [please note that they are unlikely to be available at the SU bookstore or library].  You will also see that there is some overlap:  some books are used in more than one module.

Manuscripts in Press or in Progress

To the extent possible, IQMR uses the most up-to-date readings on the methods covered at the institute. One consequence is that we are often using manuscripts that are either in press or in progress.  Please note that the authors are allowing us to use these materials as a courtesy. As with all IQMR materials, they are made available for current attendees’ use only.


Monday, June 16 Module 0 Unified” (i.e. whole institute) sessions

U1 8:30am – 9:00am – Introduction and Logistics  

Colin Elman, Syracuse University

  • U1.1. David Collier and Colin Elman, “Qualitative and Multimethod Research: Organizations, Publications, and Reflection on Integration.” In Janet Box-Steffensemeir, Henry Brady, and David Collier, eds., Oxford Handbook of Political Methodology (Oxford University Press, 2008), pp. 779-795.
  • U1.2 James Mahoney, “After KKV: The New Methodology of Qualitative Research,” World Politics 62(1) (January 2010): 120-147.

U2 9:00am-10:00am Within Case and Small-N Comparisons

Andrew Bennett, Georgetown University

This session introduces some of the philosophy of science behind case study methods, situates these methods vis-a-vis other methods, briefly introduces process tracing and case comparisons, and provides practical advice on avoiding ten common errors in case study research design.

  • U.2.1. Alexander George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (The MIT Press, 2005), Preface and chapter 1. (Book to purchase)

Recommended:

  • U.2.2. Henry Brady, “Causation and Explanation in Social Science.” In Janet Box-Steffensmeir, Henry Brady, and David Collier, eds., Oxford Handbook of Political Methodology (Oxford University Press, 2008), pp. 217-270.

10:00am - 10:30 am Coffee Break

U3 10:30am – 11:45pm Logic and Qualitative Methods

James Mahoney, Northwestern University

This session introduces the idea that logic and set theory constitute one important set of tools used in qualitative research.

  • U.3.1. Gary Goertz and James Mahoney, “Methodological Rorschach Tests: Contrasting Interpretations in Qualitative and Quantitative Research,” Comparative Political Studies 46(2) (November 2013): 236-251.

11:45 - 1:45pm Lunch

U4 1:45pm -2:45pm Statistical/multi-method strand

Jason Seawright, Northwestern University

  • U.4.1. David A. Freedman, “On Types of Scientific Enquiry: The Role of Qualitative Reasoning.” In Janet Box-Steffensemeir, Henry Brady, and David Collier, eds., Oxford Handbook of Political Methodology (Oxford University Press, 2008), pp. 300-318.
  • U.4.2. Evan Lieberman, “Nested Analysis as a Mixed-Method Strategy for Comparative Research,” American Political Science Review 99 (August 2005): 435-452.

U5 2:45pm - 3:45pm The Interpretive Approach to Qualitative Research

Lisa Wedeen, University of Chicago

  • U.5.1. Clifford Geertz, "Thick Description: Toward an Interpretive Theory of Culture" in The Interpretation of Cultures (Basic Books, 1973).
  • U.5.2. Clifford Geertz, "Deep Play: Notes on the Balinese Cockfight" in The Interpretation of Cultures (Basic Books, 1973).
  • U.5.3. Michel Foucault, "The Body of the Condemned" in Discipline and Punish: The Birth of the Prison (Vintage Books, 1995). (That's the second edition; the 1979 first edition is fine too).
  • U.5.4. Michel Foucault, “Questions of Method.” In Graham Burchell, Colin Gordon, and Peter Miller, eds., The Foucault Effect: Studies in Governmentality (University of Chicago Press, 1991).

3:45pm – 4:15pm Coffee Break

U6 4:15pm - 5:30pm Roundtable on “How Do We Bring All of this Together?” The Implications of Multiple Approaches to Qualitative and Multi-Method Research

Lisa Wedeen, James Mahoney, Andrew Bennett, Jay Seawright, David Waldner         

   

Tuesday, June 17 Module 1 Comparative Case Study Methods – Andrew Bennett and James Mahoney

8:45am - 10:15am  Comparative-Historical Analysis

James Mahoney, Northwestern University

This session provides an introduction to comparative-historical analysis in contemporary social science.  It explores three orientations associated with this approach:  macro-configurational explanation; case-based research; and temporally-oriented analysis.  It contrasts these orientations with other approaches in contemporary social science.

  • 1.1.1. Tulia Falleti and James Mahoney, "The Comparative Sequential Method." Manuscript prepared for James Mahoney and Kathleen Thelen, eds., Advances in Comparative Historical Analysis: Resilience, Diversity, and Change under review at Cambridge University Press.
  • 1.1.2. James Mahoney, Erin Kimball, and Kendra Koivu, “The Logic of Historical Explanation in the Social Sciences,” Comparative Political Studies 42(1) (January 2009): 114-146.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                     

2:00pm - 3:30pm Typological Theories

Andrew Bennett, Georgetown University 

This session discusses how typological theorizing can help address various kinds of complexity.  It outlines procedures for building typological theories and using them to assist in the selection of cases for purposes of comparison and process tracing.  The session concludes with examples and exercises.

  • 1.2.1. Alexander L. George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (The MIT Press, 2005), Chapter 11. (Book to purchase)
  • 1.2.2. Colin Elman, “Explanatory Typologies and Property Space in Qualitative Studies of International Politics,” International Organization 59(2) (April 2005): 293-326.
  • 1.2.3. Andrew Bennett, “Causal Mechanisms and Typological Theories in the Study of Civil Conflict.” In Jeff Checkel, ed., Transnational Dynamics of Civil War (Cambridge University Press, 2013), pp. 205-231.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm   Causality and Time: A Diagram Approach

James Mahoney, Northwestern University

This session explores how diagrams can be used to illustrate ideas concerning causality and time in qualitative analysis.

  • 1.3.1. James Mahoney and Rachel Sweet Vanderpoel, “Set Diagrams and Qualitative Research,” Comparative Political Studies, forthcoming.

Tuesday, June 17 Module 2 Discourse Analysis - Lisa Wedeen and Jennifer Pitts

This module provides students with an introduction to three different modes of discourse analysis. Participants will learn to "read" texts while becoming familiar with contemporary thinking about interpretation, narrative, and social construction. In these three sessions we shall explore the following methods: Foucault’s “interpretive analytics”; Wittgenstein’s understanding of language as activity and its relevance to ordinary language-use analysis (including theories of “performativity”); and the techniques of the Cambridge school.

8:45am - 10:15am Wittgenstein and Ordinary Language-Use Analysis

Lisa Wedeen, University of Chicago

This session introduces participants to Ludwig Wittgenstein’s thought and its relationship to ordinary language-use methods. We shall focus on several key ways in which Wittgensteinian-inspired methods can be used in ethnographic and analytical research. Among the questions we shall ask are: What is the “value added” of concentrating on language? Why is understanding language as an activity important? How can social scientists grapple with vexed issues of intention? What does “performative” mean, and how do political theories about language as performative differ from discussions of performance? How can social scientists uninterested in taking on new jargon use this kind of political theory to further their theoretical and empirical work?

  • 2.1.1. Hanna Fenichel Pitkin, “Justice, Socrates and Thrasymachus” in Wittgenstein and Justice: On the Significance of Ludwig Wittgenstein for Social and Political Thought (University of California Press, 1972).
  • 2.1.2. Lisa Wedeen, Peripheral Visions: Publics, Power, and Performance in Yemen (University of Chicago Press, 2008), Chapter 2, chapter 3, and conclusion. (Book to purchase)
  • 2.1.3. Ludwig Wittgenstein, The Philosophical Investigations, G. E. M. Anscombe, trans. (Blackwell Publishers, 2001), Paragraphs 1-33; paragraph 154; pages 194-195.

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Cambridge/Skinnerian Discourse Analysis

Jennifer Pitts, University of Chicago

This session will consider the so-called linguistic turn in the history of political thought, through some key statements by Quentin Skinner, as well as attention to two sources on which he has drawn, Collingwood’s idea of questions and answers, and Austin’s arguments about intention. We will consider such questions as: How do we go about reconstructing the questions that a given thinker is asking? What does it mean (and is it possible) to recover or articulate the intentions of an author? How does this differ from seeking to establish the meaning of a text? Why is the recovery of contexts important for these tasks, how do we know which contexts to recover, and has the approach been too focused on intellectual contexts at the expense of other relevant contexts?

  • 2.2.1. Quentin Skinner, “Introduction: Seeing Things Their Way,” pp. 1-7, and “Interpretation and the Understanding of Speech Acts,” pp. 103-127, in Visions of Politics (Cambridge University Press, 2002).

Recommended

  • 2.2.2. J. L. (John Langshaw) Austin, How To Do Things With Words (Harvard University Press, 1962), pp. 94-120.
  • 2.2.3. Quentin Skinner, “The Rise of, Challenge to, and Prospects for a Collingwoodian Approach to the History of Political Thought.” In Dario Castiglione and Iain Hampsher-Monk, eds., The History of Political Thought in National Context (Cambridge University Press, 2001), pp. 175-188.
  • 2.2.4. David Scott, “Prologue” in Conscripts of Modernity: The Tragedy of Colonial Enlightenment (Duke University Press, 2004), pp. 1-22.

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Foucauldian Discourse Analysis

Lisa Wedeen, University of Chicago

This session introduces participants to the techniques of Foucauldian discourse analysis or “interpretive analytics.” Participants will learn how to conduct a discourse analysis, what the underlying assumptions of such an analysis are, and how these techniques can be used to advance political inquiry. The session will consider both the power and limitations of the method, the ways in which it differs from other modes of interpretation, and its advantages over content analysis.

  • 2.3.1. Michel Foucault, “Nietzsche, Genealogy, History” in Language, Counter-Memory, Practice: Selected Essays and Interviews, Donald F. Bouchard, ed., Donald F. Bouchard and Sherry Simon, trans. (Cornell University Press, 1977), pp. 139-164.
  • 2.3.2. Michel Foucault, The History of Sexuality, Vol. 1, Robert Hurley, trans. (Vintage Books, 1990), pp. 1-35 and pp. 92-114.

Recommended

  • 2.3.3. Hubert L. Dreyfus and Paul Rabinow, Michel Foucault: Beyond Structuralism and Hermeneutics (University of Chicago Press, 1983), Part Two.
  • 2.3.4 Revisit King, Keohane, and Verba’s Designing Social Inquiry and bring this text to class. Gary King, Robert O. Keohane, and Sidney Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research (Princeton University Press, 1994).

Tuesday, June 17 Module 3 Quantitative and Qualitative – Jason Seawright

This module provides students with an introduction to research designs that combine a qualitative and a quantitative component in the service of a single causal inference: multi- or mixed-method designs. We will discuss older “triangulation” ideas about multi-method design but focus on the newer “integrative” approach that uses one method to test the assumptions of the other. The module will explore motivating ideas about causation, causal inference, and the strengths of various methods. However, the center of gravity in the module is on considering formal multi-method research designs combining case studies with regression, matching, natural experiments, and randomized experiments.

8:45am - 10:15am Causation and Causal Inference

Jason Seawright, Northwestern University

This session introduces participants to key ideas about causation and causal inference that drive contemporary statistical and multi-method thinking, centrally including the potential outcomes framework. We will discuss that framework, considering what it captures and omits from other ideas about causation. Centrally, we will discuss the way that the potential outcomes framework opens opportunities for multi-method research by specifying the assumptions needed to get causal results out of regression analysis.

  • 3.1.1. Jason Seawright, “Multi-Method Social Science.” Manuscript, Chapters 1-3.

Recommended

  • 3.1.2. Stephen L. Morgan and Christopher Winship, Counterfactuals and Causal Inference: Methods and Principles for Social Research (Cambridge University Press, 2007), Chapters 2 and 5.

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Combining Regression-Type Studies with Case Studies

Jason Seawright, Northwestern University

This session will carefully consider the central question in most discussion of multi-method research: how to combine regression-type studies with case studies. We will discuss when case studies can independently nail down causal inferences, and also how they can contribute tests for each of the key regression assumptions discussed in the previous session. Optimal case-selection strategies for each design will be analyzed.

  • 3.2.1. Jason Seawright, “Multi-Method Social Science.” Manuscript, Chapters 4-5.

Recommended:

  • 3.2.2. David Card, “The Impact of the Mariel Boatlift on the Miami Labor Market,” Industrial and Labor Relations Review 43(2) (January 1990): 245-257.
  • 3.2.3. Adam Glynn and Nahomi Ichino, “Using Qualitative Information to Improve Causal Inference,” American Journal of Political Science, forthcoming.

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Multi-Method Designs Involving Matching, Natural Experiments, and Randomized Experiments

Jason Seawright, Northwestern University

This session introduces participants to multi-method designs that include more recent, and sometimes more credible, quantitative components: matching, natural experiments, and randomized experiments. For each design, we will look at the assumptions needed for causal inference, identify relevant case-study designs, and explore case selection.

  • 3.3.1. Jason Seawright, “Multi-Method Social Science.” Manuscript, Chapters 6-8.

Recommended:

  • 3.3.2. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press, 2012), Chapter 7.
  • 3.3.3. Lawrence W. Sherman and Heather Strang, “Experimental Ethnography: The Marriage of Qualitative and Quantitative Research,” The ANNALS of the American Academy of Political and Social Science 595(1) (September 2004): 204-222.

Wednesday, June 18 Unified (i.e. whole institute) sessions- Process Tracing – David Waldner, Andrew Bennett, and James Mahoney

8:45am - 10:15am U7 - Process Tracing I

Andrew Bennett, Georgetown University 

This session discusses the philosophy of science foundations of process tracing, outlines the logic of process tracing in terms of Bayesian analysis, and provides practical advice on carrying out process tracing.  The session concludes with exercises and examples that allow the students to practice process tracing.

  • U.7.1. Andrew Bennett, and Jeffrey Checkel, eds., Process Tracing in the Social Sciences (Cambridge University Press, forthcoming), Chapter 1 and the appendix.  

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm Lunch.                     

2:00pm - 3:30pm U8   Process Tracing II: The Logic of Process Tracing Tests

James Mahoney, Northwestern University

This session provides a framework, based on set theory and sequential analysis, for carrying out process tracing tests. 

  • U.8.1. David Collier, “Understanding Process Tracing,” PS: Political Science and Politics 44(4) (October 2011): 823-830.
  • U.8.2. James Mahoney, “The Logic of Process Tracing Tests in the Social Sciences,” Sociological Methods and Research 41(4) (March 2012): 566-590.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm  U9  Process Tracing III: Process Tracing and Qualitative Causal Inference

David Waldner, University of Virginia 

This session considers the foundations of qualitative causal inference via process tracing.  It addresses the ‘fundamental challenge of causal inference,’ or the problem of drawing unit-level causal inferences given unobserved counterfactuals.  Drawing on some exemplars of comparative politics, it suggests how process-tracing analysis can meet this challenge by demonstrating the correspondence between causal graphs and event-history maps.

  • U.9.1. David Waldner, “Process Tracing and Causal Mechanisms.” In Harold Kincaid, ed., The Oxford Handbook of Philosophy of Social Science (Oxford University Press, 2012), pp. 65-84.
  • U.9.2. David Waldner, “What Makes Process Tracing Good? Causal Mechanisms, Causal Inference, and the Completeness Standard in Comparative Politics.” In Andrew Bennett and Jeffrey T. Checkel, eds., Process Tracing in the Social Sciences: From Metaphor to Analytic Tool (Cambridge University Press, forthcoming), Chapter 4.

Thursday, June 19 Module 4: Natural Experiments — Thad Dunning and Daniel Hidalgo

8:45am - 10:15am Introduction to Natural Experiments

Thad Dunning, University of California, Berkeley and

Daniel Hidalgo, Massachusetts Institute of Technology

What are natural experiments? We introduce the concept of natural experiments and discuss their strengths and limitations through a survey of recent examples from political science and economics.

  • 4.1.1. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press, 2012), Chapters 1-4. (Book to purchase)
  • 4.1.2. Rafael Di Tella, Sebastian Galiani, and Ernesto Schargrodsky, “The Formation of Beliefs: Evidence from the Allocation of Land Titles to Squatters,” Quarterly Journal of Economics 122(1) (February 2007): 209–241.
  • 4.1.3. Daniel Posner, “The Political Salience of Cultural Difference: Why Chewas and Tumbukas are Allies in Zambia and Adversaries in Malawi,” American Political Science Review 98(4) (November 2004): 529-545.
  • 4.1.4. David Clingingsmith, Asim Ijaz Khwaja, and Michael Kremer, “Estimating the Impact of the Hajj: Religion and Tolerance in Islam's Global Gathering,” Quarterly Journal of Economics 124(3) (August 2009): 1133-1170.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                    

2:00pm – 3:30pm   Natural Experiments: Quantitative Methods

Thad Dunning, University of California, Berkeley and

Daniel Hidalgo, Massachusetts Institute of Technology

We discuss the role of statistical models in the analysis of natural experiments and provide an overview of quantitative techniques suitable for estimating causal effects. We emphasize the advantages of simplicity and transparency in the quantitative analysis of natural experiments.

  • 4.2.1. F. Daniel Hidalgo, “Digital Democratization: Expanding the Electorate Through Voting Technology.”  Manuscript, Department of Political Science, MIT (2004).
  • 4.2.2. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press, 2012), Chapters 5--6. (Book to purchase)

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Natural Experiments: Qualitative Methods

We highlight the essential role of qualitative methods in the analysis of natural experiments. We present examples that illustrate how qualitative evidence can bolster the credibility of causal assumptions and aid in the interpretation of quantitative results.

  • 4.3.1. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press, 2012), Chapter 7. (Book to purchase)
  • 4.3.2. Jeremy Ferwerda and Nicholas Miller, “Political Devolution and Resistance to Foreign Rule: A Natural Experiment,” American Political Science Review, forthcoming.
     

Thursday, June 19 Module 5: Interpretive Methods for Archival and Historical Research —Thomas Dodman and Daragh Grant

This module introduces students to the challenges of working with materials drawn from different social, cultural, and historical settings, and explores creative interpretive strategies for addressing these challenges. Students will be introduced to the basics of the historical method, and will be encouraged to think about how a careful attention to questions of temporality and periodization can shape and reveal new avenues in their empirical research. All three sessions will be attentive to the problem of analyzing historical materials from the standpoint of the present. Shifting meanings over time, and transformations in the criteria for judgment, present particular problems for historical researchers. Students will be challenged to think about the problems posed by the use of familiar concepts like “culture” and “identity” against the background of processes of historical change that destabilize both the content and the contours of such concepts. Finally, in light of these challenges, students will be invited to think through the strategies available for working in a partial archive, with attention to the virtues and pitfalls of creatively thinking about historical source materials.

All three sessions will aim to situate the readings in relationship to the specific research needs of students taking the module. With that in mind, it would be helpful for students to come to the first session prepared to briefly describe the role that archival or historical research is likely to play in their dissertation.

8:45am - 10:15am  History as social science: The study of structures and events

Thomas Dodman, Boston College and Daragh Grant, University of Chicago

This session introduces students to the historical method, highlighting two key challenges to the study of historical events. Students will begin the session by working in groups to identify their own archival challenges, specifically related to two questions. First, how does the problem of temporality enter their work? And second, how do the events they study refashion the very structures of the societies on which their research is centered?

  • 5.1.1. William H. Sewell Jr., “Three Temporalities: Toward an Eventful Sociology” in Logics of History: Social Theory and Social Transformation (University of Chicago Press, 2005), pp. 81-123. (Book to purchase)
  • 5.1.2. Marshall Sahlins, “Structure and History” in Islands of History (University of Chicago Press, 1985), pp. 136-516.

Recommended:

  • 5.1.3. William H. Sewell Jr., “A Theory of the Event: Marshall Sahlins’s ‘Possible Theory of History’” in Logics of History: Social Theory and Social Transformation (University of Chicago Press, 2005), pp. 197-224. (Book to purchase)
  • 5.1.4. William H. Sewell Jr., “History, Theory, and Social Science” in Logics of History: Social Theory and Social Transformation (University of Chicago Press, 2005), pp. 1-21. (Book to purchase)

2:00pm – 3:30pm  Avoiding anachronism: Events, periodization, and the problem of conceptual change

Thomas Dodman, Boston College and Daragh Grant, University of Chicago

This session address a further challenge posed by the study of historical change: namely, how one avoids the problem of anachronism generated by conceptual transformations that accompany worldly events. The readings address two concepts of central importance to the study of political change in the present: sovereignty and revolution. Although our discussion will be oriented around the readings for the session, students will be invited to discuss the kinds of concepts that are central to their own work, and to think collaboratively about how they might address the challenges posed by shifts in meaning across time.

  • 5.2.1. Constantin Fasolt, “The Text: Bartolus of Sassoferrato” in The Limits of History (University of Chicago Press, 2004), pp. 155-218.
  • 5.2.2. William H. Sewell Jr., “Historical Events as Transformations of Structures: Inventing Revolution at the Bastille” in Logics of History: Social Theory and Social Transformation (University of Chicago Press, 2005), pp. 225-270. (Book to purchase)

Recommended:

  • 5.2.3. Constantin Fasolt, Allan Megill, and Gabrielle M. Spiegel, “The Limits of History: An Exchange,” Historically Speaking 6(5) (May/June 2005): 5-17.
  • 5.2.4. Jan Goldstein, Hysteria Complicated by Ecstasy: The Case of Nanette Leroux (Princeton University Press, 2011).

4:00pm - 5:30pm  Questions of Evidence

Thomas Dodman, Boston College and Daragh Grant, University of Chicago

At the heart of historical research is the question of evidence. Although this may seem a straightforward empirical matter, it is not, for evidence is never neutral, but systematically embroiled in relations of power and knowledge inherent to the constitution of archives. This section explores debates on the nature of historical evidence and on various approaches to historical sources. It aims to foster a critical reflection on the epistemological status of the production of history.

  • 5.3.1. Carlo Ginzburg, “Clues: Roots of an Evidential Paradigm” in Clues, Myths, and the Historical Method, John and Anne C. Tedeschi, trans. (Johns Hopkins University Press, 1989), pp. 96-125.
  • 5.3.2. Randolph Head, “Knowing the State: The Transformation of Political Knowledge in Swiss Archives, 1450-1770,” Journal of Modern History 75(4) (December 2003): 745-782.
  • 5.3.3. Michel-Rolph Trouillot, “An Unthinkable History: The Haitian Revolution as a Non-Event (excerpts).” In Alyssa Goldstein Sepinwall, ed., Haitian History: New Perspectives (Routledge, 2013), pp. 33-54.

Recommended:

  • 5.3.4. Carolyn Steedman, “Something She Called a Fever: Michelet, Derrida, and Dust,” American Historical Review 106(4) (October 2001): 1159-1180.
  • 5.3.5. Joan W. Scott, “Evidence of Experience.” In James Chandler, Harry Harootunian, and Arnold Davidson, eds., Questions of Evidence: Proof, Practice, and Persuasion across the Disciplines (University of Chicago Press, 1994), pp. 363-387.

Further Readings

Students interested in reading further might be interested in the below materials. They are neither required nor suggested for those planning to that this module:

Dipesh Chakrabarty, Provincializing Europe: Postcolonial Thought and Historical Difference (Princeton University Press, 2000), pp. 3-26.

Ranajit Guha, “The Prose of Counter Insurgency.” In Ranajit Guha and Gayatri Spivak, eds., Selected Subaltern Studies (Oxford University Press, 1988), pp. 45-86.

Reinhart Koselleck, The Practice of Conceptual History: Timing History, Spacing Concepts (Stanford University Press, 2002).

Reinhart Koselleck, Futures Past: On the Semantics of Historical Time (Columbia University Press, 2004).

David Landes and Charles Tilly, eds., History as Social Science (Prentice Hall, 1971).

Jacques Revel, “History and the Social Sciences.” In Theodore M. Porter and Dorothy Ross, eds., The Cambridge History of Science, Vol. 7: The Modern Social Sciences (Cambridge University Press, 2003), pp. 391-404.

Charles Tilly, Big Structures, Large Processes, Huge Comparisons (Russell Sage Foundation, 1984).


Thursday, June 19 Module 6 Designing and Conducting Fieldwork: Preparing and Operating in the Field – Diana Kapiszewski and Lauren MacLean

Overall Module Note:  Each session of this module will be conducted with the understanding that participants have carefully read the assigned readings.  Rather than lecturing on the readings, the instructors will facilitate discussions of concepts and ideas in small and large groups, and encourage students to practice using data-collection techniques. These discussions and activities will draw on the readings as well as our collective experiences in managing fieldwork’s diverse challenges.

8:45am - 10:15am  Borders and Varieties of Fieldwork

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

In this session we discuss our conception of fieldwork as an iterative process entailing repeated shifts among research design, data collection, and data analysis. We also consider how fieldwork impinges on the identification of a research question, and on post-fieldwork analysis and theory development.  We discuss various types of fieldwork and the different stages of a project at which it might occur, and address issues of ethics and power in the field.

  • 6.1.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Field Research in Political Science:  Practices and Principles” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).
  • 6.1.2. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “A Historical and Empirical Overview of Field Research in the Discipline” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).

Recommended

  • 6.1.3. David Collier, “Data, Field Work and Extracting New Ideas at Close Range,” APSA-CP Newsletter 10(1) (Winter 1999): 1-2, 4-6. 
  • 6.1.4. David Collier, David A. Freedman, James D. Fearon, David D. Laitin, John Gerring, and Gary Goertz, “Symposium: Case Selection, Case Studies, and Causal Inference,” Qualitative & Multi-Method Research 6(2) (Fall 2008):  2-16.
  • 6.1.5. Soledad Loaeza, Randy Stevenson, and Devra C. Moehler, “Symposium:  Should Everyone Do Fieldwork?,” APSA-CP Newsletter 16(2) (2005):  8-18.
  • 6.1.6. Elisabeth Wood, “The Ethical Challenges of Field Research in Conflict Zones,” Qualitative Sociology 29(3) (September 2006):  373-386.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.

2:00pm - 3:30pm Preparing for Fieldwork

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

    • 6.2.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Preparing for Fieldwork” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming). 
    • 6.2.2. Christopher B. Barrett and Jeffrey W. Cason, “Identifying a Site and Funding Source” in Overseas Research II: A Practical Guide (Johns Hopkins University Press, 2010).

Recommended

  • 6.2.3. Christopher B. Barrett and Jeffrey W. Cason, “Predeparture Preparations” in Overseas Research II: A Practical Guide (Johns Hopkins University Press, 2010). 

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Operating in the Field:  Collecting Data, Managing People

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

This session considers the challenges of managing data and people in the field.  We introduce a range of interactive and non-interactive data-collection techniques, and consider the trade-offs among them and how they can be combined.  We discuss hiring and working with RAs and broader issues of cooperation and managing relationships in the field.  Finally, we consider in greater depth several non-interactive forms of data-collection.

  • 6.3.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Managing in the Field: Logistical, Social, Operational, and Ethical Challenges” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).
  • 6.3.2. Lee Ann Fuji, “Working with Interpreters.”  In Layna Mosley, ed., Interview Research in Political Science (Cornell University Press, 2013).

Recommended

  • 6.3.3. Melani Cammett, “Positionality and Sensitive Topics:  Matched Proxy Interviewing as a Research Strategy.”  In Layna Mosley, ed., Interview Research in Political Science (Cornell University Press, 2013).
  • 6.3.4. Sheila Carapico, Janine A. Clark, Amaney Jamal, David Romano, Jilian Schwedler, and Mark Tessler, “Symposium: The Methodologies of Field Research in the Middle East,”  PS:  Political Science and Politics 39(3) (July 2006).
  • 6.3.5. Elisabeth Wood, “Field Methods.”  In Charles Boix and Susan Stokes, eds., The Oxford Handbook of Comparative Politics (Oxford University Press, 2007).

Friday, June 20 Unified (whole institute) sessions on publishing and funding

10:30am - 11:40am U10 Obtaining Funding (unified session)

Brian Humes,   National Science Foundation

What are the features of successful grant proposals? This session offers guidelines designed to help you not only secure funding for your project but also use the proposal writing process to move forward in the research itself.

  • U.10.1. Adam Przeworski and Frank Salomon, “On the Art of Writing Proposals: Some Candid Suggestions for Applicants to Social Science Research Council Competitions,” Social Science Research Council (1998).
  • U.10.2.Barry Weingast, “Structuring Your Papers (Caltech Rules),” Stanford University (April 1995, revised 2010).

11:45am-12.50pm U11 Getting Publishing (unified session)

John Ishiyama, Editor, American Political Science Review

On the writing and preparing:

  • U.11.1. Stephen K. Donovan, “How to Alienate Your Editor: A Practical Guide for Established Authors,” Journal of Scholarly Publishing 36(4) (July 2005): 238-242. Read pp. 240-242.
  • U.11.2. Stephen K. Donovan, “Putting Editors to Trouble (or People of That Sort).” Journal of Scholarly Publishing 41(1) (October 2009): 103-109.
  • U.11.3. James A. Stimson, “Professional Writing in Political Science: A Highly Opinionated Essay.” Paper, University of North Carolina. Available at www.unc.edu/_jstimson/Writing.pdf. [Skip portions that are not relevant for you.]

On rejection:

  • U.11.4. Stephen K. Donovan, “The Importance of Resubmitting Rejected Papers,” Journal of Scholarly Publishing 38(3) (April 2007): 151-155.
  • U.11.5. Gregory Weeks, “Facing Failure: The Use (and Abuse) of Rejection in Political Science,” PS: Political Science and Politics 39(4) (October 2006): 876-882.

Also Recommended

  • U.11.6. For reflections of a previous editor: Dina A. Zinnes, “Reflections of a Past Editor,” PS: Political Science and Politics 18(3) (July 1985): 607-612.
  • U.11.7. Data (the quantitative kind): Michael S. Lewis-Beck and Dena Levy, “Correlates of Publication Success: Some AJPS Results,” PS: Political Science and Politics 26(3) (September 1993): 558-561.

12: 50pm - 2:15pm  Lunch.        

            

Friday, June 20 Module 7: Natural Experiments — Thad Dunning and Daniel Hidalgo

8:30am - 10:00am Evaluating Natural Experiments

Thad Dunning, University of California, Berkeley and

Daniel Hidalgo, Massachusetts Institute of Technology

We critically assess natural-experimental research using an evaluative framework based on (1) the plausibility of as-if random assignment; (2) the credibility of causal and statistical assumptions; and (3) the substantive and theoretical relevance of the intervention.  We emphasize the importance of quantitative and qualitative diagnostics and substantive knowledge for building successful natural-experimental designs.

  • 7.1.1. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press 2012), Chapters 8-10. (Book to purchase)
  • 7.1.2. Devin Caughey and Jasjeet Sekhon, “Elections and the Regression Discontinuity Design: Lessons from Close U.S. House Races, 1942–2008,” Political Analysis 19(4) (October 2011): 385-408.

10:00am - 10:30am Coffee Break.

10:30am - 11:30am U10 Obtaining Funding (unified session)

Brian Humes, National Science Foundation

See reading list on preceding pages

11:30am-12.30pm U11 Getting Publishing (unified session)

John Ishiyama, Editor, American Political Science Review

See reading list on preceding pages

12:30pm-2:00pm Lunch

2:00pm - 3:30pm Design Your Own Natural Experiment

Thad Dunning, University of California, Berkeley and

Daniel Hidalgo, Massachusetts Institute of Technology

In this session, we give participants the opportunity to design a natural experiment related to their own work and receive feedback from course participants.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Multi-Method Research and Natural Experiments

Thad Dunning, University of California, Berkeley and

Daniel Hidalgo, Massachusetts Institute of Technology

We end the course by evaluating the promise and obstacles to the use of multi-method research in the analysis of natural experiments. Drawing upon the previous sessions and readings, we discuss how qualitative methods can help address some of the criticisms of natural experiments, as well as how natural experiments can bolster the inferences drawn from qualitative evidence.

  • 7.3.1. Thad Dunning, Natural Experiments in the Social Sciences: A Design-Based Approach (Cambridge University Press 2012), Chapter 11.  (Book to purchase)

Further Readings by Topic (for both Modules 4 and 7):

Standard Natural Experiments:

Christopher Blattman, “From Violence to Voting: War and Political Participation in Uganda” American Political Science Review 103(2) (May 2009): 231-247.

Raghabendra Chattopadhyay and Esther Duflo, “Women as Policy Makers: Evidence from a Randomized Experiment in India,” Econometrica 72(5) (September 2004): 1409-1443.

Daniel Doherty, Donald Green, and Alan Gerber, “Personal Income and Attitudes toward Redistribution: A Study of Lottery Winners,” Political Psychology 27(3) (June 2006): 441-458.

Claudio Ferraz and Frederico Finan, “Exposing Corrupt Politicians: The Effect of Brazil’s Publicly Released Audits on Electoral Outcomes,” Quarterly Journal of Economics 123(2) (May 2008): 703-745.

Susan Hyde, “The Observer Effect in International Politics: Evidence from a Natural Experiment,” World Politics 60(1) (October 2007): 37–63.

Jason Lyall, “Does Indiscriminate Violence Incite Insurgent Attacks? Evidence from Chechnya,” Journal of Conflict Resolution 53(3) (June 2009): 331-362.

Daniel N. Posner, “The Political Salience of Cultural Difference: Why Chewas and Tumbukas Are Allies in Zambia and Adversaries in Malawi,” American Political Science Review 98(4) (November 2004): 529-545.

Regression-Discontinuity Designs:

Thad Dunning and Janhavi Nilekani, “Ethnic Quotas and Political Mobilization: Caste, Parties, and Distribution in Indian Village Councils.” Working paper, Department of Political Science, Yale University (2010).  Available at http://www.thaddunning.com/research/all-research.

David S. Lee, “Randomized Experiments from Non-random Selection in U.S. House Elections,” Journal of Econometrics 142(2) (February 2008): 675-697.

Amy Lerman, “Bowling Alone (With my Own Ball and Chain): The Effects of Incarceration and the Dark Side of Social Capital.”  Manuscript, Department of Politics, Princeton University (2008).

Donald L. Thistlewaite and Donald T. Campbell, “Regression-discontinuity Analysis: An Alternative to the Ex-post Facto Experiment,” Journal of Educational Psychology 51(6) (December 1960): 309-317.

Instrumental-Variables Designs:

Edward Miguel, Shanker Satyanath, and Ernest Sergenti, “Economic Shocks and Civil Conflict: An Instrumental Variables Approach,” Journal of Political Economy 112(4) (August 2004): 725-753.

Analysis and Design:

Joshua D. Angrist and Alan B. Krueger, “Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments,” Journal of Economic Perspectives 15(4) (Fall 2001): 69-85.

Henry Brady and David Collier, eds., Rethinking Social Inquiry: Diverse Tools, Shared Standards, 2nd ed. (Rowman & Littlefield, 2010).

Donald T. Campbell and Julian C. Stanley, Experimental and Quasi-Experimental Designs for Research (Houghton Mifflin Co., 1963).

Thad Dunning, “Improving Causal Inference: Strengths and Limitations of Natural Experiments,” Political Research Quarterly 61(2) (June 2008): 282-293.

Thad Dunning, “Model Specification in Instrumental-Variables Regression,” Political Analysis 16(3) (July 2008): 290-302.

Thad Dunning, “Natural and Field Experiments: The Role of Qualitative Methods,” Qualitative Methods Newsletter 6(2) (2008).

David Freedman, Statistical Models: Theory and Practice (Cambridge University Press, 2005).

David Freedman, Robert Pisani, and Roger Purves, Statistics, 4th ed. (W.W. Norton & Co., 2007), Chapter 1 (“Controlled Experiments”) and Chapter 2 (“Observational Studies”).

Donald P. Green, Terence Y. Leong, Holger L. Kern, Alan S. Gerber, and Christopher W. Larimer,   “Testing the Accuracy of Regression Discontinuity Analysis Using Experimental Benchmarks,”  Political Analysis 17(4) (October 2009): 400-417.

Allison J. Sovey and Donald P. Green, “Instrumental Variables Estimation in Political Science: A Readers’ Guide,” American Journal of Political Science 55(1) (January 2011): 188-200.


Friday, June 20 Module 8: Interpretivism, Discourse Analysis, National Identity, and IR Theory— Ted Hopf

8:30am - 10:00am Interpretivism and Neo-Positivism

Ted Hopf, National University of Singapore

Many consider it impossible to combine interpretivist commitments to local intersubjectivities and mainstream social science commitments to causal explanations and generalizations. This session suggests that there is substantial potential compatibility between an interpretivist ontological approach and many mainstream social science methodological techniques. Indeed, scholars not known for being in the mainstream of social science, such as Geertz, Bourdieu, and Foucault, have adopted numerous neo-positivist, dare I say Popperian, techniques in their work, without, of course, explicitly acknowledging doing so.                                                                                                  

  • 8.1.1. Ted Hopf, “The Limits of Interpreting Evidence.” In Richard Ned Lebow and Mark Irving Lichbach, eds., Theory and Evidence in Comparative Politics and International Relations (Palgrave, 2007), pp. 55-84.

10:00am - 10:30am Coffee Break.

10:30am - 11:30am U10 Obtaining Funding (unified session)

Brian Humes, National Science Foundation

See reading list on preceding pages

11:30am-12.30pm U11 Getting Publishing (unified session)

John Ishiyama, Editor, American Political Science Review

See reading list on preceding pages

12:30pm-2:00pm Lunch

2:00pm - 3:30pm Discourse Analysis, Identity and International Politics

Ted Hopf, National University of Singapore

Identity is one of the master variables of social constructivism. How one understands oneself in relationship to others implies what interests you may have in that other, or others like her. But how does one relate this insight to international politics? One way is to hypothesize that the national identities of states imply interests in other states. How we might go about finding those national identities and relating them to foreign policy outcomes are the central questions of this session. We will learn how to execute a discourse analysis of national identity, derive testable hypotheses from it, and gather evidence that can assess those claims.

  • 8.2.1. Ted Hopf, Social Construction of International Politics. Identities and Foreign Policies, Moscow 1955 and 1999 (Cornell University Press, 2002), pp. 1-38, 42, 155, 157-58,
  • 8.2.2. Ted Hopf, “Identity Relations and the Sino-Soviet Split.” In Rawi Abelal, Yoshiko Herrera, Iain Johnston, and Rose McDermott, eds., Measuring Identity (Cambridge University Press, 2009), pp. 279-315
  • 8.2.3. Ted Hopf, “Common Sense Constructivism and Hegemony in World Politics,” International Organization 57(2) (April 2013): 317-354.

4:00pm - 5:30pm Constructing a Large-N National Identity Data Base

Ted Hopf, National University of Singapore

The relationship between interpretivists and mainstream social scientists is often seen as being most starkly different in the juxtaposition of interpretivists and large-n quantitative IR scholars. Despite the fact that these two schools rarely meet, this module explores the possibility of constructing a large-n intersubjective national identity data base that can be usefully employed by large-n IR scholars. Is it possible to remain true to interpretivist principles while, for example, providing replicable data on all great power national identities since 1810?

  • 8.3.1. “Making Identity Count” chapters: Hopf, Allan, and Vucetic

Friday, June 20 Module 9 Designing and Conducting Fieldwork: Collecting and Analyzing Data – Diana Kapiszewski and Lauren MacLean

Overall Module Note:  Each session of this module will be conducted with the understanding that participants have carefully read the assigned readings.  Rather than lecturing on the readings, the instructors will facilitate discussions of concepts and ideas in small and large groups, and encourage students to practice using data-collection techniques. These discussions and activities will draw on the readings as well as our collective experiences in managing fieldwork’s diverse challenges.

8:30am - 10:00am Interactive Forms of Data Collection

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

This session considers the challenges inherent in carrying out, and analytic upsides and downsides of, a series of interactive forms of data collection:  participant observation, ethnography, surveys, and experiments.

  • 9.1.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Site-Intensive Methods: Ethnography and Participant Observation” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).
  • 9.1.2. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Surveys in the Context of Field Research” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).                                                 
  • 9.1.3. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Experiments in the Field” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming).  

Recommended

  • 9.1.4 Brackette F.  Williams, “Skinfolk, Not Kinfolk: Comparative Reflections on the Identity of Participant Observation in Two Field Situations.”  In Diane Wolf, ed., Feminist Dilemmas in Fieldwork (Westview Press, 1996).
  • 9.1.5. Ellen Pader, “Seeing with an Ethnographic Sensibility: Explorations Beneath the Surface of Public Policies.” In Dvora Yanow and Peregrine Schwartz-Shea, eds., Interpretation and Method:  Empirical Research Methods and the Interpretive Turn (M.E. Sharpe, 2006).
  • 9.1.6. Nora Cate Schaeffer and Stanley Presser, “The Science of Asking Questions,” Annual Review of Sociology 29(1) (December 2003): 65-88.
  • 9.1.7. Paluck, Elizabeth Levy, “The Promising Integration of Qualitative Methods and Field Experiments,” The ANNALS of the American Academy of Political and Social Science 628(1) (March 2010):  59-71.
  • 9.1.8. Jan Kubik, “Ethnography of Politics: Foundations, Applications, Prospects.” In Edward Schatz, ed., Political Ethnography (University of Chicago Press, 2009).
  • 9.1.9. Henry E. Brady, “Contributions of Survey Research to Political Science,” PS: Political Science and Politics 33(1) (March 2000): 47-57.
  • 9.1.10. Seymour Sudman and Norman M. Bradburn, Asking Questions: A Practical Guide to Questionnaire Design (Jossey-Bass, 1982). 

10:00am - 10:30am Coffee Break.

10:30am - 11:30am U10 Obtaining Funding (unified session)

Brian Humes, National Science Foundation

See reading list on preceding pages

11:30am-12.30pm U11 Getting Publishing (unified session)

John Ishiyama, Editor, American Political Science Review

See reading list on preceding pages

12:30pm-2:00pm Lunch

2:00pm - 3:30pm Interviewing

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

This session explores various types of interviewing:   one-on-one in-depth interviews, oral histories, and focus groups.  We seek to “contextualize” these interactive forms of data collection, offering strategies for conducting interviews in the field.

  • 9.2.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Interviews, Oral Histories, and Focus Groups” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming). 
  • 9.2.2. Erik Bleich and Robert Pekkanen, “How to Report Interview Data.”  In Layna Mosley, ed., Interview Research in Political Science (Cornell University Press, 2013).
  • 9.2.3. Joe Soss, “Talking Our Way to Meaningful Explanations: A Practice-Centered View of Interviewing for Interpretive Research.” In Dvora Yanow and Peregrine Schwartz-Shea, eds., Interpretation and Method:  Empirical Research Methods and the Interpretive Turn (M.E. Sharpe, 2006).

Recommended

  • 9.2.4. Beth Leech and Kenneth Goldstein, “Symposium: Interview Methods in Political Science,” PS: Political Science and Politics 35(4) (December 2002): 663-672.
  • 9.2.5. Susan E. Short, Ellen Perecman, and Sara R. Curran, “Focus Groups.”  In Ellen Perecman and Sara Curran, eds., A Handbook for Social Science Field Research: Essays & Bibliographic Sources on Research Design and Methods (Sage, 2006).
  • 9.2.6. Herbert Rubin and Irene Rubin, Qualitative Interviewing. The Art of Hearing Data, 2nd ed. (Sage, 2005), Chapters 6-9.
  • 9.2.7. Oisin Tansey, “Process Tracing and Elite Interviewing: A Case for Non-Probability Sampling,” PS: Political Science and Politics 40(4) (October 2007):  765-772.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Analyzing Data, Assessing Progress, Thinking Theoretically

Diana Kapiszewski, Georgetown University

Lauren M. MacLean, Indiana University

This session considers data organization, storing, and sharing; thinking analytically and beginning to analyze data while in the field, and different data analysis options; assessing progress; and beginning to write in the field and presenting initial findings to different audiences.

  • 9.3.1. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “Analyzing, Writing, and Retooling in the Field” in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming). 
  • 9.3.2. Diana Kapiszewski, Lauren M. MacLean, and Benjamin L. Read, “The Future of Field Research in Political Science”  in Field Research in Political Science: Practices and Principles (Cambridge University Press, forthcoming). 
  • 9.3.3. Robert Emerson, Rachel Fretz, and Linda Shaw, “Processing Fieldnotes: Coding and Memoing” in Writing Ethnographic Fieldnotes (University of Chicago Press, 1995).

Recommended

  • 9.3.4. Gilbert Shapiro and John Markoff, “A Matter of Definition.”  In Carl Roberts, ed., Text Analysis for the Social Sciences: Methods for Drawing Statistical Inferences from Texts and Transcripts (Lawrence Erlbaum, 1997).
  • 9.3.5. Rose McDermott et al., “Symposium: Data Collection and Collaboration,”  PS: Political Science and Politics 43(1) (January 2010):  15-58.
     

Monday, June 23 Module 10 – Set Theory I - Gary Goertz and Claudius Wagemann

8:45am - 10:15am  Introduction to logic and set theory

Gary Goertz, University of Notre Dame

This session discusses key ideas from logic and set theory that underpin many qualitative methods. Central attention is focused on ideas of necessary and sufficient conditions (and their derivatives). The session also contrasts logic/set theory with statistics/probability theory.

  • 10.1.1. Gary Goertz and James Mahoney, “Mathematical Prelude: A Selective Introduction to Logic and Set Theory for Social Scientists” in A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences (Princeton University Press, 2012), pp. 16‐38. (Book to purchase)
  • 10.1.2. Carsten Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis (Cambridge University Press, 2012), Chapter 2, pp. 42-55. (Book to purchase)

Recommended:

  • 10.1.3. Gary Goertz, “The Substantive Importance of Necessary Condition Hypotheses.” In Gary Goertz and Harvey Starr, eds., Necessary Conditions: Theory, Methodology, and Applications (Rowman & Littlefield, 2003).

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                     

2:00pm - 3:30pm Two Cultures: Contrasting Qualitative and Quantitative Research

Gary Goertz, University of Notre Dame

This session contrasts an approach to qualitative and multimethod research based on the statistical paradigm with one based on within‐case causal analysis and logic.

  • 10.2.1. Gary Goertz and James Mahoney, A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences (Princeton University Press, 2012), Chapters 4‐6, 8‐9, and 15. (Book to purchase)

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm   Social Science Concepts

Gary Goertz, University of Notre Dame

This session provides basic guidelines for the construction and evaluation of concepts. It particular it provides a framework dealing for dealing complex concepts which are typical in much social science research.

  • 10.3.1. Gary Goertz, Social Science Concepts: A User’s Guide (Princeton University Press, 2006), Chapters 1‐2.
  • 10.3.2. Gary Goertz and James Mahoney, A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences (Princeton University Press, 2012), Chapters 11 and 13. (Book to purchase)

Monday, June 23  Module 11 Ethnography I, Fred Schaffer and Timothy Pachirat

Overall Description

How does sustained attention to meaning making in the research world contribute to the study of politics? What are the promises, and perils, of social research that invites the unruly minutiae of lived experience and conceptual lifeworlds to converse with, and contest, abstract disciplinary theories and categories? In this practice-intensive short course, we explore two ethnographic methods - participant observation and ordinary language interviewing - with specific attention to their potential to subvert, generate, and extend understandings of politics and power.

8:45am - 10:15am

Introductions [Pachirat and Schaffer]

Part A: Introduction to Ethnography  [Pachirat]

This part of the session explores the promises and pitfalls of ethnographic approaches to the political.

  • 11.1.1. Clifford Geertz, "Thick Description: Toward an Interpretive Theory of Culture" in The Interpretation of Cultures (Basic Books, 1973) (assigned as U.5.1)
  • 11.1.2. Bent Flyvbjerg, “The Power of Example,” in Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again, Steven Sampson, trans. (Cambridge University Press, 2001).
  • 11.1.3. Edward Schatz, “Ethnographic Immersion and the Study of Politics” and “What Kind(s) of Ethnography does Political Science Need?” In Edward Schatz, ed., Political Ethnography: What Immersion Contributes to the Study of Power (University of Chicago Press, 2009).
  • 11.1.4. Timothy Pachirat, "The Political in Political Ethnography: Dispatches from the Kill Floor." In Edward Schatz, ed., Political Ethnography: What Immersion Contributes to the Study of Power (University of Chicago Press, 2009).

Part B: Introduction to Ordinary Language Interviewing  [Schaffer]

Ordinary language interviewing is a tool for uncovering the meaning of words in everyday talk. By studying the meaning of words (in English or other languages), the promise is to gain insight into the various social realities these words name, evoke, or realize. This part of the session covers some basic questions about ordinary language interviewing: what it is, what can be discovered through it, and how it is similar to and different from other types of ethnographic interviewing.

  • 11.1.5. Barbara Sherman Heyl, “Ethnographic Interviewing.” In Paul Atkinson, Amanda Coffey, Sara Delamont, John Lofland and Lyn Lofland, eds., Handbook of Ethnography (Sage, 2001), pp. 369-383.
  • 11.1.6. Ludwig Wittgenstein, The Blue and Brown Books (Harper, 1965), pp. 17-20.
  • 11.1.7. Hanna Fenichel Pitkin, Wittgenstein and Justice: On the Significance of Ludwig Wittgenstein for Social and Political Thought (University of California Press, 1972), pp. 274-279.
  • 11.1.8. Frederic Charles Schaffer, Democracy in Translation: Understanding Politics in an Unfamiliar Culture (Cornell University Press, 1998), pp. ix-xii, 54-85.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

Session 2 (1:40-3:30) How to Do an Ordinary Language Interview [Schaffer]

In this session participants will learn how to conduct an ordinary language interview, and practice doing one focusing on words of their own choosing. Participants will also learn and practice different strategies for approaching people to interview.

At the beginning of this session, participants will sign up for their fieldsite locales and be organized into groups. Participants will work with their groups during this session’s exercises and in the short course’s subsequent exercises.

Session 3 (4:00-6:00) Ordinary Language Interviewing Field Exercise and Write-Up [Schaffer]

Participants will go to fieldsites (around campus or at the Carousel Center Mall) to conduct ordinary language interviews. They will then write-up their main findings.


Monday, June 23 Module 12 Managing and Sharing Qualitative Data – Diana Kapiszewski and Dessislava Kirilova

Increasingly, journals, funding organizations, and professional associations are requiring that research data be shared.  In this module, we discuss the processes of managing and sharing qualitative research data and the benefits of doing so, including enhanced citation and more transparent research, more secondary data analysis, improved teaching, and more and deeper collaboration. While sharing data occurs at a particular moment in the “research lifecycle,” it requires careful planning and systematic data management. We highlight techniques for both, and illustrate their importance by demonstrating the process of sharing qualitative data via the new Qualitative Data Repository.  Finally, we consider factors that make sharing data more challenging, and offer strategies and techniques for overcoming them.   

Notes and caveats: First, for this module, it is useful to have an actual research project in mind, including a good idea of what sorts of data you will collect.

Second, as you will quickly see when you begin to do the readings, we are drawing a great deal on materials from the United Kingdom (although many of the referenced practices are European-wide).  This is the case because the normative infrastructure for managing and sharing qualitative data is much more developed in Europe than it is in the United States, and there is thus a great deal we can learn from understanding how they operate.  In fact, a challenge we will take on in the module is considering how we can adopt and adapt their norms and standards in the context of U.S. social science.

8:45am ‐ 10:15am Preparing for Sharing: Data and the Research Lifecycle

Diana Kapiszewski, Georgetown University

Dessislava Kirilova, Syracuse University/Yale University

Research data can have a longer lifespan than the research project from which they arose, and that likelihood is greatly increased when they are shared. We begin the session by discussing the benefits – to the researcher who collected them, and to other scholars – of sharing qualitative data. We then examine how to design research projects in anticipation of sharing the data they generate. We introduce the notion of the “research lifecycle,” and use examples of real research projects to identify points in that lifecycle at which data sharing considerations come into play; we also establish which protocols might be needed at key stages.

10:15am ‐ 10:45am Coffee Break.

10:45am ‐ 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm ‐ 2:00pm Lunch.

2:00pm ‐ 3:30pm Managing, Documenting, and Sharing Data  

Diana Kapiszewski, Georgetown University

Dessislava Kirilova, Syracuse University/Yale University

We discuss a range of strategies and techniques that can be used to effectively manage data; consider a few sample data management plans; and engage in some exercises to help students learn to create their own data management plan. We also consider the role of ‘documenting’ data, thinking creatively and critically about them and providing background and contextual information to help others scholars make sense of shared data. Finally, we illustrate the process of sharing data via the new Qualitative Data Repository, revealing the importance to data sharing of careful organization, and knowledge, of one’s data. 

  • 12.2.1. Louise Corti and Paul Thompson, “Secondary Analysis of Archived Data.” In Clive Seale, Giampietro Gobo, Jaber F. Gubrium, and David Silverman, eds., Qualitative Research Practice (Sage, 2004).
  • 12.2.2. MIT Libraries, “Data Management: Write an NSF Data Management Plan” (2013). Available at https://libraries.mit.edu/data-management/plan/create-plan/nsf-plans/.
  • 12.2.3. Libby Bishop, “A Proposal for Archiving Context for Secondary Analysis,” MethodologicalInnovationsOnline 1(2) (2006): 10‐20. Available at http://dx.doi.org/10.4256/mio.2006.0008

3:30pm ‐ 4:00pm Coffee Break.

4:00pm ‐ 5:30pm Challenges to Sharing Data:  Copyright and Human Subjects

Diana Kapiszewski, Georgetown University

Dessislava Kirilova, Syracuse University/Yale University

In this session, we explore factors that make sharing qualitative data more challenging, focusing in particular on two central concerns: copyright and human subjects.  We discuss the elastic notion of “fair use,” and techniques for discerning the copyright status of one’s data. And we consider how the process of gaining informed consent might need to be adapted to enable data sharing at the end of a project, and examine strategies for anonymizing qualitative data aiming to preserve original content while minimizing disclosure risk.

  • 12.3.1. Cathy Bailey, Josephine Baxter, Maggie Mort, and Ian Convery, “Community Experiences of the 2001 Foot and Mouth Disease Epidemic in North Cumbria: An Archiving Story,” Methodological Innovations Online 1(2) (2006): 83‐94. Available at http://dx.doi.org/10.4256/mio.2011.0014.
  • 12.3.2. Libby Bishop, “Ethical Sharing and Re‐Use of Qualitative Data,” Australian Journal of Social Issues, 44(3) (Spring 2009): 255-272.
  • 12.3.3. Andrew Clark, “Anonymising Research Data.” Working Paper, ESRC National Centre for Research Methods, Working Paper Series 7/06 (December 2006).

Tuesday, June 24 Module 13 – Set Theory II - Gary Goertz and Claudius Wagemann

8:45am - 10:15am   Basics of QCA

Claudius Wagemann,  Goethe University Frankfurt

This session applies the basics of set theory to Qualitative Comparative Analysis (QCA). Starting from set operations it introduces the central concept of calibration and formalizes the notion of sufficiency and necessity. Doing so, it also refers to causal complexity as it is typical for QCA.

  • 13.1.1. Carsten Q. Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences (Cambridge University Press, 2012), Chapters 1-3. (Book to purchase)
  • 13.1.2. Charles C. Ragin, Redesigning Social Inquiry: Fuzzy Sets and Beyond (University of Chicago Press, 2008), Chapters 4-5. (Book to purchase)

Recommended:

  • 13.1.3. Charles C. Ragin, Redesigning Social Inquiry: Fuzzy Sets and Beyond (University of Chicago Press, 2008), Chapters 1-2. (Book to purchase)
  • 13.1.4. Axel Marx, Benoît Rihoux, and Charles C. Ragin, “The Origins, Development, and Application of Qualitative Comparative Analysis: The First 25 Years,” European Political Science Review 6(1) (February 2014): 115-142.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12:30pm - 2:00pm  Lunch.                     

2:00pm - 3:30pm  Truth Table Analysis

Claudius Wagemann,  Goethe University Frankfurt

This session presents truth table analysis as a fundamental tool of QCA. It also introduces the so-called “measures of fit”, such as consistency and coverage. This also draws our attention to the fundamental trade-off in comparative research between explaining well and explaining much.

  • 13.2.1. Carsten Q. Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences (Cambridge University Press, 2012), Chapters 4-5. (Book to purchase)
  • 13.2.2. Charles C. Ragin, Redesigning Social Inquiry: Fuzzy Sets and Beyond (University of Chicago Press, 2008), Chapter 3. (Book to purchase)

Recommended:

  • 13.2.3. Charles C. Ragin, Fuzzy-Set Social Science (University of Chicago Press, 2000), Chapters 2-3.

3:30pm - 4:00pm Coffee Break.

         

4:00pm - 5:30pm   Limited Diversity and Counterfactual Analysis

Claudius Wagemann,  Goethe University Frankfurt

Limited diversity is a fundamental problem of comparative research in general and QCA in particular. This session introduces the phenomenon and shows ways how to deal with it. This also includes a discussion on counterfactual analysis.

  • 13.3.1. Carsten Q. Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences (Cambridge University Press, 2012), Chapters 6-7. (Book to purchase)
  • 13.3.2. Charles C. Ragin, Redesigning Social Inquiry: Fuzzy Sets and Beyond (University of Chicago Press, 2008), Chapters 8-9. (Book to purchase)

Recommended:

  • 13.3.3. Charles C. Ragin, The Comparative Method (University of California Press, 1987), Chapter 7.
  • 13.3.4. Patrick Emmenegger, “How Good Are Your Counterfactuals? Assessing Quantitative Macro-Comparative Welfare State Research with Qualitative Criteria,” Journal of European Social Policy 21(4) (October 2011): 365-380.

Tuesday, June 24 Module 14 Ethnographic Methods II, Frederic Schaffer and Timothy Pachirat

8:45 - 10:15 Ordinary Language Interview Debriefing [Schaffer]

First we will discuss the challenges participants encountered in approaching people to interview, conducting ordinary language interviews, and writing up results. Next we will catalogue the different word uses/meanings that participants discovered in doing their fieldsite interviews.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.

2:00 - 3:30 Ethics and Praxis in Participant Observation [Pachirat]

An exploration of the practice of participant observation, with special emphasis on jottings, fieldnote writing, and the ethics of fieldwork.

  • 14.2.1. Robert M. Emerson, Rachel I. Fretz, and Linda L. Shaw, Writing Ethnographic Fieldnotes (University of Chicago Press, 1995), Chapters 1-5. (Book to purchase)

Session 6 (4:00 - 5:30) Participant Observation Fieldwork Exercise [Pachirat]

In small groups, participants will conduct participant-observation exercises in pre-selected fieldsites.

Session 7 (5:30 - 7:30) Fieldnote Writing

Participants will use this time to write up a set of fieldnotes based on jottings taken in their fieldsites.


Tuesday, June 24 Module 15 Computer-Assisted Content Analysis I, Will Lowe and Sven Oliver Proksch

This module presumes basic mathematical and statistical concepts and will involve work with computers. The computational elements will include using point-and-click software and dedicated R packages.  Prior expertise with these is not required, just a willingness to explore new tools.  The instructors will provide as much support as necessary to ensure that students can effectively participate in the course and apply these tools in their own projects. 

Participants choosing this module should bring a laptop and be prepared to install software beforehand (A handout with the software prerequisites will be provided before the course).

Participants with problems following the installation instructions will be able to meet with the instructors on Monday, June 23, at 8pm in the lobby of the Sheraton hotel.

8:45am - 10:15am Computer-assisted coding of document topics

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

In the first session we investigate dictionary-based content analysis in old and new style.  We will focus on identifying the assumptions of these widely used measurement models, learning how to use their results effectively in subsequent analyses, validating them, and maybe even correcting them. Finally, we’ll consider the mechanics of getting other people to do the content analysis for us.

  • 15.1.1. Michael Laver and John Garry, “Estimating Policy Positions from Political Texts,” American Journal of Political Science 44(3) (July 2000): 619-634.
  • 15.1.2. Kennth Benoit, Drew Conway, Michael Laver, and Slava Mikhaylov, “Crowd-Sourced Data Coding for the Social Sciences: Massive Non-Expert Coding of Political Texts.” Working Paper prepared for the New Directions in Analyzing Text as Data conference at Harvard University (October 2012).

Optional

  • 15.1.3. Judith Bara, Albert Weale, and Aude Biquelet, “Analysing Parliamentary Debate with Computer Assistance,” Swiss Political Science Review, 13(4) (Winter 2007): 577-605.
  • 15.1.4. Klaus Krippendorf, Content Analysis: An Introduction to Its Methodology (Sage, 1980).

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12:30pm - 2:00pm  Lunch.                     

2:00pm - 3:30pm Practical I

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

We present open source text analysis tools for dictionary-based content analysis and replicate several studies using various text sources e.g. parliamentary speeches, media reports. Participants should bring a laptop on which they have pre-installed software (see instruction sheet).

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Computer-assisted assignment of categories to documents

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

Classification methods automate the assignment of texts to categories in a content typology without the need to construct a dictionary.  This session considers applications of both approaches and considers their advantages and limitations for social scientific research.

  • 15.3.1. Justin Grimmer and Brandon Stewart, “Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts,” Political Analysis 21(3) (July 2013): 267-297.

Optional

  • 15.3.2. Michael Evans, Wayne McIntosh, Jimmy Lin, and Cynthia Cates, “Recounting the Courts? Applying Automated Content Analysis to Enhance Empirical Legal Research,” Journal of Empirical Legal Studies 4(4) (December 2007): 1007-1039.
  • 15.3.3. Dustin Hillard, Stephen Purpura, and John Wilkerson, “Computer-Assisted Topic Classification for Mixed-Methods Social Science Research,” Journal of Information Technology and Politics 4(4) (2008): 31-46.

Tuesday, June 24 Module 16 Mixed-method research and causal mechanisms, part 1 – Nicholas Weller

This module provides students with an introduction to the use of mixed-methods in the study of causal mechanisms.  We will address a variety of topics including: the role of quantitative and qualitative studies of causal mechanisms, how to use quantitative data to select promising cases, and how to use mixed-methods to improve measurement.

8:45am - 10:15am Mixed-method research and causal mechanisms: challenges and opportunities for pathway analysis

Nicholas Weller, University of Southern California

This session introduces participants to the appropriate role of mixed-method research in the context of studying causal mechanisms. We will focus on identifying the value-added of each component of mixed-methods research.

  • 16.1.1. Nicholas Weller and Jeb Barnes, Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms (Cambridge University Press, 2014), Chapters 1-2.
  • 16.1.2. John Gerring, “Causal Mechanisms: Yes, But...,” Comparative Political Studies 43(11) (November 2010): 1499-1526.

Recommended

  • 16.1.3. John Gerring, “Is There a (Viable) Crucial-Case Method?,” Comparative Political Studies 40(3) (March 2007): 231-253.

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Selecting Cases for Pathway Analysis

Nicholas Weller, University of Southern California

This session will provide present a set of general steps for selecting cases for pathway analysis that guides scholars towards how to read the relevant literature, how to identify relevant research questions, and how to think about the types of cases that are relevant given the extant literature and research questions.

  • 16.2.1. Nicholas Weller and Jeb Barnes, Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms (Cambridge University Press, 2014), Chapter 3-4.
  • 16.2.2. Jason Seawright and John Gerring, “Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options,” Political Research Quarterly 61(2) (June 2008): 294-308.

Recommended

  • 16.2.3. Derek Beach and Rasmus Brun Pedersen, “Case Selection Techniques in Process-Tracing and the Implications of Taking the Study of Causal Mechanisms Seriously.” Working Paper (2012).

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Case selection using regression

Nicholas Weller, University of Southern California

This session introduces participants to the use of regression to select cases. We discuss both the benefits and the pitfalls of this approach, and we will walk through multiple examples. The examples include both cross-sectional data and panel data so that we can explore case selection in both instances.

  • 16.3.1. Nicholas Weller and Jeb Barnes, Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms (Cambridge University Press, 2014), Chapter 4-5. Chapter 4 is germane to both sessions 16.2 and 16.3.
  • 16.3.2. Kenneth F. Schulz and David A Grimes, “Case-control studies: research in reverse,” Lancet 359(9304) (February 2002): 431-434

Wednesday, June 25 Module 17 – Set Theory III - Gary Goertz and Claudius Wagemann

8:45am - 10:15am  Potential Pitfalls in Qualitative Comparative Analysis

Claudius Wagemann,  Goethe University Frankfurt

When executing a QCA, some problems might occur with regard to the simultaneous analysis of necessary and sufficient conditions. Also, a less thoughtful application of the algorithm might lead to untenable results. In this session, various potential problems are discussed; their solution should lead to a more reflective use of QCA.

  • 17.1.1. Carsten Q. Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences (Cambridge University Press, 2012), Chapters 8-9. (Book to purchase)
  • 17.1.2. Gary Goertz, “Assessing the Trivialness, Relevance, and Relative Importance of Necessary and Sufficient Conditions in Social Science,” Studies in Comparative International Development 41(2) (Summer 2006): 88-109.

    Recommended:

  • 17.1.3. Carsten Q. Schneider and Claudius Wagemann, “Doing Justice to Logical Remainders in QCA: Moving Beyond the Standard Analysis” Political Research Quarterly 66(1)(March 2013): 211-220.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                     

2:00pm - 3:30pm Extensions of QCA

Claudius Wagemann,  Goethe University Frankfurt

Over the years, various extensions of QCA have been developed. Also based on the interest of the participants, some of them will be presented in this session. A special emphasis will be devoted to multi-value concepts as well as to integrating the time aspect into QCA.

  • 17.2.1. Carsten Q. Schneider and Claudius Wagemann, Set-Theoretic Methods for the Social Sciences (Cambridge University Press, 2012), Chapters 10-11. (Book to purchase)
  • 17.2.2. Lasse Cronqvist and Dirk Berg-Schlosser, “Multi-Value QCA (mvQCA).” In Benoit Rihoux and Charles C. Ragin, eds., Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques (Sage, 2009).

Recommended:

  • 17.2.3. Carsten Q. Schneider and Claudius Wagemann, “Reducing Complexity in Qualitative Comparative Analysis (QCA): Remote and Proximate Factors and the Consolidation of Democracy,” European Journal of Political Research 45(5) (August 2006): 751-786.
  • 17.2.4. Maarten P. Vink and Olaf Van Vliet, “Not Quite Crisp, Not Yet Fuzzy? Assessing the Potentials and Pitfalls of Multi-Value QCA,” Field Methods 21(3) (August 2009): 265-289.

3:30pm - 4:00pm Coffee Break.

        

4:00pm - 5:30pm   Case Selection and Multimethod Research Design

Gary Goertz, University of Notre Dame

    • 17.3.1. Gary Goertz, “Statistical Multimethod and Case Selection,” “Case Studies, Causal Mechanisms, and Selecting Cases: Part I,” and “Case Studies, Causal Mechanisms, and Selecting Cases, Part II: Necessary Conditions.” Manuscripts (2013).

Recommended:

  • 17.3.2. Gary Goertz and James Mahoney, “Case Selection and Hypothesis Testing” in A Tale of Two Cultures: Contrasting the Qualitative and Quantitative Research Paradigms (Princeton University Press, 2012), pp. 177-191. (Book to purchase)
  • 17.3.4. Evan Lieberman, “Nested Analysis as a Mixed-Method Strategy for Comparative Research,” American Political Science Review 99(3) (August 2005): 435-452.

 

Wednesday, June 25 Module 18 Ethnographic Methods III, Frederic Schaffer and Timothy Pachirat

Session 8 (9:15-10:15) Triad Reviews of Fieldnotes

Participants exchange and comment on each other’s fieldnotes.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

Session 9 (2:00 - 3:30) Paired-triad Discussions and Presentations

Participants combine with other triads to discuss the experience of doing participant observation.

Session 10 (4:00 - 5:30) Overall Debriefing (ordinary language interviewing and participant observation) [Pachirat and Schaffer]

In this session, we will reflect together on the following two clusters of questions: (1) How can ordinary language interviewing and participant observation be fruitfully combined when doing ethnographic fieldwork? What are the potential pitfalls of such a combination? (2) To what extent does the method one adopts shape what one apprehends? Specifically, do we learn something different when we access meaning by means of (relatively unstructured) participant observation as opposed to (relatively structured) ordinary language interviewing?


Wednesday, June 25 Module 19 Computer-Assisted Content Analysis II, Will Lowe and Sven Oliver Proksch

8:45am - 10:15am Practical II

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

In this exercise session, we show how to use open source text analysis tools for supervised automated classification. The lab will take the form of a worked example using R and its various text analysis packages.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                    

2:00pm - 3:30pm Computer-assisted inference about document positions

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

Scaling models try to estimate actors' positions on interesting dimensions using differential word usage. In this session we learn how to fit and interpret such models, how to think about dimensionality of texts, what important discourse features are left out or abstracted away, and what we have to assume about how words are generated in order to be able to apply them. We then consider to what extent those assumptions are reasonable, and also how square them with the idiosyncratic and often strategically structured institutional contexts in which political language actually appears.

  • 19.2.1. Michael Laver, Kenneth Benoit, and John Garry, “Extracting Policy Positions from Political Texts Using Words as Data,” American Political Science Review 97(2) (May 2003): 311-332.
  • 19.2.2. Sven-Oliver Proksch and Jonathan B. Slapin, “Position Taking in European Parliament Speeches,” British Journal of Political Science 40(3) (July 2010): 587-611.

Optional

  • 19.2.3. Will Lowe, Kenneth Benoit, Slava Mikhaylov, and Michael Laver, “Scaling Policy Positions from Coded Political Texts,” Legislative Studies Quarterly 36(1) (February 2011): 123-155.
  • 19.2.4. Jonathan B. Slapin and Sven-Oliver Proksch, “A Scaling Model for Estimating Time-Series Party Positions from Texts,” American Journal of Political Science 52(3) (July 2008): 705-722.
  • 19.2.5 Sven-Oliver Proksch and Jonathan B. Slapin, “Institutional Foundations of Legislative Speech,” American Journal of Political Science 56(3) (July 2012): 520-537.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Practical III

Will Lowe, University of Mannheim;  and Sven-Oliver Proksch, McGill University

During this exercise session, we present open source text analysis tools for extracting policy positions from political texts.

3:30pm - 4:00pm Coffee Break.

4:00pm – 5:30pm Content Analysis and Research Design

Will Lowe, University of Mannheim; and Sven-Oliver Proksch, McGill University

In this session we discuss the strengths and limitations of the methods shown in the previous sessions with an eye to how computer assisted content analyses can be best integrated into your research designs. We also welcome practical questions concerning data acquisition, computing issues, and effective presentation of text analysis results.

This session is organized in a question and answer format: You ask the questions, and we (as a group) will try to provide some useful answers.


Wednesday, June 25 Module 20 Mixed-method research and causal mechanisms, part 2– Nicholas Weller

This module continues with the material from Part 1 (Module 16).

8:45am - 10:15am  Case selection using matching

Nicholas Weller, University of Southern California

This session introduces participants to the use of matching as a way to select cases for mixed-methods research. We will discuss matching at a general level and then turn to how to use matching to select cases.

  • 20.1.1. Nicholas Weller and Jeb Barnes, Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms (Cambridge University Press, 2014), Chapter 6.
  • 20.1.2. Elizabeth Stuart, “Matching Methods for Causal Inference: A Review and a Look Forward,” Statistical Science 25(1) (February 2010): 1-21.

Recommended

  • 20.1.3. Richard Nielsen, “Case Selection via Matching,” Sociological Methods and Research (forthcoming).

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Contextualizing and extending prior research

Nicholas Weller, University of Southern California

This session will discuss how to use large-N methods to conceptualize prior research when scholars are building directly on results from other studies.  In particular, we will consider issues related to knowledge accumulation across multiple case studies.

  • 20.2.1. Nicholas Weller and Jeb Barnes, Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms (Cambridge University Press, 2014), Chapters 7-8.
  • 20.2.2. Michael Ross, “How Do Natural Resources Influence Civil War? Evidence from Thirteen Cases,” International Organization 58(1) (Winter 2004): 35-67.

Recommended

  • 20.2.3. Karen Luftey and Jeremy Freese, “Toward Some Fundamentals of Fundamental Causality. Socioeconomic Status and Health in Routine Clinic Visit for Diabetes,” American Journal of Sociology 110(5) (March 2005):1326-1372

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Pathway analysis/mixed method research and measurement

Nicholas Weller, University of Southern California

This session introduces participants to use of mixed-method research in measurement.  We will review the basic concerns related to measurement and then delve in to a variety of ways that mixed-methods research can improve our ability to measure nettlesome concepts.

  • 20.3.1. Gerardo Munck and Jay Verkulien, “Conceptualizing and Measuring Democracy: Evaluating Alternative Indices,” Comparative Political Studies 35(1) (February 2002): 5-34.
  • 20.3.2. Chapter 4 from Statistics in Political Science, read the entire chapter; focus on pp. 82-90 and 104-107.

Recommended

  • 20.3.3. Jeb Barnes and Thomas F. Burke, “Making Way: Legal Mobilization, Organizational Response, and Wheelchair Access,” Law & Society Review 46(1) (March 2012): 167-198

Thursday, June 26 Module 21  Challenges of Medium-N Analysis I – David Collier, Bear Braumoeller, and Christopher Krogslund

This module explores challenges of medium-N analysis, i.e., of research focused on roughly 10 to 70 cases. Charles Ragin has strongly recommended this scope of inquiry as lending itself to combining close knowledge of cases with systematic comparison.

There is some disagreement as to whether—over the past two decades—medium-N analysis has been neglected or has prospered. Yet clearly, a sharp contrast is evident in the unequal development of relevant analytic tools. We have seen major advances in methods both for the qualitative analysis of small numbers of cases, and for large-N research. By contrast, medium-N methods are in a state of flux and evolution.

This deficit is unfortunate, given that (a) many substantive questions are most produc-tively addressed with medium‐N data sets; (b) focusing on an intermediate number of cases can yield findings of substantial generality (i.e., more cases!), while still being anchored in strong case knowledge; and (c) Developing and refining tools for medium‐N analysis can productively expand opportunities for research that is attentive to case knowledge, context and causal complexity—for example, interactions and asym¬metric causation. The lectures and readings for this module will examine both the debates of the past two decades regarding methods for medium N work, and promising tools that require further develop-ment.

8:45am ‐ 10:15am. Framing Recent and Current Debates on Medium-N Methods. 

David Collier, University of California, Berkeley

Among the methodological debates of the past 20 years, two that are especially relevant to IQMR have focused on (a) the application of a conventional “quantitative template” to qualitative research, as proposed in 1994 with the publication of King, Keohane, and Verba’s Designing Social Inquiry; and (b) research strategies for medium-N analysis that seek to combine case knowledge and systematic comparison – very centrally involving a spectrum of the set-theoretic comparative method (STCM). Charles Ragin’s QCA is the most distinguished and creative contribution to this tradition, and the acronyms QCA and STCM (which includes QCA) are both used below, according to which is most relevant to the immediate context of discussion.   

To provide a framework for IQMR’s two modules on medium-N analysis, this first session focuses on (a) selections from Brady and Collier, Rethinking Social Inquiry, and (b) a brief essay by David Collier.

  • 21.1.1. Henry E. Brady and David Collier, eds. Rethinking Social Inquiry: Diverse Tools, Shared Standards, 2nd edition. Lanham, MD: Rowman and Littlefield, 2010.   Assigned reading in RSI: pp. xiii–xviii, 1–10, 15–31, 161–199.   Read other sections, according to your interest and available time.
  • 21.1.2 David Collier, “Problematic Tools: Introduction to Symposium on Set Theory in Social Science.” Qualitative and Multi-Method Research  (2014) 12(1): 2–9.

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Simulation Tests and Evaluation of Medium-N Methods

David Collier, University of California, Berkeley

Christopher Krogslund, University of California, Berkeley

A substantial literature has emerged that employs simulation tests to evaluate methods for medium-N analysis in terms of the stability and validity of findings. This literature has focused centrally on the set-theoretic comparative method (to reiterate, STCM), and it includes contributions from both inside and outside of that tradition. The readings cover key issues regarding simulations.

1. Garson offers a brief introduction to simulations.

2. Krogslund, Choi, and Poertner provide a full example of a simulation test.

3. A valuable idea in contemporary reasoning about causal inference is the concept of the underlying “data generating process” (DGP), together with the insight that causal assess¬ment consists of using real-world data to make inferences about the DGP. The idea of the DGP can be introduced into simulation tests, and it represents a particularly illuminating form of assessment.

4. In using simulations to evaluate any method, finding tests that are well-matched to the method is a challenge. A debate has emerged over how to assess the match to the procedures of STCM, but unfortunately some of the key statements this debate are not yet available from the authors. This reading illustrates the major challenges of such assessment.

  • 21.2.1. David G. Garson, “Computerized Simulation in the Social Sciences: A Survey and Evaluation,” Simulation & Gaming 40(2) (April 2009): 267-279.
  • 21.2.2. Christopher Krogslund, Donghyun Danny Choi, and Matthias Poertner, “Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA.” Revised version of paper presented at the 2013 Annual Meeting of the American Political Science Association, Chicago (2013).
  • 21.2.3. Christopher Krogslund and Katherine Michel, “Recovering the Data-Generating Process: How Well Do Set-Theoretic Methods Perform?” Revised version of paper presented at the 2014 annual meeting of the Midwest Political Science Association April 3–6, 2014, Chicago IL.
  • 21.2.4. Christopher Krogslund and Katherine Michel. “A Larger-N, Fewer Variables Problem? The Counterintuitive Sensitivity of QCA.” Qualitative & Multi-Method Research (2014) 12(1).

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Models for Analyzing Causal Interactions.

Bear Braumoeller, Ohio State University

David Collier, University of California, Berkeley

In medium-N analysis, and indeed in all forms of research, a fundamental priority is to gain insight into causal interactions – i.e., the effect of one explanatory factor is contingent on other factors. This unit explores the analysis of interactions.

1. Braumoeller provides an overview of this topic, focusing on four models of interactions and the trade-offs among them.

2. Ragin discusses the treatment of interactions in STCM, involving combinations of conditions—a perspective that is one of the four modes discussed in (1).

3. Franzese offers a concise summary (as of several years ago) of interaction terms in quantitative studies that have appeared in leading political science journals.

4. Tanner provides excellent examples of interactions in policy evaluation research.

5. Kam and Franzese is a standard text on interactions in quantitative research.

6. Note that the STCM perspective on interactions—specifically combinations of conditions—is also treated in reading for other IQMR modules 

  • 21.3.1. Bear Braumoeller, “Analyzing Interactions: Four Alternative Models,” Qualitative & Multi-Method Research 12(1) (2014).
  • 21.3.3. Charles C. Ragin, “QCA Versus Statistical Interaction.” Department of Sociology, University of California, Irvine (2013).
  • 21.3.3. Robert J. Franzese, “Quantitative Empirical Methods and the Context-Conditionality of Classic and Modern Comparative Politics,” APSA-CP Newsletter 14(1) (2003): 20-24.

Recommended

  • 21.3.4. Cindy D. Kam and Robert J. Franzese, Modeling and Interpreting Interactive Hypotheses in Regression Analysis (University of Michigan Press, 2007).
  • 21.3.5. Sean Tanner, “Evaluating QCA: A Poor Match for Public Policy Research.” Qualitative & Multi-Method Research (2014) 12(1).


Thursday, June 26 Module 22  Geographic Information Systems I: Introduction to GIS as a Qualitative Research Method – Jonnell Robinson

8:45am - 10:15am: Introduction to GIS: GIS, not as quantitative as you might assume

This module will introduce participants to GIS as a tool for qualitative research, present basic GIS terminology and concepts and the basic functions of ESRI’s ArcGIS software suite, particularly those functions that are most commonly used by researchers who use GIS in their post-positivist/feminist research.

  • 22.1.1. Mei-Po Kwan, “Feminist Visualization: Re-envisioning GIS as a Method in Feminist Geographic Research,” Annals of the Association of American Geographers 92(4) (December 2002): 645-661.
  • 22.1.2. Mei-Po Kwan and LaDona Knigge, “Guest Editorial: Doing Qualitative Research Using GIS: An Oxymoronic Endeavor?,” Environment and Planning A, 38(11) (2006): 1999-2002.
  • 22.1.3. Stan Openshaw, “A View of the GIS Crisis in Geography, or, Using GIS to Put Humpty-Dumpty Back Together Again,” Environment and Planning A (23) (1991): 621-628.

Recommended:

  • 22.1.4. Meghan Cope and Sarah Elwood, Qualitative GIS: A Mixed Methods Approach (Sage, 2009).
  • 22.1.5. William J. Craig, Trevor M. Harris, and Weiner Daniel, Community Participation and Geographic Information Systems (Taylor & Francis Inc., 2002).
  • 22.1.6. Steven J. Steinberg, GIS: Geographic Information Systems for the Social Sciences: Investigating Space and Place (Sage, 2006). 

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                    

2:00pm - 3:30pm Basic GIS Functions

This module will explore basic visualization and analytical functions such as building and querying attribute tables, selecting map features, and symbolizing data.

  • 22.2.1. Samuel F. Dennis Jr., “Prospects for Qualitative GIS at the Intersection of Youth,” Environment and Planning A, 38(11) (November 2006): 2039-2054.
  • 22.2.2. Jeremy Mennis, Michael Mason, and Yinghui Cao, “Qualitative GIS and the Visualization of Narrative Activity Space Data,” International Journal of Geographical Information Science 27(2) (February 2013): 267-291.

Recommended:

  • 22.2.3. David Allen, GIS Tutorial 2: Spatial Analysis Workbook, 10.1 ed. (ESRI Press Inc., 2013).
  • 22.2.4. David W. Allen and Jeffery M. Coffey, GIS Tutorial 3: Advanced Workbook, 10.0 ed. (ESRI Press Inc., 2010).
  • 22.2.5. Gorr L. Wilpen and Kristen S. Kurland, GIS Tutorial 1: Basics Workbook, 10.1. ed. (ESRI Press Inc., 2013).

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm  GIS Data Sources and Data Integration

This module will review the types and sources of data that are available for GIS users, the ethics of mapping certain data, how metadata can be used to communicate qualitative information, and data overlay analysis.

  • 22.3.1. Read through the “Introduction to Geospatial Metadata – FGDC CSDGM Metadata” course files, particularly “Introduction to Metadata.” Available at http://service.ncddc.noaa.gov/cdn/metadata-training-materials/Intro-to-FGDC/list.html. (accessed April 2014).
  • 22.3.2. Jin-Kyu Jung and Sarah Elwood, “Extending the Qualitative Capabilities of GIS,” Transactions in GIS 14(1) (February 2010): 63-87.
  • 22.3.3. Pamela Wridt, “A Qualitative GIS Approach to Mapping Urban Neighborhoods with Children to Promote Physical Activity and Child-Friendly Community Planning,” Environment and Planning B: Planning and Design 37(1) (2010): 129-147.

Recommended:

  • 22.3.4. Ian N. Gregory, A Place in History: A Guide to Using GIS in Historical Research, 2nd ed. (Centre for Data Digitisation and Analysis, 2005).
  • 22.3.5. Mark Monmonier, How to Lie With Maps, 2nd ed. (University of Chicago Press, 1996).
     

Thursday, June 26 Module 23  Counterfactual Analysis: Jack Levy and Frank Harvey

All causal claims have counterfactual implications. They generate statements about what is likely to happen if certain variables were to assume different values. This module focuses on the uses of counterfactuals in historical interpretation, with the primary aim of assessing the utility of counterfactuals in supporting causal inferences. In the first session Jack Levy discusses the importance of counterfactuals, various types of counterfactuals, and the criteria for evaluating the utility of counterfactuals. In the second session Frank Harvey applies these and related criteria to the 2003 Iraq War. In the third session Jack Levy examines some of the counterfactuals of the First World War, and Frank Harvey explores the policy relevance of counterfactual analysis, the contributions the approach can make to narrowing the gap between theory and policy, and the value of projectibility when assessing competing counterfactual claims.

8:45am - 10:15am: Counterfactuals: Uses, Types, and Criteria for Evaluation

Jack S. Levy, Rutgers University

We distinguish between “plausible world” and “miracle” counterfactuals and between idiographic and nomothetic counterfactuals, and note the potential role of counterfactuals in facilitating new ways of thinking and exposing logical inconsistencies and moral double standards. We then focus on idiographic, plausible world counterfactuals. We develop methodological criteria for evaluating the utility of these counterfactuals in analyzing how history might have taken a different course and in assessing the validity of causal inferences in particular historical episodes.

  • 23.1.1. Jack S. Levy, “Counterfactuals and Case Studies.” In Janet Box-Steffensmeier, Henry Brady, and David Collier, eds., Oxford Handbook of Political Methodology (Oxford University Press, 2008), pp. 627-644.
  • 23.1.2. Philip E. Tetlock and Geoffrey Parker, “Counterfactual Thought Experiments: Why We Can’t Live without Them and How We Must Learn to Live with Them.” In Philip E. Tetlock, Richard Ned Lebow, and Geoffrey Parker, eds., Unmaking the West: “What If?” Scenarios That Rewrite World History (University of Michigan Press, 2006), pp. 14-44.

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                    

2:00pm - 3:30pm Comparative Counterfactual Analysis: A Case Study of the 2003 Iraq War

Frank P. Harvey, Dalhousie University

We discuss the contributions of comparative counterfactual analysis and explore theory-based linkages between historical evidence (compiled across multiple levels of analysis) and the relative strengths and weaknesses of competing theories and causal claims about the onset of the Iraq war. The presentation will focus on the need to engage both confirming and disconfirming evidence when assessing the quality of any necessary condition theory, and will discuss the relative strengths of path dependence as an alternative explanation of US/UK decision-making from 2002-2003 that challenges conventional accounts.

  • 23.2.1. Frank Harvey, “President Al Gore and the 2003 Iraq War: A Counterfactual Test of Conventional ‘W’isdom, Canadian Journal of Political Science 45(1) (March 2012): 1-32. Online access also includes Supplementary Material: Appendix 1, Appendix 2, and Bibliography for Appendices: www.journals.cambridge.org/cjp/S0008423911000904sup001.
  • 23.2.2. Klaus Dodds, “Counter-Factual Geopolitics: President Al Gore, September 11th and the Global War on Terror, Geopolitics 13(1) (December 2008): 73-99.

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm  Applications: The First World War; Post-Iraq War Intelligence Reform; Projectibility and Continuity in US Foreign Policy.

Jack S. Levy, Rutgers University and Frank P. Harvey, Dalhousie University

This session has two parts. First, we use the criteria developed in the last two sessions to explore some famous “what if’s” of the First World War. Would the war still have occurred if Austrian Archduke Franz Ferdinand had not been assassinated, or if Germany had abandoned the Schlieffen Plan and adopted instead an eastern-oriented “defensive” military strategy toward Russia, or if Britain had clearly signaled in advance its intentions to intervene in a continental war? Second, we examine the policy relevance of counterfactual analysis with specific emphasis on post-Iraq war intelligence reform, and discuss the value of projectibility when assessing the quality of competing counterfactual claims, focusing specifically on continuity in US foreign policy across the Clinton, Bush and Obama administrations.

  • 23.3.1. Richard Ned Lebow, “Contingency, Catalysts, and Nonlinear Change: The Origins of World War I.” In Gary Goertz and Jack S. Levy, eds., Explaining War and Peace: Case Studies and Necessary Condition Counterfactuals (Routledge, 2007), pp. 85-111.
  • 23.3.2. Frank Harvey, Explaining the Iraq War: Counterfactual Theory, Logic, and Evidence (Cambridge University Press, 2012), Chapters 1 and 10.
  • 23.3.3. Richard J. Evans, “‘What if’ is a Waste of Time: Counterfactual History is Misguided and Outdated, as the First World War Debate Shows,” The Guardian (March 14, 2014). Available at http://www.theguardian.com/books/2014/mar/13/counterfactual-history-what-if-waste-of-time.

Recommended Reading for Counterfactuals Module 

Robert Cowley, What If?: The World’s Foremost Military Historians Imagine What Might Have Been (G. P. Putnam’s Sons, 1999). See also Cowley’s other “What If?” volumes.

Niall Ferguson, ed., Virtual History: Alternatives and Counterfactuals (Basic Books, 1999).

Gary Goertz and Jack S. Levy, eds., Explaining War and Peace: Case Studies and Necessary Condition Counterfactuals (Routledge, 2007).

Frank P. Harvey, Explaining the Iraq War: Counterfactual Theory, Logic, and Evidence (Cambridge University Press, 2012).

Richard Ned Lebow, Forbidden Fruit: Counterfactuals and International Relations (Princeton University Press, 2010).

David Lewis, Counterfactuals (Harvard University Press, 1973).

Neal J. Roese and James M. Olson, eds., What Might Have Been: The Social Psychology of Counterfactual Thinking (Lawrence Erlbaum Associates, 1995).

Paul W. Schroeder, “Embedded Counterfactuals and World War I as an Unavoidable War” in Systems, Stability, and Statecraft: Essays on the International History of Modern Europe, David Wetzel, Robert Jervis, and Jack S. Levy, eds. (Palgrave, 2004), pp. 157-191.

Philip E. Tetlock, Expert Political Judgment (Princeton University Press, 2005).

Philip E. Tetlock and Aaron Belkin, eds., Counterfactual Thought Experiments in World Politics (Princeton University Press, 1996).

Philip E. Tetlock, Richard Ned Lebow, and Geoffrey Parker, eds., Unmaking the West: “What If?” Scenarios That Rewrite World History (University of Michigan Press, 2006).


Friday, June 27 Module 24  Challenges of Medium-N Analysis II – David Collier, Burt L. Monroe, Bear Braumoeller, and Christopher Krogslund

This module continues the discussion of medium-N methods, focusing both on (a) the wider discussion of these methods, and (b) two clusters of methods that show promise for addressing a medium-N: Correlation and Regression Trees (CART) and resampling methods.

8:45am ‐ 10:15am. Alternative Approaches to Boolean Inference.

David Collier, University of California, Berkeley

Burt L. Monroe, Pennsylvania State University

Qualitative Comparative Analysis has been formulated for evaluating complex causal conditions in the small and medium-N settings common in comparative politics and sociology. One line of critical commentary has centered the unusual features of Boolean inference, but in fact these issues are isomorphic to problems faced in genetics and other fields that are "scientific." This unit focuses instead on the concern that QCA does not generate useful Boolean inferences in any setting, and also that Boolean inference is poorly suited to small- and medium-N analysis. The present discussion considers serious flaws in QCA and introduces alternative approaches to Boolean inference, including those developed in operations research and bioinformatics.

  • 24.1.1. Jack Crenshaw, “All about Quine-McClusky,” Embedded (August 19, 2004). Available at http://www.embedded.com/electronics-blogs/programmer-s-toolbox/4025004/All-about-Quine-McClusky.
  • 24.1.2. Bear F. Braumoeller, “Causal Complexity and the Study of Politics.” Political Analysis (2003) 11(3): 209–33.
  • 24.1.3. Carsten Q. Schneider and Claudius Wagemann, “Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets,” Comparative Sociology 9(3) (2010): 397-418.

Background Reading:

In the following symposium, focus on Carsten Q. Schneider and Claudius Wagemann, “Doing Justice to Logical Remainders in QCA: Moving Beyond the Standard Analysis,” pp. 211-220 and Martino Maggetti and David Levi-Faur, “Dealing with Errors in QCA,” pp. 198-204.

  • 24.1.3. Benoit Rihoux and Axel Marx, “QCA, 25 Years after ‘The Comparative Method’ Mapping, Challenges, and Innovations—Mini-Symposium,” Political Research Quarterly 66(1) (2013): 167-235.

10:15-10:45am  Coffee Break

10:45am-12.30pm Research Design Discussion Sessions

12: 30pm – 2:00pm pm Lunch

2:00pm – 3:30pm Resampling Methods

Bear Braumoeller, Ohio State University

David Collier, University of California, Berkeley

Over many years, methodologists have developed a number of tools—called resampling methods—that can increase analytic leverage in situations where the size of the N is sufficiently small as to raise questions about the reliability of findings. Some of these tools have clever, metaphorical names: Jackknife and Bootstrap; others more conventional names—e.g., cross-validation; and also names that are tongue-twisters—e.g., non-parametric combination of dependent tests (NPC).

These tools show great promise for strengthening medium-N analysis, but questions arise. For example, what sorts of hypotheses can they test? Which are most promising for medium-N analysis, and under what circumstances? Is their focus on randomization-based rather than population-based inference problematic for social science applications that emphasize external validity? How should the contribution of permutation tests be assessed, given the answers to these questions?

  • 24.2.1. John Ludbrook and Hugh Dudley, “Why Permutation Tests Are Superior to T and F Tests in Biomedical Research,” The American Statistician 52(2) (May 1998): 127-132.
  • 24.2.2. Bear F. Braumoeller, “Fuzzy Dice: Probability and the Study of fsQCA.” Department of Political Science, Ohio State University.
  • 24.2.3. Devin Caughey, Allan Dafoe, and Jason Seawright, “Testing Elaborate Theories in Political Science: ‘Nonparametric Combination’ of Dependent Tests.” Department of Political Science, Massachusetts Institute of Technology, 2013.

Recommended

  • 24.2.4. Thomas M. Carsey and Jeffery J. Harden, Monte Carlo Simulation and Resampling Methods for Social Science (Sage, 2013).

3:30pm - 4:00pm Coffee Break

4:00pm - 5:30pm Classification and Regression Trees (CART).

David Collier, University of California, Berkeley

Christopher Krogslund, University of California, Berkeley

The decision-tree learning methods called CART (Classification and Regression Trees) and “Random Forests” (CART) appear well-suited to medium‐N data, because they: (a) Provide valuable transparency that allows the analysts to readily mobilize case knowledge, while at the same time benefitting from highly systematized analytic procedures; (b) Can readily accommodate both categorical and continuous data and do not require data transformations that can lead to loss of information; and (c) Are not unduly influenced—in contrast to some other classificatory algorithms—by the higher levels of error routinely found in social science data.

The following issues are addressed in this unit: Given that CART and Random Forests are routinely treated as a “data mining” method, a research tradition quite removed from the larger spectrum of methods considered here, do they in fact contribute to the goals that motivate these modules? Are they well-suited to medium-N analysis? Can they address contextual effects, interaction, and asymmetric causation?

  • 24.3.1. Jonathan P. Kastellec, “The Statistical Analysis of Judicial Decisions and Legal Rules with Classification Trees.” Journal of Empirical Legal Studies 7, no. 2 (2010): 202–30. Read pp. 209–213 only, for a concise summary of CART.
  • 24.3.2. Jason Seawright,  “Warrantable and Unwarranted Methods: The Case of QCA.” Paper Presented at the 2013 Annual Meeting of the American Political Science Association, Chicago IL., 2013. Read pp. 13–22 only..
  • 24.3.3. Leo Breiman, “Statistical Modeling: The Two Cultures,” Statistical Science 16(3) (August 2001): 199-231.
  • 24.3.4. Leo Breiman, “Random Forests,” Machine Learning 45(1) (October 2001): 5-32.

Recommended

  • 24.3.5. Leo Breiman, Leo, Jerome Friedman, Charles J. Stone, and R.A. Olshen, Classification and Regression Trees (Chapman and Hall, 1984).
  • 24.3.6. Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. (Springer, 2013).


Friday, June 27 Module 25  Geographic Information Systems II: Exploring GIS Analytic Capabilities – Jonnell Robinson

8:45am - 10:15am: Geocoding and XY Coordinate Data

This module will explain geocoding (plotting street addresses) and coordinate-based data such as GPS waypoints, as well as common tools for point data analysis such as buffering and labeling.

Recommended:

  • 25.1.3. Juliana Maantay and John Ziegler, GIS for the Urban Environment (ESRI Press Inc., 2006).

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.                    

2:00pm - 3:30pm Digitizing and Editing

This module will demonstrate “heads-up” digitizing, or turning print images and scanned maps into a GIS map, and editing map features manually in ArcGIS.

Recommended:

  • 25.2.2. John Pickles, Ground Truth: The Social Implications of Geographic Information Systems (The Guilford Press, 1995).
  • 25.2.3. Denis Woods, The Power of Maps (The Guilford Press, 1992).

3:30pm - 4:00pm Coffee Break.

4:00pm - 5:30pm Projections and Map Design

This module will cover projections and coordinate systems, map design, integrating narrative and photos with GIS, and a discussion about how to further hone GIS skills.

Recommended:

  • 25.3.3. Cynthia A. Brewer, Designing Better Maps: A Guide for GIS Users (ESRI Press, Inc., 2005).
  • 25.3.4. Heather MacDonald and Alan Peters, Urban Policy and the Census (ESRI Press, Inc., 2011).
  • 25.3.5. Andy Mitchell, The ESRI Guide to GIS Analysis: Geographic Patterns & Relationships, vol. 1 (ESRI Press, Inc., 1999).
  • 25.3.6. Andy Mitchell, The ESRI Guide to GIS Analysis: Spatial Measurements & Statistics, vol. 2 (ESRI Press, Inc., 2005).
  • 25.3.7. Andy Mitchell, The ESRI Guide to GIS Analysis: Modeling Suitability, Movement, and Interaction, vol. 3 (ESRI Press, Inc., 2012).
  • 25.3.8. Mark Monmonier, Mapping it Out: Expository Cartography for the Humanities and Social Sciences (University of Chicago Press, 1993).

Friday, June 27 Module 26  Archival Research and Elite Interviews - James Goldgeier, Andrew Moravcsik, and Elizabeth Saunders

Archival and Interview Research with Primary Sources: What Do You Need to Know, How Do You Know Where to Look, and How Do You Get What You Need?

In this module, we will discuss how political scientists decide they need to use primary records of policy-making—archives, interviews, and published primary sources—in their research. This includes how one prepares for, structures, conducts, and manages the information flow from archival visits, interviews or structured examination of published materials. We focus on practical research skills scholars can use, and judgments they must make in everyday research. We conclude with a discussion of making qualitative research transparent.

8:45am - 10:15am Selecting and Preparing for Archival and Interview Research

This session highlights the practical trade-offs between different types of textual and interview research and the ways in which one must prepare for them.  It focuses on issues to think about before you start your research.  We will talk about different types of repositories, briefly explain how to use the Freedom of Information Act, and strategies for maximizing the output of interviews.

  • 26.1.1. Fred I. Greenstein and Richard H. Immerman, “What Did Eisenhower Tell Kennedy About Indochina? The Politics of Misperception,” Journal of American History 79(2) (September 1992): 568-587.
  • 26.1.2. Cameron Thies, “A Pragmatic Guide to Qualitative Historical Analysis in the Study of International Relations,” International Studies Perspectives 3(4) (November 2002): 351-372.
  • 26.1.3. Ian Lustick, “History, Historiography, and Political Science: Multiple Historical Records and the Problem of Selection Bias,” American Political Science Review 90(3) (September 1996): 605-618.
  • 26.1.4. Marc Trachtenberg, The Craft of International History (Princeton University Press, 2006), Appendix I and Appendix II. Available at

http://www.sscnet.ucla.edu/polisci/faculty/trachtenberg/methbk/AppendixI.html

http://www.sscnet.ucla.edu/polisci/faculty/trachtenberg/methbk/AppendixII.html

10:15am - 10:45am Coffee Break.

10:45am - 12:30pm Research Design Discussion Sessions (not part of Module).

12: 30pm - 2:00pm  Lunch.

2:00pm - 3:30pm Structuring Your Data Collection: Making Sure You Can Use What You Find

This session will address concerns that arise during your research.  We will discuss hands-on electronic strategies for structuring, organizing, and storing your oral and documentary data so that you can easily and systematically access it as you move to the analysis and writing phase of your project.  The process of structuring your data begins before you leave for the archives, and informs how you conduct your research in the archives and your analysis of documents when you get home.3:30pm -4:00pm Coffee Break.

4:00pm – 5:30pm  Transparency in Qualitative Research

This session focuses on analyzing your data after you gather it, as well as making it available for other scholars to examine and utilize in the future.  In addition to discussing the uses of historical research for building theoretical arguments, we will discuss how different scholars can read documents differently, and also how documents can be used in thinking about counterfactuals.  We will also introduce participants to the emerging debate about practical research standards for transparency and replicability of qualitative work: active citations, data archiving, etc.

  • 26.3.1. Andrew Moravcsik, “Active Citation: A Precondition for Replicable Qualitative Research,” PS: Political Science and Politics 43(1) (January 2010): 29-35.
  • 26.3.2. Colin Elman, Diana Kapiszewski, and Lorena Vinuela, “Qualitative Data Archiving: Rewards and Challenges,” PS: Political Science and Politics 43(1) (January 2010): 23-27.



Follow Maxwell at:

twitter facebook youtube linkedin blog rss
Syracuse University Give Now

Center for Qualitative and Multi-Method Inquiry | Maxwell School | Syracuse University | 346 Eggers Hall | Syracuse, NY 13244-1090 | 315.443.6198