Qualitative Comparative Analysis (QCA) in public policy analysis: an extensive review.
| Verfasser | Rihoux, Benoit |
| Posten | P. 52-82 |
On a linked topic, Sprinz & Kaan (2007) seek to probe which design features account for highly effective international environmental regimes, and which are more persistently associated with ineffective regimes. They process a dataset of 23 international environmental treaty regimes and 92 subregimes, focusing on 4 potential explanatory factors (enforcement, compliance-monitoring, legalization, and knowledge) both through cross-sectional statistical procedures and fsQCA. First results of the latter indicate that strong legalization, as well as low enforcement, are necessary conditions for regime effectiveness.
Finally, Kostadinova (2003) analyzes the European Commission's (EC) opinions on applicant countries and examines the conditions for being invited to start negotiations for EU membership. Through a csQCA on 15 country cases as seen from the EC (therefore the transnational nature of her cases), she demonstrates that satisfaction of the political requirements, jointly with the presence of a functioning market economy are necessary, but not sufficient, conditions for being in the first wave of expansion. In addition, a country has had to show progress in either its capacity to meet the other obligations of membership or its ability to withstand the competitive pressures within the Union. With respect to the second wave of negotiations, she shows that the only necessary condition is meeting the political criteria. Finally, the factor which determines which countries were invited to join the EU in 2004 is the presence of the necessary administrative structures.
7 Focus on policy evaluation
This strand of applications is, so far, less diversified, and has been developed chiefly through a specific team of scholars.
[20] [21] Mostly at the subnational level, Befani, Sager and Ledermann elaborate ways to feed QCA into a 'Realistic Evaluation' perspective. A first empirical application is developed by Ledermann (2004, also 2011) on 12 cases of external evaluations of developmental aid projects supported by Swiss funds. In a more refined empirical application (Befani & Sager 2006; Sager & Ledermann 2006a, 2006b; Befani, Ledermann & Sager 2007), they exploit a study from the evaluation of the Swiss Environ- mental Impact Assessment (EIA), in which three types of different outcomes are evaluated. Following the realist paradigm, initial assumptions are made on which Context-Mechanism-Outcome (CMO) configurations explain the different types of policy results. The propositions stemming from these assump-tions are then translated into a set of Boolean variables, and a csQCA model is then constructed. The csQCA, performed on 15 case studies across Switzerland, produces core combinations of conditions which are, in turn, used to refine the initial assumptions (on which mechanisms were activated in which contexts to achieve which outcomes). The theory refinement made possible by QCA covers both directions on the abstraction to specification scale: downward, it offers more elaborate configurations able to account for a certain outcome; upward, it aggregates relatively specific elements into more abstract ones ('realist synthesis'). They finally argue that QCA has the potential to expand the scope and possibilities of Realistic Evaluation, both as an instrument of theory refinement and as a tool to handle realist synthesis when the number of cases is relatively high.
In another vein, Balthasar (2006) analyses the influence of the institutional distance between evaluators and evaluees on the utilization of policy evaluations. He uses csQCA to perform a metaanalysis of 10 case studies in the Swiss context, which involve evaluations that were carried out in different institutional contexts. This analysis is complemented by a larger-N statistical analysis on the whole population of ca. 300 evaluations in Switzerland (Balthasar 2009). His csQCA results indicate that, under certain conditions, the institutional distance between evaluators and evaluees has no influence on the use of evaluations. In particular, formative objectives can be achieved quite independently of distance.
[24] Finally, in a more generic way, Varone, Rihoux & Marx (2006) discuss how QCA can contribute to facing up key challenges for policy evaluation. They identify four challenges: linking policy interventions to outcomes and identifying causal mechanisms which link interventions to outcomes; identifying a 'net effect' of policy intervention and purge out the confounding factors; answering the 'what if'-question (i.e. generate counterfactual evidence); and triangulating evidence. They argue that QCA offers some specific answers to these challenges, as it allows for a three-way comparison: a cross-case analysis, a withincase analysis, and a comparison between empirical reality and theoretical ideal types. They also discuss how QCA could deal with the contradictions/uniqueness trade-off (too few v/s too many conditions), to further develop the use of QCA in policy evaluation.
Conclusion: the state of the field - and the next steps
In this conclusion - which will, naturally, remain open due to the relatively recent nature of QCA and its current further developments and expected innovations - we first aim to take stock of what has been achieved so far. On that basis, we discuss some main avenues for further developments.
With regards to the strengths and assets in the field so far, one should first mention the research results. The application of QCA techniques (csQCA, mvQCA, fsQCA) to public policy analysis topics has begun to develop a 'niche' which has, so far, yielded quite an amount of useful empirical results. In a significant number of instances, it has also enriched the theories and models in the field. Hence QCA has proven useful to feed the "dialogue between ideas and evidence" (Ragin 1987). For instance, as Befani, Ledermann & Sager (2006) state it with regards to evaluation research specifically, QCA, in connection with other policy analysis tools, can produce empirically well-grounded context-sensitive evidence on policy instruments. Indeed the QCA results, in the form of core combinations of conditions, are both quite easy to grasp once they are translated verbally (from the perspective of policy practitioners and policy-makers) and quite complex due to their combinational nature (from the perspective of policy analysts).
Also noticeable is the fact that some of those applications have made their way to toptier generalist journals - clearly more so in the sociology journals in the 1990s and the turn of 2000 (e.g. Amenta et.al. (1992) in American Journal of Sociology, or Amenta & Halfmann (2000) in American Sociological Review), but since then quite a number of well-ranked political science or policy analysis journals have also become receptive (e.g. European Journal of Public Policy, European Journal of Political Research, Mobilization, Regulation and Governance, Evaluation, West European Politics, ...).
Further, it should be noted that a large proportion of such applications are not singlemethod (i.e. solely relying on QCA) - they combine, or confront, or enrich QCA with at least one other method or approach. On the one hand, many applications triangulate, or exploit in complementary ways, both statistical analyses (mainly regression analyses of different types) and QCA. On the other hand, many applications are "case-oriented" and rely on thick case studies - a rejoinder to the call to exploit QCA to add leverage to comparative case studies (Rihoux & Lobe 2009; Byrne, Olsen & Duggan 2009). This case-informed QCA work takes many forms, e.g. through the exploitation of interviews (Hyttinen et.al. 2000), in-depth cases studies (Kitchener et.al. 2002), sequence analyse (Bleijenbergh & Roggeband 2007), etc.
Another obvious point is that QCA applications in the field have by now become quite diverse in their scope, both in terms of policy domains, of stage of the policy-making process, and of level of analysis. Some subfields are however much better covered than others - especially analyses: (a) on socioeconomic (in particular welfare state) or environmental policies; (b) focusing on policy design or policy outputs; (c) with cases being defined at the country level, or at the local or subnational level. In terms of number of cases, the bulk of applications also remains within the usual range for (cs)QCA as it was initially developed by C. ragin, i.e. between 7-8 and 25-30 cases. This is probably due to the fact that most applications are conducted at the macro level (typically: countries) where the number of cases is 'naturally' limited. However there are a few exceptions to this standard practice, with some larger-N analyses, e.g. Aleman (2009 - 78 cases), Rudel (2005 - 80 countries comprising subnational cases), Olsen & Nomura (2009 - 82 cases); Sprinz & Kaan (2007 - 92 cases), Schwellnus et.al. (2009 - 93 cases), McBride & Mazur (2010 - 130 cases), Gran (2005 - 190 cases) and Gran & Aliberti (2003 - 193 cases).
Next, the largest share of applications so far have followed a rather 'inductive' or 'soft theoretical' approach - i.e.: either QCA was used to test a series of conjectures or proposals (translated into conditions), or to test a theory translated into a series of separate conditions, out of which eventually some configurational results emerged. Only few applications have more fully exploited the configurational and set-theoretic nature of QCA, which can be done it at least two ways. On the one hand, 'configurational' theories or hypotheses can be tested as such - including some 'upstream' causal or configurational statements linking the conditions in the model - as first applied by Amenta, Caren & Olasky (2005; see above) and advocated a.o. by Yamasaki & Rihoux (2009: 128-129). On the other hand, QCA applications can be (and should be, insofar as possible) explicitly based on set-theoretic relations for causal analysis (in terms...
Um weiterzulesen
FORDERN SIE IHR PROBEABO ANCOPYRIGHT GALE, Cengage Learning. All rights reserved.
