Policy analysis, science, and politics:

from ‘speaking truth to power’ to ‘making sense together’.

by Robert Hoppe

(University of Twente)



1 Still speaking truth to power?

According to Lasswell (1971), policy science is about the production and application of knowledge of and in policy. Policymakers who desire to successfully tackle problems on the political agenda, should be able to mobilize the best available knowledge. This requires high-quality knowledge in policy. Policymakers and, in a democracy, citizens, also need to know how policy processes really evolve. This demands precise knowledge of policy. There is an obvious link between the two: the more and better knowledge of policy, the easier it is to mobilize knowledge in policy. Lasswell expresses this interdependence by defining the policy scientist’s operational task as eliciting the maximum rational judgment of all those involved in policymaking

For the applied policy scientist or policy analyst this implies the development of two skills. First, for the sake of mobilizing the best available knowledge in policy, s/he should be able to mediate between different scientific disciplines. Second, for the sake of optimizing the interdependence between science in and of policy, s/he should be able to mediate between science and politics. Hence Dunn’s (1994:84) formal definition of policy analysis as an applied social science discipline that uses multiple research methods in a context of argumentation, public debate (and political struggle, rh) in order to create, critically evaluate, and communicate policy-relevant knowledge. Historically, the differentiation and successful institutionalization of policy science can be interpreted as the scientization of the functions of knowledge organization, storage, dissemination and application in the knowledge system (Dunn & Holzner, 1988; Van de Graaf & Hoppe, 1989:29). Moreover, this scientization of hitherto ‘un-scientized’ functions, by expressly including science of policy, aimed to gear them to the political system. In that sense, Lasswell and Lerner’s (1951) call for policy sciences anticipated, and probably helped bring about the scientization of politics.

Peter Weingart (this issue) claims that the development of the science-policy nexus can be analyzed as a dialectical process of the scientization of politics/policy and the politicization of science. Science Technology and Society (STS) studies can claim particular credit for showing the latter tendency (Cozzens & Woodhouse, 1995:551). Applying critical sociology, symbolic interactionism and ethnomethodology to the innermost workings of the laboratories, STS-scholars have shown that the idealist image of science as producer of privileged, authoritative knowledge claims, supported by an ascetic practice of Mertonian norms for proper scientific conduct (commonality or communism, universalism, disinterestedness, organized skepticism - CUDO’s) is just the outside, legitimizing veneer of scientific practices and successes. Using interpretive frames from Marxist science studies, conflict theory, interest theory, and social constructivism, a much more realistic perspective on science has been developed. Instead of Mertonian CUDO-norms, contemporary scientists de facto behave as if science were proprietary, local, authoritarian, commissioned, and expert (Ziman, 1990 - PLACE). From Olympian heights of abstraction, curiosity-driven speculation, innovative but stringent experiments, and Humboldtian institutional autonomy, small-s science came down to earth as a social movement (Yearley, 1988:44ff) driven by local and practical, sometimes openly political interests, entrepreneurial, fiercely competitive, speculative, with an ‘anything goes’ methodology, and selling itself to government and big business in the race for financial resources. Thus, the politics of science extended into the political domain. But it would be wrong to attribute this just to science’s institutional self interest. To the extent scientists were successful in producing authoritative cosmopolitan knowledge claims, and upholding them in their translation into successful large technological projects, they were invited by politicians and administrators as useful advisers. Thereby politics paradoxically contributed to its own scientization. At first, till the early seventies, it looked like the science-politics nexus would be just mutually beneficial. The institutional ‘convenant’ between the two spheres, aptly named "Science, the Endless Frontier" meant a high degree of institutional autonomy, lots of resources, and privileged access to political decisionmaking through advisory positions for science. Politics, impressed by and grateful for science’s contribution to the war effort and to large infrastructural projects, rested content in expecting more of the same high pay-offs. As these promises turned out empty or merely disappointing, sciences’ cognitive authority waned, and politics gradually revised the convenant by tightening its conditions for financial support and scientific autonomy. The new inter-institutional contract has been relabeled "Strategic Science". On the one hand, politics forces criteria of relevance on scientists, which clearly indicates the politicization of science. On the other hand, "(s)cientists have internalized the pressure for relevance, but at the same time have captured it for their own purposes by claiming a division of labour. Typical stories emphasize strategic research as the hero at the core of one or more ‘innovation chains’ where the switch from open-ended research to implementation would occur" (Rip,1997:631). This, of course, points to the continued scientization of politics.

Even though numerous studies of political controversies showed that science-advisors behave pretty much like any other self-interested actor (Nelkin, 1995), science somehow managed to maintain its functional cognitive authority for politics. This may be due to its changing shape, which has been characterized as the diffusion of the authoritative allocation of values by the state, or the emergence of a postparliamentary and postnational network democracy (Andersen & Burns, 1996: 227-251). National political developments are backgrounded by a pulp of ideas about uncontollable, but apparently inevitable international developments; and, in Europe, national state authority and power in public policymaking is leaking away to a new political and administrative élite, situated in the institutional ensemble of the European Union. National representation is in the hands of political parties who no longer control ideological debate but remain intact as venues to national governmental power. The authority and policymaking power of national governments is also leaking away towards increasingly powerful policy subsystems or policy issue networks, dominated by functional representation by interest groups and functional experts.

In this situation, public debate has become even more fragile than it has been before. It has become diluted by the predominance of purely pragmatic, managerial and administrative argument, and underarticulated due to an explosion of numerous new political schemata that crowd out the more conventional ideologies. To wit, the new schemata do feed upon the conventional ideologies; but in larger part they consist of a random and unarticulated ‘mish-mash’ of attitudes and images derived from ethnic, local-cultural, professional, religious, social movement, and personal political experiences. On the one hand, the marketplace of political ideas and arguments is thriving; on the other, politicians and citizens are at a loss in judging its nature and quality. Neither political parties, nor public officials, nor interest groups, nor social movements and citizen groups, nor even the public media show any inclination, let alone competency, in ordering this inchoate field. In such conditions, scientific debate provides a much needed minimal amount of order and articulation of concepts, arguments, and ideas. Although frequently more in rhetoric than substance, reference to scientific ‘validation’ does provide politicians, public officials, and citizens alike with some sort of compass in an ideological universe in disarray.

For policy analysis to have any political impact under such conditions, it should be able to somehow continue ‘speaking truth’ to political élites who are ideologically uprooted, but cling to power; to the élites of administrators, managers, professionals and experts who vie for power in the jungle of organizations populating the functional policy domains of postparliamentary democracy; and to a broader audience of an ideologically disoriented and politically disenchanted citizenry. But what does it mean to ‘speak truth to power’ in contemporary society and politics? To answer this question, first, I turn to the megatrends in epistemological debate in the second half of the twentieth century. On that basis, second, I will try to delineate its implcations for the actual and future development of policy analysis.



2 Epistemology: from instrumental to fallibilist-pragmatist rationality

Once upon a time social, political, managerial and administrative élites genuinely believed in scientific rationality as a key to solving collective (and personal) problems. Like scientists themselves, they were inheritors of the Enlightenment, who pictured unfettered growth of scientific knowledge as the driving force of social progess and individual ‘pursuit of happiness’. But after two World Wars, the Shoah, the nuclear race, the ecological crisis, and the fall of ‘scientific’ communism, belief in scientific rationality is decaying. In all cases mentioned, science and scientists are, to a greater or lesser extent, accessory to human suffering and ecological degradation. For religious fundamentalists and modern neo-tribalists this suffices to reject science in a ‘rage against reason’. But even postmodernists reject claims to ground political and social ideas in scientific, rational, logical, and consistent argument as potentially exclusive, imposing, suppresive, technocratic, and ultimately undemocratic. Instead, they celebrate otherness, incompatibilities and ruptures between life styles, cultures, discourses, pluralism, the decentered ego, and the uniqueness, contingency, and fragmentation of all social phenomena. Richard Bernstein (1991) has aptly characterized this new intellectual force-field as the polarity of a ‘Both/And’ situation: the modernist idea of the Enlightenment as ‘Unfinished Project’ and the post-modernist idea of the Enlightenment as ‘Historical Error’ are like opposites that can never be reconciled, yet are inextricably intertwined in that they mutually elicit and illuminate each other. Therefore, it is unnecesary to push matters to an extreme. I would rather cast the modernism-postmodernism divide as different accents within a markedly revised concept of scientific rationality.

First, the conviction that empirical-analytic scientific procedure alone may lay claim to scientific rationality has become untenable. In this (neo)positivist conception, science is based on strictly neutral, objective, carefully controlled sense observation of physical and social facts. Long observation is supposed to uncover regularities and patterns, which, crafted into abtract hypotheses, are amenable to further rigorous testing. Hypotheses surviving these further tests, may be used in the formulation of deductive systems of lawlike propositions, in which they enter as the general premises in the covering-law model of truly scientific explanation and prediction. Habermas (1971) has shown that this idea corresponds to just one knowledge interest constitutive of science, i.e. the domain of labor, work, and human control over a physical or social environment. But humans know more action domains, and therefore knowledge interests. Interaction and mutual understanding of action motives and meanings is a second knowledge interest. It lends the interpretive and hermeneutic sciences their legitimate claim to scientific rationality. Where meaningful interactions are suffocated by unconscious collective images or pre-understandings which deserve articulation, reflection, and critique, there is a legitimate task for critical science. Empirical analysis of data, skillful interpretation of socially constructed meanings, and social critique are equally important, vital elements of an enlarged concept of scientific rationality.

Second, it is now near universally acknowledged by scientists that scientific knowledge is fallible. The Cartesian ‘Either/Or’ position has been left. Who likes to be considered ‘scientifically rational’ can no longer appeal to rocklike cognitive certainties or axioms (be they God, the Cogito, or sense observation). Modern rationality rests on acknowledging that "although we must begin any inquiry with prejudgments and can never call everything into question at once, nevertheless there is no belief or thesis - no matter how fundamental - that is not open to further interpretation and criticism" (Bernstein, 1991: 327). Fallibilism implies the expectation of being proven wrong, and therefore the willingness to revise one’s insights. Rationality as openness to learning further presupposes the embeddedness of the scientist in a durable social context of dialogue and action. An action context, because only there the pragmatic alternation between thought and action exists which brings error to light. A context of critical dialogue, because this catalyzes the learning process. It is not accidental, then, that Habermas, defender of the idea of the Enlightenment par excellence, has strongly argued the position that cognitive-analytic rationality is unthinkable absent a rationality oriented towards mutual undertanding; a rationality which, thus, needs to be social, interactive, and dialogical.

Trying to save science from overcynicism, and attempting to preserve its functional authority to politics/policy, some practice-oriented epistemologists, building on the above mentioned new constellation, have moved beyond the futile quest for clear a priori demarcation criteria to distinguish science from non-science. Instead, they try to delineate rules for ‘good’ scientific practice in the context of boundary work (Gieryn, 1995; Jasanoff, 1990) at the science-politics nexus. Recognizing the patent inadequacy of normal applied science and professional consultancy in political controversies under high uncertainty and high decision stakes over issues which show emergent complexity, epistemologists Funtowicz and Ravetz (1993) have proposed new rules for post-normal science. These rules apply when (based on Van der Sluys, 1997:21):

Under such conditions, Funtowicz & Ravetz recommend application of a fine-grained system of types of uncertainty to painstakingly sift out the reducible from the irreducible uncertainties in order to set feasible research goals and priorities. Another recommendation is to strengthen the quality control of scientific arguments through systems of extended peer review. In fact, following these proposals would mean to systematize intra-boundary work between scientific disciplines and groups (internally extended peer review) and extra-boundary work between scientists, policymakers, and, sometimes, non-expert citizens (externally extended peer review).

In the work Funtowicz & Ravetz we see the implications of the paradox between the scientization of politics and the politicization of science at its highest level of reflexivity. What use is it to policy science and policy analysis?

3 Policy analysis: from analycentrism to the argumentative turn

3.1 Democratic aspirations in beginning policy science

The beginnings of policy science are usually traced back to Harold Lasswell’s intellectual underpinning of the endeavor to systematically and methodically gear the applied social sciences to the needs of long term public policymaking (Lasswell & Lerner, 1951; Lasswell, 1971). In Lasswell’s designs the relationship between policy science and the practice of politics and administration was to be democratic and pragmatist. Policy science was not a technocratic strategy in order to substitute politics with enlightened administration; nor was it cast in the role of a social technology, always at the service of politicians and administrators. For Lasswell policy science was a vital element in a political strategy to maintain democracy and human dignity in a post-World War II world. He follows in the footsteps of his pragmatist teachers, Dewey and Merriam. In the pragmatist view politics is modeled after peer review in science: it is a dialogue between expert opinion and the opinions of a larger public, in a community united by the quest for answers to shared problems. Politics is seen as probing and honest debate, and not as conflict management wich succeeds by cleverly exploiting the ignorance and incomplete knowledge of citizens. In a sense, political and policy science’s goal is not to replace ‘ordinary’ political prudence and common sense with cognitively superior scientific knowledge, but to reinvigorate and systematize them (Van de Graaf & Hoppe, 1989:61-63; Torgerson, 1995: 234, 238-239).

Lasswell’s position is remarkable. He had read Freud and Marx, and had been exposed to war propaganda enough to be sensitive to the realities of ideological manipulation and the pathological sides of politics. He had even written books about it (Lasswell, 1927, 1930). He was also keenly aware of the impossibility to re-embed political wisdom and prudence in the existing ‘communities’ of post-war America. Yet, Lasswell opted for a policy science in the service of democracy, and rational, active citizenship. Unlike famous contemporaries like Lippmann, Schumpeter, and Dror, who, convinced of the irremediable irrationality and lack of common sense of ordinary people, chose the more ‘realistic’ strategy of developing an applied social science for an enlightened political and administrative élite..

  1. Technocratic aspirations and instrumental rationality

Reality usually disappoints high aspirations. But it is ironic that policy science’s breakthrough was intimately connected to a half-hearted post-behavioral turn in political science. Political scientists’ call to recapture relevance in the face of exaggerated methodological rigor, was translated into curriculum and research program innovations focussing on the study of the content, processes, and impacts of public policy. But its purpose remained technocratic: replacing politicians’ and citizens’ ‘ordinary and local knowledge’ of policy and policymaking with a new, scientifically validated type of applied, general knowledge (Torgerson, 1995: 229-230). Better knowledge of causation, and know-how about the application of scientific logic in decisionmaking were the dominant claims on which the schools of public policy were erected in one after another American univerity, and, later, in many Eurpean countries. Testimony to the dialectics between the scientization of politics and the politicization of science, the successful institutionalization of policy science in American academia was also due to favorable labor market prospects fueled by a rising demand for policy analysis in the Kennedy and Johnson administrations (DeLeon, 1989). In Europe similar influences were at work, especially in countries with social-democratic governments (Wagner, 1991).

From an epistemological point of view, in beginning policy analysis three cross-cutting and nonexclusive currents can be discerned: analycentrism, neo-positivism, and critical rationalism (Dryzek, 1993: 217-222). Analycentric policy analysis claims cognitive superiority over practice on the basis of the scientific logic and consistency built into analytic techniques like cost-effectiveness analysis, cost-benefit analysis, statistical decision theory, and planning-programming-budgeting. The analycentric policy analyst relies on algorithms, filled with data and insights from secondary sources, either scientific or practical. His ‘value-added’ is merely to see to it that actual decisionmaking follows rigorous scientific canons of procedural rationality (Behn & Vaupel, 1982). Analycentric policy analysis has been effectively criticized for its lack of political realism, and, in spite of its alleged procedural neutrality, its introduction of politically biased assumptions in the guise of ‘technicalities’ (Tribe, 1972; Self, 1975; Wildavsky, 1979; Fischer, 1980).

Neo-positivist policy analysis grounds its claim to cognitive superiority in its knowledge of causal links. The attractiveness of a neo-positivist concept of science is that knowledge of scientific laws, in technical-instrumental fashion, may be applied to the explanation of the emergence of policy problems and the prediction of impacts of certain policy interventions. After all, if a policy is a plan for achieving particular objectives with the help of certain means, certified causal knowledge is indispensible. For objectives are consequences preferred by policymakers; and means are their chosen and manipulated causes. Although the grounding of policy analysis in causal knowledge lingers on, neo-positivist policy analysis has whithered away. The above mentioned Habermasean criticism certainly played a role here. But applied to policy analysis, neo-positivism leads to obvious self contradictions. If human behavior generally is driven by laws governing the behavior of ordinary people, why grant immunity of such laws to politicians and policymakers (Bobrow & Dryzek, 1987:132)? Also, neo-positivists overlook that causal knowledge, through men’s capacity for learning, may ‘self destruct’ the causal laws on which a policy is based.

Critical-rationalist policy analysis shares with neo-positivism its claim to superior causal knowledge. However, it strongly differs in on how to acquire it in the real world. In this respect, critical-rational policy analysis means an enormous step towards a fallibilist and learning concept of rationality. Building on Poppers falsificationism and his political philosophy of piecemeal social engineering (Popper, 1945), Campbell & Stanley (1963) have developed critical-rational policy analysis into a sophisticated methodology of (quasi-)experimental impact evaluation. In their view, knowledge acquisition and progress is an evolutionary process of learning from trial and error in successive efforts to compare hypotheses to experimentally generated impacts. This is true for both ordinary and scientific knowledge. Science is the more efficient learning strategy due to stricter requirements for the conditions of learning and the interpretation of results. Applied to policymaking, a policy’s content is seen as hypothesis, and implementation is a social experiment. Lindblom and Braybrooke (1963) have observed such processes of serial policy adjustment in practice. But, unlike routine practice, in critical-rational policy analysis the controlled nature of the experiment is of prime importance. This means that policy analysts are responsible for keeping objectives and conditions for implementation stable during the process. Afterwards one may compare the impact of an intervention on the properties of an experimental group to those of a similarly composed control group. Any differences found may then be attributed to the policy intervention. Repeated experiments will gradually lead to better knowledge due to error elimination through criticism of the policy experiments. Ideally, true to the ideals of an open society, not just the experimenting and evaluating policy analysts, but also those subjected to the experiment can offer their views and criticisms.

Critical-rational policy analysis has many strengths. By conceiving policy as hypothesis and implementation as experiment, it escapes from the neo-positivist illusion that delay of action may improve knowledge. The analogy between policymaking and experimenting better fits a political reality of permanent time pressure and action imperatives. In addition, the doctrine of an open and experimenting society returns to pragmatist notions of the polity as a community of problem-solvers. In principle, therefore, critical-rational policy analysis escapes the technocratic tendencies inherent to analycentric and neo-positvist approaches.

But there there are several catches to critical-rationalist policy analysis. Some of the criticism focuses on the incremental or piecemeal nature of policy experiments and the slow progress of knowledge in implementing the critical-rational program. It is argued that this does not fit a world of rapid change in which some policy experiments depend for their success on nonincremental increases in resources, and on enthusiasm rather than critique. Another type of criticism addresses the gap between the doctrine of the open, experimenting society and the practice of quasi-experimental impact evaluation. Stringent top-down implementation in different sites is a prerequisite for controlled social experiments. In practice, this justifies and leads to a ‘cozy relationship’ between reform-minded politicians, administrators, and the scientific policy evaluators, who jointly treat citizens like objects not entitled to any criticism during or after the experiment (Dryzek, 1993:220).

The most lethal criticis, however, concerns the analogy to scientific experiment underlying Popper’s and Campbell’s views. Especially Dunn (1993) has convincingly shown that the analogy runs into crippling objections if applied to social systems and policy problems. Even if reform-minded policymakers and evaluators go to great lengths in arranging the experiments in such a way that results that run counter to their expectations and preferences may occur, the social dynamics of human symbol internalization and externalization (Berger & Luckman, 1967) or structuration (Giddens, 1979) imply that

"…experimental (design and, R.H.) outcomes are unavoidably mediated by diverse standards of appraisal which are unevenly distributed among stakeholders in policy reforms. … Social theories, unlike physical ones, are difficult to falsify with experimental data because the interpretation of such data is mediated by the assumptions, frames of reference, and ideologies of social scientists and other stakeholders in reform" (Dunn, 1993: 259-260).

This poses no insurmountable problems in cases of well-structured, rather static, and nearly decomposable policy issues. But such issues decrease in frequency (Bobrow & Dryzek, 1989:148) and urgency (Hoppe, 1989) in contemporary politics. Therefore, it may be concluded, as a fallibilist and error-eliminating method, critical-rationalism is only fot for avoiding first-order errors concerning the selection of the better of two or more causal hypotheses. It is of little significance and help in avoiding second-order errors of picking the more adequate of two or more problem definitions. Althogh some critical-rationalists have embraced methodological multiplism as a remedy (Dunn, 1994: 8-10), on balance, critical-rationalism relies on "qualitative, common-sense knowing of wholes and patterns…" (Campbell, 1974: 3) when it comes to selection of problem definition and theoretical frames. Campbell has conceded that, where the results of a policy experiment frequently remain open to conflicting and ambiguous interpretation, "an experiment is of itself no more than an argument" (Campbell, 1982: 330-331). Therefore, I conclude that critical-rational policy analysis is on the verge an argumentative turn (see last section).

  1. The postpositvist turn in policy analysis

Somewhere around 1980 policy science’s original wave of success subsided. Lindblom & Cohen’s Usable Knowledge (1979) marks a period where policy scientists and analysts publicly doubt the ‘vallue-added’ for ‘ordinary knowledge’ of their ‘professional social inquiry’. From the disappointements with analycentric, neo-positivist and critical-rational policy analysis Carol Weiss draws the conclusion that the field is in intellectual crisis:

"That social scientists shape the world they study by the way they define the problem has come to be accepted not only by social scientists but by sophisticated political actors as well. They are aware that researchers’ assumptions, theories, and choice of variables can have large effects on the answer they find. This new understanding throws into doubt the accommodation (with political and administrative practice, R.H.) that earlier generations of social scientists had negotiated. If they no longer claim to find "truth" about "reality", what is their role in the policy process? The time seems to have arrived for a new set of assumptions and arrangements" (Weiss, 1991: 321).

The new assumptions - not, the new arangements - have arrived in the shape of the post-positivist turn. This means that even policy analysts - in the social sciences a rearguard in leaving the positvist and pure critical-rationalist trenches - admit interpretive, hermeneutic, and critical approaches to their stock of knowledge and methods. Within the post-positivist turn broadly perceived, four main currents may by discerned: relativistic, critical, forensic, and participatory policy analysis.

A relativistic policy analysis can be attributed to the ‘early’ Lindblom and Wildavsky. His empirically grounded insights in the disjointed incrementalist practice of policymaking (Braybrooke & Lindblom, 1963; Lindblom, 1965; 1968) have always held Lindblom back from any enchantment with the idea of the attainment in practice of a more comprehensive rationality intimated by a Lasswellian policy science. As a ‘science of muddling through’, the most policy analysis could hope for was to provide policy practice with clever strategic shortcuts and simplifications (Lindblom, 1979). But to escape from the dangers of oversimplification, one had to trust the practice of pluralist politics, its partisian mutual adjustment, and its trial-and-error learning in the successive limited comparisons of serial adjustments. Take note that Lindblom’s theory harbors strong fallibilist and pragmatist convictions. In Usable Knowledge (1979) he holds on to these vital insights. The impact of professional policy analysis is limited, and adds only modest increments to the ordinary knowledge of politicians and public officials. Policy analysts are condemned to provide argumentative ammunition for the rhetorical struggles of politicians (policy analysis as argument or data, Weiss, 1991); only occasionally they discover a nugget of enlightenment (policy analysis as idea).

Wildavsky’s views do not differ much from Lindblom’s, but they are more optimistic about the ‘art and craft of policy analysis’ (1979). After all, Wildavsky is the founding father of the University of Calfornia at Berkeley’s policy analytic curriculum. Policy analysis Wildavskian style is depicted as a dialogical and prudential balancing act in which the policy analyst helps both politicians and citizens find a practical middle ground between the ever present tensions of resources and constraints, cogitation and interaction, and dogma and scepsis. Like Lindblom in his widely acclaimed Politics and Markets (1977), Wildavsky, in the beginning of the eighties, lost his trust in political pluralism as an error-correcting safety net for biased, incremental policies ((Wildavsky, 1987: xv-xxi;1988). Concerned about increasing ideological cleavages among the American political elite and their impotence to forge a new national consensus, he turned to group-grid cultural theory to better grasp their diverging political frames (Douglas & Wildavsky, 1982).

Until 1980, Lindblom and Wildavsky have defended interpretive-hermeneutic approach to policy analysis, in the sense that they, like anthropologists among the tribes of policy experts, have inquired into the policy practitioners’ rules for problem definition, policy design, formulation and adoption, implementation, and evaluation. This method accounts for the widespread acceptance of their empirical findings. Normatively speaking, however, their approach often meant unquestioned compliance with the rules of thumb and the supposed checks and balances of pluralist political practice. This is comparable to a hermeneutic approach to shared traditions and pre-understandings without any thought of the possibility of ideological, psychopathological or any other reprehensible bias or prejudice (Torgerson, 1993; but see Lindblom, 1990; Lindblom & Woodhouse, 1993). Many have pointed out that such an uncritical interpretivist-hermeneutic approach to policy analysis can lead to a scientifically (Wittrock, 1991) or morally objectionable relativism (Dryzek, 1993).

The relativist approach has been attacked most by a critical-theoretical approach to policy analysis, advocated by Forester (1985; 1989) and Dryzek (1990; 1993; Bobrow & Dryzek, 1987). Their main accusation is that relativists disregard the conditions for consensus formation. Forester blames Wildavsky for failing to differentiate between political interaction (as a problem-solving strategy on its own right, in addition to cogitation or analysis) which does and does not elicit true learning among citizens (Forester, 1985: 265 ff). Forester deems this distinction essential in a political system where common sense and shared meaning can no longer be presupposed, and groups with clashing political frames of reference have an interest in maintaining public deception and bias. Habermas’ communicative ethics, especially his thoughts on the ideal speech situation in which people communicate free from power relations, deception and self-deception, is used as a standard for judging to what extent policymakers form a rational and genuine consensus. Policy analysts would have as their main task to monitor and foster means of authentic consensus formation.

To this end, Fox and Miller (1995:118-12) have proposed criteria for legitimate contributions to public debate: sincerity, situation-regarding intentionality, willing attention, and unique and indispensable expertise. These criteria demonstrate that the critical policy analyst does not pursue public participation for its own sake. He advocates discursive pluralism with an eye to the quality of decisionmaking and the authenticity of consensus formation. Nonetheless, Fox & Miller admit that in the virtual reality and image-struggles of the media it is difficult to judge to what extent political debate observes these four criteria. Forester has developed a typology of biased and distorted policy communication, and correponding counterstrategies for restoring trust and authenticity (Forester, 1989). The implication is that policy analysts themselves ought to see to it that their own communicative and argumentative practices are in order (Forester, 1989: 148 ff). The art of listening, respectful treatment of target groups, avoidance of unnecessary ‘officialese’ and other expert discourse, and the craft of initiating and conducting mutually enlightening debate - such are the professional skills of the critical-cum-interpretive policy analyst

Critical analysis is often criticized on two counts. Both regard the dangerous consequences of giving too much weight to the guiding ideal of the ideal speech situation. The first objection is that, however attractive from a theoretical perspective, these ideals are of limited validity in practice. Where is the borderline between deception and misunderstanding? Who is to determine what is the ‘better’ argument? To what lenghts should we go in debate and communication, where we also know that human rationality is bounded and fragile, and, sooner or later, we have to act? In other words, in all collective decisionmaking we reach dead ends, or undecidabilities, where debate, reasoning, and the force of the better argument are exhausted, and we have to shift to some form of collective will formation and legitimate power to bring the process to closure (Hoppe, 1983: 231-235; Bernstein, 1991: 221-222). All political systems are in need of procedures of managing conflicts unresolvable by debate and reasoned argument. The critical approach to policy analysis turns a blind eye to this problem. A second objection is that critical analysis often gets stuck in a form of counter-expertise disinclined to serious mutual reflection and learning. In such cases, the critical policy analyst just provides rhetorical ammunition for political fights, and just contributes to polarization, zigzag policies, and stalemate (Schön, 1983: 349-350). Torgerson (1995: 245) holds that "critique turns against both the domain of common understandings and the restricted nature of technocratic reason. … By… setting itself in judgment of common understandings, critique has an ironic potential to manifest itself as a mirror-image of technocracy." In addition, a critical policy analyst, although a partisan of ‘the people’, easily overlooks or downplays divergent opinions among ordinary citizens.

This danger is nonexistent for the forensic policy analyst (Dunn, 1981, 1993; Paris & Reynolds, 1983; Jennings, 1987; Fischer, 1980, 1995; Schön, 1983; Schön & Rein, 1994; Torgerson, 1995; Parsons, 1995: 440-444). To him it is self evident that, like in post-empiricist epistemology after Kuhn or the conditions for post-normal science specified by Funtowicz and Ravetz, policy practice is flooded by different thinking styles, diverging interpretive frames, competing policy belief systems, various ideologies, alternative professional paradigms, different world views, contrasting images of man and nature, multiple perspectives, and what have you. Such frames (Rein & Schön, 1994) are clusters of interlocking casusal and normative beliefs, whose functions are at once cognitive, communicative, and expressive of one’s identity. In order to infuse a polyvalent world with meaning, sense and purpose, and to make action and judgment possible at all, people need such frames as a sort of mental grappling hook. For instance, professional frames have been labeled the languages and cultures of ‘tribes of experts’ (Dryzek, 1993: 222) which create ‘contradictory certainties’ (Schwarz & Thompson, 1990). What people ‘see’, deem ‘relevant’, and judge ‘persuasive evidence’ on the basis of such frames, may indeed render them almost beyond comparison or translation.

The forensic policy analyst considers it his task to use the differences between frames to forge an innovative policy design from a combination of plausible and robust arguments (‘frame-reflective analysis’), or to test and bolster some frames (‘frame-critical analysis’, like in Mason & Mitroff, 1981; Paris & Reynolds, 1983; Thompson, 1997). Ideally, following rules of hermeneutic policy evaluation for arriving at shared constructions with policy stakeholders (Guba & Lincoln, 1989), and acting on the precepts of reflective practitionership (Schön, 1983), analysts marry frame-reflection and frame-criticism in an optimal mix of hermeneutic and critical moments in policy analysis. Forensic analysts do not unreflectively impose a particular professional or political frame on a problematic situation. Rather they consider the problem as unstructured to begin with. Structuring problems in a simultaneous process of reflection, action, and political strife, is the challenge of good analysis (Hisschemöller & Hoppe, 1996). Schön (1983) and Schön and Rein (1994) depict the foresic approach to analysis as an iterative itinerary among these three force-fields; a continuous process of bricolage between the policy analyst/designer, the policy design, and its wider environment, in which the policy design ought to eventually function independent of the analyst/designer. The process of analysis and design cannot be a straightforward one. Rather, the idea is to sustain creativity in one’s response to empirical uncertainties and normative ambiguities in an ever changing world. Neither goals nor means are fixed; they are transactionally constructed over and over again in intelligent deliberation and political argument, in a process of ‘naming and framing’(Schön, 1983: 40-48; 68) which may repeatedly unsettle and attack apparently dominant concepts and frames of meaning.

It is obvious that the forensic approach, especially one that successfully combines frame-analysis, frame-reflection, and frame-criticism, fully corresponds to the enlarged concept of rationality as learning. But the approach faces serious hazards. First, although some authors go to considerable lengths in describing and prescribing rules of thumb, adequate skills, and examples of best practice (Schön, 1983; Schön & Rein, 1994; Hoppe & Grin, 1995; Grin et al., 1997), the forensic approach remains relatively uncodified. This means that replication and error detection and elimination are weak. Partially, this is due to the nature of hermeneutics and critical theory, which share scepticism, and sometimes downright rejection, of codifying rules and formulating anything beyond the most general precepts of an approach to analysis. Second, the forensic approach, more than any other, is caught in a tension between the demands of good analysis and the daily practice of politics and public administration. The critical-rationalist and the relativist policy analyst uncritically adjust to common practice in the role of trusted adviser of the politico-administrative élite; and even the critical analyst easily slips into the role of a counter-expert. It is far more difficult to carve out an acceptable niche for a forensic analyst as ‘counselor’ (Jennings, 1987) or ‘participatory expert’ (Fischer, 1993). Much more thought ought to be given to the institutional aspects forensic policy analysis (cf. George, 1980). This is why, above, I argued that the new post-positivist epistemological assumptions may be considered in place, but the new institutional arrangements for developing and implementing them in practice have not yet arrived.

Finally, a fourth, participatory current in post-positivist policy analysis should be distinguished. Theoretically, this current is heterogeneous, in that participatory analysts appeal to relativist, critical, and forensic concepts and themes. What unites their paradigm, is a principled selection of a fairly elaborate range of participatory analytic techniques, in which citizens qua citizens play important roles (Mayer, 1997). Primarily those inspired by critical theory insist on the intrinsic merit of direct citizen participation in political decisionmaking. They justify participatory analysis by claiming that it vitally contributes to participatory democracy as the only rational form of life for policy scientists and true democrats (Torgerson, 1986; Dryzek, 1990). These analysts systematically favor participatory techniques in which a panel of citizens is at the heart of the analytic process, like methods for conducting consensus conferences (Klüver, 1992) or planning cells (Dienel, 1992). The policy analyst’s role is to serve and bolster citizens’ policy recommendations (Hoppe & Grin, 1995: 101-102).

Relativist, critical, and forensic analysts value participatory analysis for instrumental and contextual reasons. They specify three situations in which the use of participatory techniques is indispensable. First, when a policy problem addresses citizens’ actions up-front, and finding an acceptable solution depends on appealing to and mobilizing citizens’ knowledge of local or regional conditions. Second, when policy issues have a strong ethical component (where experts have no privileged knowledge to bring to bear on the problem), or directly pertain to citizens’ needs and wants. Third, when experts are strongly divided over an issue. Those who view participatory analysis more as an instrument than a goal per se will prefer participatory techniques which produce structured debate between citizens, politicians, officials, interest group representatives, and experts, like scenario workshops (Mayer, 1997) and propositions debates (Hoppe & Grin, 1999). Here the analyst remains in control of the analytic process; citizens’ participation, in certain situations and under particular conditions, vitally contributes to the information base, validity or representativeness of the analyst’s interpretation of public debate.and his recommendations.

The advantages of participatory analysis are obvious. In the three conditions mentioned, citizens’ input to analysis is equally important, or even more important than the experts’. Methods of participatory analysis are excellent means of harnessing citizens’ ordinary knowledge to analytic purposes. Participatory methods are hardly disputed as expansion of the tool kits of relativist , critical, and forensic policy analysis. The most important criticism is that it is far from beyond doubt whether citizen participation actually improves and enriches the quality of policy debate. Formal evaluations document that citizens rate the quality of participatory debates systematically higher than policymakers and experts (Mayer, 1997: 138-140). In the absence of objective measurement and evaluation grounded in argumentation theory, it is difficult to judge to what extent such ratings are based on self-interested prejudice by policymakers and experts. More fundamental criticism remains focused on the aspirations for participatory democracy. In spite of the impressive possibilities of interactive use of contemporary information and communications technology, the practical objections to participatory democracy are likely to stay. The results of participatory analytic exercises, even when the size of citizen panels runs to the hundreds or thousands (like in some recent applications), will never be able to claim the same representativeness as elections, referendums, or even large scale opinion surveys. In that sense, policy science and analysis still face the dilemma between serving either participatory democracy and active citizenship, or an allegedly enlightened political and policymaking élite of the administrative state. A dilemma which is as urgent as ever, now that the political means for ‘making sense together’ look very fragile in the face of the fragmentation, incommensurabilities, ruptures, and confusions between value systems and world views.

  1. The future of policy analysis: an argumentative turn?

Given the positivist beginnings and the post-postivist turn, what will the future of policy analysis look like? I would place my bets on an argumentative turn, within post-positvist constraints. An argumentative turn, the contours of which have been delineated already (Fischer & Forester,1993), would ban relativism, and simultaneously elaborate the usable elements of critical-rationalist, critical, forensic and participatory policy analysis in a new tool kit for policy advice. Such an argumentative turn coheres around three core insights: policy analysis is about crafting arguments; it cannot but deal simultaneously with the substantive and pragmatic aspects of argumentation; and it badly needs a more comprehensive set of quality warrants for policy argumentation (Hoppe & Peterse, 1998).

The first and most essential insight, lending its name to the argumentative turn, is that crafting policy advice - even in the critical-rationalist mode, as shown above - ultimately depends on argumentative practices (Forester, 1989; Fischer & Forester, 1993: 2; Hoppe, 1993: 78-79). Whatever other activities engaged in by a policy analyst, everything serves the gathering, selecting, weighing, and combining of arguments, in order to weld an argumentative chain strong enough for launching and defending a policy proposal to be put to severe tests by several audiences. Second, in policy advice as argumentation for an audience there are always two parallel stakes involved: the analytical substance of the the policy argument, measured against critical-dialectical standards, and the argument’s illocutionary and perlocutionary functions in the communication process between a sender and a receiver, measured by rhetorical-performative standards. This implies that no piece of policy analysis, like in the positivist days, can ignore its audience and its institutional context. Third, relativism can be avoided by making the quality of argument the organizing principle of policy science and analysis. Systematically applying the quality principle, and deriving specific quality criteria for judging arguments in many different types of debate settings, would enable the policy analyst to keep distinguishing between better and worse argumemts. Dunn (1993: 263-264) proposes as the aim of a critical applied social science

"to investigate concepts and procedures used to argue and settle practical claims. … (A)rgumentation is a process of rational advocacy in which stakeholders engage in competitive reconstruction of knowledge claims. This competitive reconstruction, in contrast to the competitive reconstruction of experiments (italics, rh), leads toward a pragmatic and dialectical conception of truth in which social discourse plays a reflective and critical role in producing new knowledge. Knowledge is no longer based on deductive certainty or empirical correspondence but on the relative adequacy of knowledge claims (italics, rh) embedded in ongoing social processes."

The future of policy analysis, then, would entail the development of an increasingly comprehensive set of quality warrants for valid argumentation. To fulfill this task, next to obvious excursions into argumentation theory, policy science can in principle use hermeneutic-interpretive and critical-theoretical insights, but policy analysts will have to show much more creative and pragmatic ingenuity to give them a practical bent. Furthermore, critical-rational insights, especially rules for valid causal inference, remain part and parcel of an argumentative policy analysis worthy of its name.

An argumentative turn in policy analysis methodology would substantially affect the practice of policy advice. Argumentative policy nalysis entails a looser coupling, sometimes even a decoupling, of policy analysis from its traditional context of decision support for government-initiated public policy programs. In argumentative policy analysis it is no longer government decisions, but public argument and debate that claim center stage. Public argument and debate are either an established context gratefully used, or, in cases of as yet underdeveloped public fora, a context to be created by good forensic and participatory analysis (Hoppe & Peterse, 1993; 1998). Like market inspectors who judge the fairness of market conditions and issue measures to restore them, argumentative policy analysts would sometimes claim the role of ‘inspector’ of the fairness of the marketplace for ideas (Asard & Lance Bennett, 1997), and assume democratic-pedagogical functions (Fischer & Forester, 1993: 6-7) - they would, literally, make (smaal-d) democratic (capital-D) deliberation happen.

After bringing public debate to a timely but always temporary closure, the argumentative analyst would, of course, draw conclusions for issues where a genuine consensus for further policy design and implementation has been created. But also where consensus is still lacking, and even where dissensus has sharpened, the argumentative policy analyst does not stand empty-handed. In the former case, s/he may advice governments and other stakeholders on how to jointly elaborate a strategy for partisan and serial adjustments that increases the likelihood of greater consensus at a later stage. In the latter case, s/he may detect, in the chaos of discord and confusion, those rare opportunities which may still exist for joint inquiry and continued dialogue (Roe, 1994; Van Eeten, 1999); in the hope that opportunities for consensus formation are kept open, and in the certainty that continuation of dialogue in spite of discord is rational for sustaining the delicate fabric of the body politic (Diesing, 1962).

In sum, argumentative policy analysis is, first, epistemologically grounded in a fallibilist-dialogical concept of scientific rationality, and a social-constructivist perspective on social reality; second, it is based upon a selection in context (Bobrow & Dryzek, 1987) of the most usable parts of the critical-rationalist, critical, forensic, and participatory traditions; and, third, it does not advocate a sudden and complete paradigm shift, but a patient and persistent process of revamping and testing a new tool kit for professional policy analysis. In this way, ‘speaking truth to power’ may be transformed into an argumentative policy analysis which reinvigorates political prudence as ‘making sense together’, even in this paradoxical age of politicization of science and scientization of a post-national and post-parliamentary politics.



Andersen, S.S. & T. Burns, The European Union and the erosion of parliamentary democracy: a study of post-parliamentary governance, in: S.S. Andersen & K.A. Eliassen (eds.), The European Union: how democratic is it?, Sage, London, 1996, pp. 227-251.

Asard, A. & W. Lance Bennett, Democracy and the Marketplace of Ideas. Communication and Government in Sweden and the United States, Cambridge UP, Cambridge, 1997.

Bauman, Z., Modernity and the Holocaust, Polity Press, Cambridge, 1989.

Behn, R.D. & J.W. Vaupel, Quick analysis for busy decision makers, Basic Books, New York, 1982.

Berger, P.L. & T. Luckmann, The social construction of reality, Anchor Books, New York, 1967.

Bernstein, R.J., The new constellation. The ethical-political horizons of modernity/postmodernity, MIT Press, Cambridge, Massachusetts, 1991.

Bobrow, D.B. & J.S. Dryzek, Policy analysis by design, Pittsburgh University Press, Pittsburgh, 1987.

Braybrooke, D. & Ch.E. Lindblom, A strategy of decision, Free Press, New York, 1963.

Campbell, D.T. & J.C. Stanley, Experimental and quasi-experimental designs for research, Rand McNally, Chicago, 1963.

Campbell, D.T., Qualitative knowing in action research. Kurt Lewin Award Address, presented to the Society for the Psychological Study of Social Issues, American Psychological Association, New Orleans, September 1, 1974.

Campbell, D.T., Experiments as arguments, Knowledge: Creation, Diffusion, Utilization, 3, 1982, pp. 327-337.

Cozzens, S.E., & E.J. Woodhouse, Science, Government, and the Politics of Knowledge, in: S. Jasanoff et al. (eds.), 1995, pp. 533-553.

Converse, P.E., The nature of belief systems in mass publics, in: D. Apter, (ed.), Ideology and discontent, Free Press, New York, 1964.

DeLeon, P., Advice and consent. The development of the policy sciences, Russell Sage Foundation, New York, 1989.

Dienel, P.C., Die Planungszelle. Der Bürger plant seine Umwelt. Eine Alternative zur Establishment-Demokratie, Westdeutscher Verlag, Opladen, 1992.

Diesing, P., Reason in society: five types of decision and their social conditions, University of Illinois Press, Urbana, 1962.

Douglas, M. & A. Wildavsky, Risk and culture, University of California Press, Berkeley, 1982.

Dror, Y., Design for Policy Sciences, Elsevier, New York, 1971

Dryzek, J.S., Discursive democracy: politics, policy, and political science, Cambridge University Press, Cambridge, 1990.

Dryzek, J.S., Policy analysis and planning: from science to argument, in: F. Fischer & J. Forester (eds.), 1993, pp. 213-232.

Dunn, W.N., Public policy analysis. An introduction, Prentice-Hall, Englewood Cliffs, 1981 (2nd ed., 1994).

Dunn, W.N. & B. Holzner, Knowledge in society: anatomy of an emergent field, Knowledge in Society: The International Journal of Knowledge Transfer, 1, 1, 1988, pp. 1-26.

Dunn, W.N., Policy reforms as arguments, in: F. Fischer & J. Forester (eds.), 1993, pp. 254-290.

Fischer, F. Politics, values and public policy. The problem of methodology, Westview Press, Boulder, 1980.

Fischer, F., Citizen participation and the democratization of policy epertise: from theoretical inquiry to practical cases, Policy Sciences, 26, 1993a, pp. 165-187.

Fischer, F. & J. Forester (eds.), The argumentative turn in policy analysis and planning, Duke University Press, Durham, 1993.

Fischer, F., Evaluating public policy, Nelson-Hall, Chicago, 1995.

Forester, J. (ed.), Critical theory and public life, MIT Press, Cambridge, Mass., 1985.

Forester, J., Planning in the face of power, University of California Press, Berkeley, 1989.

Fox, Ch.J. & H.T. Miller, Postmodern Public Administration. Toward discourse, Sage, Thousand Oaks, 1995.

Funtowicz, S.O., & J.R. Ravetz, Three Types of Risk Assessment and the Emergence of Post-Normal Science, in: Krimsky, S., & D. Golding (eds.), 1993, pp. 251-274.

Giddens, A., Central problems in social theory: action, structure, and contradiction in social analysis, University of California Press, Berkeley, 1979.

George, A.L, Presidential Decisionmaking in Foreign Policy. The Effective Use of Information and Advice, Westview Press, Boulder, 1980.

Graaf, H. van de & R. Hoppe, Beleid en politiek, Coutinho, Muiderberg, 1989 (1992, 2de druk).

Grin, J., & H. van de Graaf, Implementation as communicative action, Policy Science, 31(1), 1996, pp.1-29.

Grin, J., H. van de Graaf & R. Hoppe, Technology assessment through interaction. A guide, SDU, Den Haag (working document Rathenau Institute W57), 1997b.

Habermas, J., Theorie und Praxis, Suhrkamp, Frankfurt am Main, 1971.

Hisschemöller, M. & R. Hoppe, Coping with Intractable Controversies: The case of problem structuring in policy design and analysis, Knowledge and Policy: The International Journal of Knowledge Transfer, 8, 4, 1996, pp. 40-60.

Hoppe, R., Economische Zaken schrijft een nota, VU Uitgeverij, Amsterdam, 1983.

Hoppe, R., Het beleidsprobleem geproblematiseerd: over beleid ontwerpen en probleemvorming, Coutinho, Muiderberg, 1989.

Hoppe, R., Political judgment and the policy cycle: the case of ethnicity policy arguments in the Netherlands, in: F. Fischer & J. Forester (eds.), 1993, pp. 77-100.

Hoppe, R. & A. Peterse, Handling frozen fire. Political culture and risk management, Westview Press, Boulder, 1993.

Hoppe, R., & J. Grin (eds.), Special Issue: Interactive Strategies in Technology Assessment, Industrial and Environmental Crisis Quarterly, 9(1), 1995.

Hoppe, R, & A. Peterse, Bouwstenen voor Argumentatieve Beleidsanalyse, Elsevier, Den Haag, 1998.

Hoppe, R. & J. Grin, Traffic problems go through the technology assessment machine, in: Vig, N., & H. Paschen (eds.), 1999.

Jasanoff, S., The Fifth Branch: Science Advisers as Policymakers, Harvard UP, Cambridge, 1990.

Jasanoff, S., & G.E. Markle & J.C. Petersen & T. Pinch (eds.), Handbook of Science and Technology Studies, Sage, Thousand Oaks, 1995.

Jennings, B., Policy analysis: science, advocacy, or counsel?, in S.S. Nagel, (ed.), 1987, pp. 121-134.

:S. Joss. & J. Durant (eds.), Public Participation in Science. The Role of Consensus Conferences in Europe, Science Museum and EC DG XII, 1995.

Klüver, L., Consensus conferences at the Danish Board of Technology, in: S. Joss. & J. Durant (eds.), 1995, pp. 41-49.

Knorr-Cetina, K., Laboratory Studies: The Cultural Approach to the Study of Science, in: Jasanoff, S. et al. (eds.), 1995, pp. 140-166.

Krimsky, S., & D. Golding, Social Theories of Risk, Praeger, Westport, 1992.

Lasswell, H.D., Propaganda technique in the world war, New York, 1927.

Lasswell, H.D., Psychopathology and politics, New York, 1930.

Lasswell, H.D., The policy orientation, in: D. Lerner & H.D. Lasswell (eds.), The policy sciences, Stanford University Press, Stanford, 1951.

Lasswell, H.D., A pre-view of policy sciences, Elsevier, New York, 1971.

Latour, B., Wetenschap in actie, Ooievaar Pockethouse, Amsterdam, 1988.

Lindblom, Ch.E., The intelligence of democracy, Free Press, New York, 1965.

Lindblom, Ch.E., The policy-making process, Prentice Hall, Englewood Cliffs, N.J., 1968 (1st edition).

Lindblom, Ch.E., Politics and Markets, Free Press, New York, 1977.

Lindblom, Ch., Still muddling, not yet through, Public Administration Review, 29, 1979, pp. 517-526.

Lindblom, Ch.E. & D. Cohen, Usable Knowledge, Yale University Press, Yale, 1979.

Lindblom, Ch.E., Inquiry and change. the troubled attempt to understand and shape society, Yale University Press, Yale, 1990.

Lindblom, Ch.E. & E.J. Woodhouse, The policy-making process, Prentice-Hall, Englewood Cliffs, N.J., 1993 (3rd edition).

Mason, R.O. & I.I. Mitroff, Challenging strategic planning assumptions. theory, cases, and techniques, John Wiley & Sons, New York etc., 1981.

Mayer, I., Debating technologies. a methodological contribution to the design and evaluation of participatory policy analysis, Tilburg University Press, Tilburg, 1997.

Nagel, S.S., (ed.), Research in Publicm Policy Analysis and Management, vol. 4, JAI Press, Greenwich, 1987.

Nelkin, D., Science Controversies: The Dynamics of Public Disputes in the United States, in: Jasanoff, S. et al. (eds.), 1995, pp. 444-456.

Paris, D.C. & J.F. Reynolds, The logic of policy inquiry, Longman, New York, 1983.

Parsons, W., Public policy. An introduction to the theory and practice of policy analysis, Edward Elgar, Aldershot, 1995.

Popper, K., The open society and its enemies, Routledge en Kegan Paul, London, 1945.

Restivo, S., The Theory Landscape in Science Studies: Sociological Traditions, in: Jasanoff, S. et al. (eds.), 1995, pp. 95-110.

Rip, A., A cognitive approach to relevance of science, Social Science Information, 36(4), Sage, Thousand Oaks, pp. 615-640.

Roe, E., Narrative policy analysis, theory and procedure, Duke University Press, Durham, 1994.

Schön, D.A., The reflective practitioner. how professionals think in action, Basic Books, New York, 1983.

Schön, D. & M. Rein, Frame reflection: toward the resolution of intractable policy controversies, Basic Books, New York, 1994.

Schwarz, M. & M. Thompson, Divided we stand: redefining politics, technology, and social choice, Harvester & Wheatsheaf, New York, 1990.

Self, P., Econocrats and the policy process. The politics and philosophy of cost-benefit analysis, Macmillan, London, 1975.

Thompson, M., Rewriting the precepts of policy analysis, in: R.J. Ellis & M. Thompson (eds.), Culture matters. Essays in honor of Aaron Wildavsky, Westview Press, Boulder, 1997.

Torgerson, D., Between knowledge and politics: three faces of policy analysis, Policy Sciences, 1986, pp. 33-59.

Torgerson, D., Policy analysis and public life: the restoration of phronesis?, in: J. Farr, J.S. Dryzek, & S.T. Leonard, Political science in history, Research programs and political traditions, Cambridge University Press, Cambridge, 1995, pp. 225-252.

Tribe, L.H., Policy science: analysis or ideology, Philosophy and Public Affairs, 2, 1, 1972, pp. 66-110.

Van Eeten, M., Dialogues of the Deaf: New Agendas and…, Ph.D. dissertation Technical University Delft, 1999.

Van der Sluys, J.P., Anchoring Amid Uncertainty. On the Management of Uncertainties in Risk Assessment of Anthropogenic Climate Change, Mostert en Van Onderen!, Leiden, 1997

Vig, N.J., & H. Paschen, Parliaments and Technology: the Development of Technology Assessment in Europe, SUNY Press, Albany, 1999.

Wagner, P., C. Weiss, B. Wittrock & H. Wollmann (eds.), Social science and modern states: national experiences and theoretical crossroads, Cambridge University Press, Cambridge, 1991.

Weingart, P., Verwissenschaftlichung der Gesellschaft - Politisierung der Wissenschaft, Zeitschrift fur Soziologie, 12(3), 1983, pp. 225-241.

Weiss, C., Policy research: data, ideas, or arguments?, in: P. Wagner, e.a. (eds.), 1991, pp. 307-332.

Wildavsky, A., The art and craft of policy analysis, Macmillan, London, 1979 (1987, 2nd edition).

Wildavsky, A., The New Politics of the Budgetary Process, Scott, Foresman, Glenview, 1988

Wittrock, B., Social knowledge and public policy: eight models of interaction, in: P. Wagner e.a. (eds.), 1991, pp. 333-353.

Yearley, S., Science, Technology, and Social Change, Unwin Hyman, London, 1988.

Ziman, J.M., What is happening to science?, in: Cozzens, S.E., P. Healy, A. Rip, & J.M. Ziman (eds.), The research system in transition, Kluwer Academic, Dordrecht, 1990, pp. 23-33.