From PolicyWTF to PolicyFTW
By Shyam Kumar
We were explorers in an unknown land with monsters lurking in every covert. It had all the exhilaration and the danger of making policy where there was none.”
Neal Blewett, Australian Minister for Health, on tackling the AIDS crisis (1996).
I joined the Takshashila Public Policy course four months ago. I wanted to observe how people in power made decisions, learn the traditions of policy making, and discover what I cared about.
As I (somewhat naively) cut through the jargon in my quest to make sense of the policy making world, I find myself occasionally thrown to moods of angst and resignation, especially when the literature dwells on what’s wrong with our governments and society.
In the latest edition of his excellent public policy newsletter, Anticipating the Unintended, Pranay Kotasthane points to a new framework for evaluating policy success, published in the book “Successful Public Policy: Lessons from Australia and New Zealand”
The opening lines of the book:
This book swims against the currents of scepticism, negativism and disillusionment with politics and government that beset many nations today.
It looks purposefully at the ‘bright side’ of public policy, and provides detailed case studies of instances in which governments by and large got it right and made positive differences to the lives and wellbeing of countless citizens, to the strength of their economies and, sometimes, to their country’s standing in the world.
I was ready for this invitation.
The meaning of success.
The authors make the early move to emphasise that ‘success’ is subjective. They acknowledge the inherent interpretative nature of evaluations, and how contingencies and the vagaries of society’s emotions and concerns shape policy outcomes.
I find it refreshing that this acknowledgement is not apologetic (as seems the norm for academic papers) - the subjectivity doesn’t make the lessons less valuable, and it makes the stories more relevant and real.
The authors give narratives about power by considering political intentions, and give power to narratives by evaluating policies from diverse perspectives.
This policy analysis is not an academic exercise which abstracts away the human experience for the sake of theoretical scaffolding. The authors build stories about the stories, while being grounded in their academic traditions and comprehensive in their analysis.
The framework for evaluation.
The following assessments are used as the framework for evaluation:
Programmatic assessment: This is ‘classic’ evaluation research focus on a policy’s goals, the theory of change underpinning it and the selection of the policy instruments it deploys—all culminating in judgements about the degree to which a policy achieves valuable impacts.
Process assessment: The focus here is on how the processes of policy design, decision-making and delivery are organised and managed, and whether these processes contribute to not only the policy’s technical problem-solving capacity (effectiveness and efficiency), but also its social appropriateness and in particular the sense of procedural justice among key stakeholders and the wider public.
Political assessment: This dimension assesses the degree to which the policymakers and agencies involved in driving and delivering the policy are able to build and maintain fungible political coalitions supporting it, and the degree to which their association with it enhances their reputations.
Temporal Assessment: The success or otherwise of a public policy, program or project should be studied not as a snapshot but as a film. A policy’s success is therefore also to be assessed in terms of how performance and legitimacy develop over time as a policy advances from proposal, design and delivery to impact.
While the authors do not claim this explicitly, I find that the categories are used as storytelling devices. The categories themselves are not the story. In particular, using the lenses of time and politics seems to give them the freedom and space to imagine what it was really like for the various actors in each case study.
For example, the first case study - how Australia dealt with the AIDS crisis - describes in great detail the historical context, day to day realities, and shifting concerns of the government, gay community, local councils and the conservative society across 30 years.
The nature of assessments.
Assessments are not objective; they cannot be. We bring our own histories to the exercise, we are already in moods when we make assessments, and we make them for the sake of specific concerns in the future. The assessments we make reveal what we care about.
(When I make an assessment about this book, I make it as a 36yr old Indian frustrated with the state of the country, nervous as a beginner to the domain of public policy, and concerned about how I engage with the work of policy making.)
Policy thinkers who read these case studies may naturally use the four categories of assessments in the framework - programmatic, process, political and temporal - to analyse the policies they are working on. I worry they would miss an opportunity if the reading is limited to that diagnostic lens.
These case studies give us access to the worlds of the various actors. We can step in to their shoes with some mental gymnastics, and try to experience and assess the policy making as they would. We should not hope to find the ‘objectively true’ assessments. How could we?
I cannot definitively imagine what it was like for the disenfranchised gay person facing a disease which seemed to target their community. I cannot feel the confusion of the drug user who was given sterilised needles supporting their addiction through a local needle exchange program. I cannot fully know the political concerns of the new health minister who was asking for public money to minimise harm to criminals.
Yet, it is in the futility of this imagination that I can see what was at stake. It gives meaning to the traditions of policy making. It shifts my mood from the indifference and detachment of rational thinking, to one of wonder and intense engagement with the drama of human life. For that, I am grateful.
The PolicyWTF stories which highlight the gaffes and shortcomings of public policy can be indulgent and useful to learn from, but I also hope to discover and tell more PolicyFTW stories to build trust, mystery and purpose.