The single topic that I have written on most extensively to date is the use of randomised control trials (RCTs) in economics to identify causal effects, generalise those findings and make policy claims. Much of this was done before this approach was awarded the 2019 Nobel Memorial Prize in Economics: I started work on RCTs in 2010 for my PhD in economics.
The purpose of this page is to collate links to all of that work in one place. I have ordered the publications based on how some interested readers might want to go through them (which is why the link to my 200+ page PhD thesis comes at the end!).
Some of the academic articles are, unfortunately, gated – I put 🔐 symbols next to those. Feel free to contact me if you’d like a copy of any of them.
The 2019 Nobel Memorial Prize in Economics was awarded to three scholars for the methodological approach that has been the focus of critique in my work. In a short paper in a special issue of the journal World Development on the 2019 Nobel, “The implications of a fundamental contradiction in advocating randomized trials for policy” (🔐), I aim to provide a succinct version of my argument against that ‘randomista’ approach.
Two articles with co-authors (Grieve Chelwa and Nimi Hoffmann) in The Conversation aimed at a more general audience also respond to the 2019 Nobel award. The first, “How randomised trials became big in development economics”, provides some background. The second, “Randomised trials in economics: what the critics have to say”, explains some of the criticisms – including my own.
In a special issue of the CODESRIA Bulletin on Randomised Control Trials and Development Research in Africa, I argue that RCTs are “A Dead-End for African Development”. In other words, I argue that the likely outcome of the emphasis on such methods will be to retard development – in sharp contrast to what proponents claim.
One example I discuss in that working paper is the use of RCTs in the context of education policy debates in South Africa. In a seminar given as a visiting fellow at the Johannesburg Institute for Advanced Study, “The new colonial missionaries: basic education policy and randomised trials in South Africa”, I explain how academics with a ‘missionary zeal’ (Bardhan) have sought, with some success, to inappropriately dominate basic education policy debates in South Africa. (I also wrote a fairly lengthy blog post on related matters in 2016, “Some thoughts on Taylor and Watson’s (2015) RCT on the impact of study guides on school-leaving results in South Africa”).
The crux of the formal (technical/econometric) argument I have made against the ‘randomista’ use of randomised trials in economics and for public policy was published well before the Nobel was awarded, in an article in the World Bank Economic Review, “Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations”.
Some of the limitations of using RCTs for policy, and insisting on them as the only truly credible basis for decision-making, have been revealed during the Covid-19 pandemic. I wrote a short paper on that for a special issue of History and Philosophy of the Life Sciences, “Masks, mechanisms and Covid-19: the limitations of randomized trials in pandemic policymaking”.
The misuse of an RCT to distort a public policy process in a manner that facilitated private sector rent-seeking – rather than the policy objective of reducing youth unemployment – was the focus of my detailed analysis of South Africa’s ‘Youth Employment Tax Incentive’ (YETI) published in Development and Change, “Evidence for a YETI? A Cautionary Tale from South Africa’s Youth Employment Tax Incentive” (🔐).
In a recent chapter in the Edward Elgar compilation A Modern Guide to Philosophy of Economics, “Randomised trials in economics“(🔐), I provide my most detailed assessment of these issues. A notable additional contribution of this chapter is that it examines the strategies advocates of these methods are using in an attempt to counter criticisms and explains why those are unconvincing and cannot succeed.
In another chapter forthcoming in the Routledge volume The Positive and the Normative in Economic Thought, “The Unacknowledged Normative Content of Randomised Control Trials in Economics and Its Dangers”(🔐), I explain how normative factors (biases, prejudices, ideologies, etc) enter a process that is typically represented as ‘objective’, ‘scientific’ and ‘neutral’. This develops a point I alluded to in earlier work.
All of this work began with the research conducted for my PhD in economics, “The external validity of treatment effects: an investigation of educational production”, which I started at the beginning of 2010 and completed in 2014 – under the supervision of Martin Wittenberg, examined by Gary Solon, Jeff Smith and Steve Koch.