Category: philosophy

A note on the philosophy literature on external validity

Later this month (August 2019) I’ll be presenting a paper at the 14th conference of the International Network for Economic Method (INEM). The paper is titled, “From ‘data mining’ to ‘machine learning’: the role of randomised trials and the credibility revolution”. An apparent puzzle is that there’s a session on external validity – which was the subject of my economics PhD, a working paper and short publication – in which I’m not presenting. Surely if I am going to be presenting at conferences on the method or philosophy of economics I should be presenting my work on external validity? The short answer is: I already did in 2012. But I think the longer explanation is also worth giving.

First, the paper I will be presenting at INEM (and ENPOSS) builds explicitly on work I’ve done on external validity (henceforth ‘EV’).

Second, and more importantly, my contribution to the philosophy literature on EV was not just presented at the Evidence and Causality in the Sciences (ECitS) 2012 conference but subsequently finalised as a paper in 2012, revised in 2013. Unfortunately that paper was not published at the time, for reasons that were at best flimsy. Preoccupied with finishing my economics PhD and changing jobs, I delayed resubmitting the manuscript. When I returned to academia in 2016 I discovered that a paper on the subject had been published in Philosophy of Science. More surprising was that, apart from some differences in verbiage and references, the core arguments of the paper seem to be the same as about 30-40% of my own paper but with no reference to that or my work in economics. Then, earlier this year, another paper appeared in the Journal of Economic Methodology. The core arguments of this paper, too, are very similar to the other 30-40% of my paper (dealing with issues like causal process tracing and related matters). In the second instance, my economics work is cited by misunderstood or misrepresented: suggesting that my views are different to the author’s when in fact, as is clear from the 2012/13 paper, they are almost entirely the same.

Needless to say, this creates a rather awkward situation. Not least because I believe, for reasons I will not ventilate in detail at this point, that it is implausible that the two authors were unaware of, or uninfluenced by, my 2012/13 work. But it is now simply impossible to publish my own work, despite clearly having a claim to intellectual priority. These concerns have been taken-up in the relevant fora, but the wheels turn slowly. And it will be informative to test the extent to which academic philosophy is committed to principles of intellectual priority. In the interim it makes for an ‘interesting’ context for intellectual engagement…

Economics: scientists and plumbers, or bullshit and mathiness?

On the 6th of January 2017 the Annual American Economic Association conference is scheduled to host a plenary address entitled The Economist as Plumber: Large Scale Experiments to Inform the Details of Policy Making. The speaker is the academic economist Esther Duflo, widely-acclaimed for popularising the use of randomised control trials (RCTs).

Given my PhD work in economics on external validity of RCTs and implications for policy, and parallel work in philosophy, I have a few thoughts on this subject. In a draft paper (first presented in 2015) entitled When is Economics Bullshit? I argue that practitioners promoting RCTs have systematically overstated the policy-relevance of results and thereby produced ‘bullshit’ (as defined in the famous essay by philosopher Harry Frankfurt).

A consistent problem in critiquing so-called ‘randomistas’ is that the goalposts have been constantly shifted. Early advocacy for RCTs within economics reflected a ‘missionary zeal’ (Bardhan). It has been suggested that experimental methods have led to a ‘credibility revolution‘: giving credibility to applied microeconomic work that apparently did not exist before. One recipient of the Bates Clarke medal argued that the introduction of RCTs indisputably rendered economics ‘a science’. In the policy domain I, along with other economists, have come across much grander and/or more extreme claims. But when challenged, proselytisers scale back the claims and deny ever overclaiming. So from missionary zeal, revolution and science we now have plumbing….

I look forward to reading Duflo’s speech/paper, but my own view of the methodology and philosophy of economics and RCTs suggests that plumbing is a very poor analogy.

In my own paper, motivated in part by claims that RCTs render economics ‘a science’, I tackle the question of scientific status head on. Using a revival of the so-called demarcation question (basically: how do we demarcate science from non-science or pseudoscience?) in philosophy, I argue that economics cannot (yet) be classified as a science, may never be classifiable as such and in the way it is used by some economists too-often verges on pseudoscience and/or bullshit.

The similarities between this very critical view and that of Romer’s recent critique of macroeconomics (which was made public later) are interesting. Romer focuses more on the use of mathematical modelling whereas my focus is on empirical methods. I will write a detailed comment on Romer’s piece later this year; I agree with some aspects but strongly disagree with others.

In its two presentations so far, my paper on bullshit has been relatively well-received by philosophers of science but not so well-received by philosophers of economics. There is good reason for this: the paper is even more an indictment of the current trend in philosophy of economics than it is of economics itself. The paper notes that in the absence of sufficient technical training and understanding of economics, philosophers in this area have increasingly taken the safer route of becoming apologists for the discipline. In effect, they compete to provide explanations of why economists are correct in their approach. (Exceptions to this, such as Nancy Cartwright – who has collaborated with Angus Deaton in providing important and influential critiques of RCTs – arguably prove the rule: Cartwright’s reputation was already established in philosophy of physics, causality and metaphysics).

The result, unfortunately, is that philosophy of economics currently has very little to add to economists’ critical understanding of their own discipline. Some critics, such as Skidelsky, argue that economists should read more philosophy, but while I am sympathetic to his overall stance I do not think economists would find much worth reading at present. Combining the abject failure of the ‘mainstream’ of philosophy of economics with the low quality of most economists’ reflections on methodological issues leaves us with few critical insights that could move the discipline beyond parochial or self-interested debates.