A letter rejected by the South African Medical Journal

Throughout South Africa’s Covid-19 pandemic response, I have been raising concerns about the basis for the government’s decisions – starting with an op-ed when the lockdown was announced. It has been particularly concerning how uncritical academics and journalists were at the outset. I am still in the process of writing a number of academic pieces on this, but unfortunately these will only come out later in this year or next year. One effort I made in the interim was to write a cautionary letter to the South African Medical Journal, which has been responsible for publishing some concerning editorials that contribute to the problematic stance of the South African academy. Today I received notification that my correspondence was rejected as “The editors have determined that this submission is not appropriate for this journal and will not be considered for publication.” Given that the pieces criticised were journal editorials, this is perhaps not very surprising. Decide for yourself.

Unmitigated praise of government’s Covid-19 response is premature and inconsistent with available evidence

A recent editorial[1] expresses concern with statements by the Minister and Department of Health[2,3] and Medical Research Council (MRC)[4] in response to remarks attributed to Dr Glenda Gray[5,6] relating to the government’s approach to lockdown regulations and public health consequences of the lockdown. I concur with the authors on the primacy of Constitutional principles of free speech and academic freedom for members of the Ministerial Advisory Committee (MAC) on Covid-19, and the institutional independence of the MRC (which has now found no transgression[7]). However, premature, unsubstantiated statements about government’s response to Covid-19 contribute to an environment of uncritical praise that preceded, and arguably contributed to, the controversy in question.
The theoretically optimal policy response to Covid-19 remains unknown under the usual standards of academic and scientific justification, given extensive uncertainty about characteristics of the virus itself along with the dynamics of contagion, morbidity and mortality in different populations and contexts. Furthermore, thorough assessment of the efficacy and optimality of government responses can only be made on evidence that will become available after the pandemic is over. Only preliminary assessments are possible at present and cannot be exempt from basic standards of justification and evidence. The authors’ statements[1], as with others elsewhere[8], do not meet such standards and thereby undermine impartial, evidence-based criticism. For example:

Notwithstanding the concerns raised above, the Minister of Health’s management of the country’s COVID-19 pandemic, to date, is laudable

And:

SA’s response to COVID-19 has been swift and science based, and merits praise

The apparent premise, that “other governments around the world have not grounded their response to the pandemic in science and evidence”[1], is too low a bar. The British government, for example, has rightly been criticised not for failing to use science and evidence, but for doing so selectively and secretively[9]. Yet the authors fail to critically examine the composition and conduct of the MAC, asking only for “involvement of experts from academia outside of the biomedical sciences, and statutory bodies”[1].
There is evidence that contradicts this stance. As noted by others[10], and reflected in international open Covid-19 databases[11], South Africa has not been forthcoming in publishing detailed data on testing, screening, contact tracing and patient characteristics – despite international calls for transparency[9,12]. Even more concerning, while the original lockdown decision was premised on modelling[13], only limited details of the strategy and basis were provided much later[14] while current projections and model details have only recently been made public[15].

Good intent along with science- and evidence-based decision-making are not sufficient to ensure the best policy decisions are taken. Transparency in evidence, modelling, decision-making, use of expertise and balancing of societal priorities is paramount. The South African government has performed badly on some of these dimensions. Rhetoric of “unity and solidarity” in that context potentially undermines the role of dissent, rather than deference, in contributing to the public good. Unsubstantiated and premature praise may contribute to a sub-optimal response to the pandemic.

1. Singh JA. Freedom of speech and public interest, not allegiance, should underpin science advisement to government. S Afr Med J. 2020 May 26;
2. Mkhize Z. Health Minister’s statement on Prof Glenda Gray’s public attack of government based on inaccurate information [Internet]. National Department of Health; [accessed 26 May 2020]. Available from: http://www.health.gov.za/index.php/2014-03-17-09-48-36/2014-03-17-09-49-50?download=4247:statement-by-minister-mkhize-prof-glenda-gray-public-attack-of-government-20-may-2020
3. Human L, Geffen N. Health department boss calls for investigation into Glenda Gray. GroundUp [Internet]. 22 May 2020 [accessed 26 May 2020]; Available from: https://www.groundup.org.za/article/health-department-boss-calls-investigation-glenda-gray/
4. Herman P. SAMRC board apologises for Prof Gray’s comments, bars staff from speaking to media. News24 [Internet]. 25 May 2020; Available from: https://www.news24.com/SouthAfrica/News/breaking-samrc-board-apologises-for-glenda-grays-comments-bars-staff-from-speaking-to-media-20200525
5. Karrim A, Evans S. Unscientific and nonsensical: Top scientist slams government’s lockdown strategy. News24 [Internet]. 16 May 2020 [accessed 16 May 2020]; Available from: https://www.news24.com/SouthAfrica/News/unscientific-and-nonsensical-top-scientific-adviser-slams-governments-lockdown-strategy-20200516
6. Karrim A. I didn’t criticise the lockdown, but the regulations ‒ Prof Glenda Gray after Mkhize slams criticism. News24. 21 May 2020
7. SAMRC. Media statement from the SAMRC Board [Internet]. 26 May 2020 [accessed 26 May 2020]. Available from: https://www.samrc.ac.za/media-release/media-statement-samrc-board
8. Academy of Science of South Africa (ASSAf). Public Statement on COVID-19. 18 May 2020.
9. Alwan NA, Bhopal R, Burgess RA, Colburn T, Cuevas LE, Smith GD, et al. Evidence informing the UK’s COVID-19 public health response must be transparent. Lancet. 2020 Mar;395(10229):1036–7.
10. Marivate V, Combrink HM. Use of Available Data To Inform The COVID-19 Outbreak in South Africa: A Case Study. Data Science Journal. 6 May 2020; 19(1):19.
11. Xu B, Kraemer MUG, Xu B, Gutierrez B, Mekaru S, Sewalk K, et al. Open access epidemiological data from the COVID-19 outbreak. The Lancet Infectious Diseases. 2020; 20(5):534.
12. Barton CM, Alberti M, Ames D, Atkinson J-A, Bales J, Burke E, et al. Call for transparency of COVID-19 models. Sills J, editor. Science. 2020; 368(6490):482.2-483.
13. Republic of South Africa. President Cyril Ramaphosa: Escalation of measures to combat Coronavirus COVID-19 pandemic [Internet]. 23 March 2020 [accessed 25 May 2020]. Available from: https://www.gov.za/speeches/president-cyril-ramaphosa-escalation-measures-combat-coronavirus-covid-19-pandemic-23-mar
14. Abdool Karim SS. SA’s Covid-19 epidemic: Trends & Next steps. Presentation for the Minister of Health; 13 April 2020. [accessed 16 May 2020].
15. Silal S, Pulliam J, Meyer-Rath G, Nichols B, Jamieson L, Moultrie H. Estimating cases for COVID-19 in South Africa Update: 19 May 2020. South African COVID-19 Modelling Consortium; 19 May 2020.

Economics: scientists and plumbers, or bullshit and mathiness?

On the 6th of January 2017 the Annual American Economic Association conference is scheduled to host a plenary address entitled The Economist as Plumber: Large Scale Experiments to Inform the Details of Policy Making. The speaker is the academic economist Esther Duflo, widely-acclaimed for popularising the use of randomised control trials (RCTs).

Given my PhD work in economics on external validity of RCTs and implications for policy, and parallel work in philosophy, I have a few thoughts on this subject. In a draft paper (first presented in 2015) entitled When is Economics Bullshit? I argue that practitioners promoting RCTs have systematically overstated the policy-relevance of results and thereby produced ‘bullshit’ (as defined in the famous essay by philosopher Harry Frankfurt).

A consistent problem in critiquing so-called ‘randomistas’ is that the goalposts have been constantly shifted. Early advocacy for RCTs within economics reflected a ‘missionary zeal’ (Bardhan). It has been suggested that experimental methods have led to a ‘credibility revolution‘: giving credibility to applied microeconomic work that apparently did not exist before. One recipient of the Bates Clarke medal argued that the introduction of RCTs indisputably rendered economics ‘a science’. In the policy domain I, along with other economists, have come across much grander and/or more extreme claims. But when challenged, proselytisers scale back the claims and deny ever overclaiming. So from missionary zeal, revolution and science we now have plumbing….

I look forward to reading Duflo’s speech/paper, but my own view of the methodology and philosophy of economics and RCTs suggests that plumbing is a very poor analogy.

In my own paper, motivated in part by claims that RCTs render economics ‘a science’, I tackle the question of scientific status head on. Using a revival of the so-called demarcation question (basically: how do we demarcate science from non-science or pseudoscience?) in philosophy, I argue that economics cannot (yet) be classified as a science, may never be classifiable as such and in the way it is used by some economists too-often verges on pseudoscience and/or bullshit.

The similarities between this very critical view and that of Romer’s recent critique of macroeconomics (which was made public later) are interesting. Romer focuses more on the use of mathematical modelling whereas my focus is on empirical methods. I will write a detailed comment on Romer’s piece later this year; I agree with some aspects but strongly disagree with others.

In its two presentations so far, my paper on bullshit has been relatively well-received by philosophers of science but not so well-received by philosophers of economics. There is good reason for this: the paper is even more an indictment of the current trend in philosophy of economics than it is of economics itself. The paper notes that in the absence of sufficient technical training and understanding of economics, philosophers in this area have increasingly taken the safer route of becoming apologists for the discipline. In effect, they compete to provide explanations of why economists are correct in their approach. (Exceptions to this, such as Nancy Cartwright – who has collaborated with Angus Deaton in providing important and influential critiques of RCTs – arguably prove the rule: Cartwright’s reputation was already established in philosophy of physics, causality and metaphysics).

The result, unfortunately, is that philosophy of economics currently has very little to add to economists’ critical understanding of their own discipline. Some critics, such as Skidelsky, argue that economists should read more philosophy, but while I am sympathetic to his overall stance I do not think economists would find much worth reading at present. Combining the abject failure of the ‘mainstream’ of philosophy of economics with the low quality of most economists’ reflections on methodological issues leaves us with few critical insights that could move the discipline beyond parochial or self-interested debates.