Transparency and Simplicity in Criminal Risk Assessment

In “The Age of Secrecy and Unfairness in Recidivism Prediction,” Rudin, Wang, and Coker (2020, hereafter, RWC) contend that the current focus on questions of algorithmic fairness is misplaced. Rather, they argue, we must insist first and foremost that the algorithms used in high-stakes settings such as criminal justice are transparent, prioritizing transparency over other forms of fairness. The authors make their case by taking us on a deep dive into what can be learned about a proprietary risk assessment tool, COMPAS. Through their methodical statistical detective work and reflections on stories such as that of Mr. Rodrïguez, who was denied parole on the basis of an erroneously calculated COMPAS score, they make clear that without transparency “algorithms may not do what we think they do,[…] they may not do what we want,” and we may be left limited in our recourse against it.

I agree that algorithmic transparency is in many ways of fundamental importance. However, to ensure that algorithms ‘do’ what we want them to, we need more than the transparency afforded by having a simple model whose formula is publicly accessible. A key issue here is that algorithms do not do very much. In settings such as criminal justice, secret (or not-so-secret) algorithms commonly influence and inform decisions about individuals, but they do not make them. Recidivism risk, such as may be accurately estimated through the simple models lauded by the authors, is only one part of the decision-making equation. As I will discuss further in this response, a big part of the challenge in assessing an algorithm’s fit-for-purpose is that there is no decision-making equation …