Comment on “Fairness Is More Than Algorithms: Racial Disparities in Time-to-Recidivism”

Cross-domain generality proof (public_policy white-space): a full-text critique of an AI-in-criminal-justice algorithmic-fairness audit, span-grounded to the gold-OA arXiv full text via the source store.

This arXiv preprint reframes recidivism from a yes/no outcome to a time-to-event (survival) process and offers a survival-based falsification test: if, within a COMPAS risk band, the timing of re-arrest differs by race, that supports rejecting the null that non-algorithmic (socioeconomic) factors do not affect recidivism. Applied to the ProPublica COMPAS data (~10,000 Broward County defendants), the headline is that medium- and high-risk strata show no racial difference, but among low-risk defendants a disparity emerges after roughly seven months, read as evidence that structural factors accumulate over time. The conceptual move is genuine, and the paper is candid in its limitations (jurisdiction, missing socioeconomic variables) and consistently hedged ('possibly', 'may accumulate'). Three cautions remain. First, the headline rests on a single low-risk cell crossing significance, drawn from a battery of log-rank tests across two races, three risk bands, two risk-score outcomes and the whole follow-up window, with the other cells null and with no multiple-comparison correction, hazard ratios, confidence bands or subgroup sample sizes reported, so it reads as weakly supported and the seven-month threshold looks data-selected. Second, on identification, the paper does engage COMPAS's documented race-differential miscalibration and states its exclusion restriction openly, so the issue is narrower than 'a hidden assumption': rejecting the null shows some non-algorithmic factor matters but cannot separate residual algorithmic miscalibration from socioeconomic context, so the specific attribution to housing/employment/support outruns the test. Third, treating non-criminal returns to custody as non-informative censoring is asserted, not defended; those returns are plausibly correlated with the same socioeconomic instability and with race, and informative differential censoring over a long window can itself manufacture the late low-risk disparity. (One span was re-anchored to a clean substring after an arXiv-HTML LaTeX artifact; the identification flaw was softened from high to moderate after a defender confirmed the paper's explicit calibration discussion; a reproducibility item was dropped as non-groundable and double-counted.)

Reframing recidivism from a binary outcome to a time-to-event (survival) process, the paper proposes a causal/survival framework and a falsification test: within a COMPAS risk band, a difference in re-arrest timing by race supports rejecting the null that non-algorithmic (socioeconomic) factors do not affect recidivism. Applied to the ProPublica COMPAS data (~10,000 Broward County defendants), the medium/high-risk strata show no racial difference while a low-risk disparity emerges after roughly seven months, read as accumulating structural disadvantage.

The significant low-risk cell is one of many: log-rank tests are run across two races, three risk strata, two risk-score outcomes, and the full follow-up window, with the medium/high cells explicitly null (p>0.1). No multiple-comparison correction, hazard ratios, confidence intervals/bands, or per-subgroup sample sizes are reported, and the 'approximately seven months' threshold reads as data-identified rather than pre-specified. Under this much uncorrected multiplicity, one significant low-risk cell is about what chance predicts, so the headline that disparities significantly emerge is weakly supported as presented.

On the merits the paper does better here than a first pass suggests: it states its exclusion restriction openly (it IS the tested null) and explicitly engages COMPAS's documented race-differential miscalibration. The narrower, surviving issue is that rejecting the null shows some non-algorithmic factor matters but cannot separate residual within-bin algorithmic miscalibration from socioeconomic context — both occupy the same unobserved node. The jump from the disjunctive null to specifically 'housing, employment, and social support' therefore outruns what the test identifies, though the paper's hedging ('possibly', 'may accumulate') keeps this to bounded interpretive over-reach.

Log-rank inference requires non-informative censoring; the paper asserts this as a formal precondition for non-criminal returns to custody but never tests it. Such technical/parole returns are plausibly SES- and race-correlated, so the censoring is plausibly informative and differential — and over a long follow-up window that alone can manufacture the late low-risk disparity, an unrefuted rival for the very signal attributed to structural factors. Rearrest as the outcome adds race-correlated measurement error from differential policing.

The conceptual contribution is genuine: recasting recidivism as time-to-event can surface divergences that single-time-point binary comparisons miss, and most algorithmic-fairness work ignores temporal dynamics. The framing is explicitly a falsification test rather than a positive identification of mechanisms, the wording is consistently hedged ('possibly', 'may accumulate', 'suggests'), and the paper reports the null medium/high cells rather than only the favourable one. The limitations section is candid — naming sampling bias, variation in law-enforcement practice, the absence of socioeconomic/community-support variables, and single-jurisdiction (Broward County) non-generalizability — and §2.1 engages ProPublica's documented miscalibration and the Chouldechova impossibility result directly.