JT/DL: Does Justice Tech Work? 🤷

Key Takeaways

• No rigorous evidence shows most justice tech improves outcomes
• Success metrics focus on throughput, not people‑centered results
• Agency capacity, funding, and interoperability are critical but overlooked
• Vendor lock‑in limits competition and drives costly, subpar tools
• Proposed agenda calls for audits, post‑mortems, and cross‑disciplinary research

Pulse Analysis

Justice technology has exploded over the past two decades, with governments and nonprofits investing millions in AI chatbots, e‑filing platforms, and data dashboards. Yet the sector remains dominated by descriptive surveys and pilot reports that catalog tools without assessing real‑world impact. This evidence gap means policymakers cannot distinguish between pilots that merely speed up case processing and those that actually resolve disputes, leaving billions of dollars funneled into unproven experiments.

A deeper problem lies in how success is measured. Most evaluations rely on superficial indicators—website traffic, user counts, or reduced processing times—while ignoring whether individuals achieve fair, timely outcomes. Compounding this, many justice agencies lack the technical staff, interoperable systems, and secure infrastructure needed to deploy and maintain new tools. Vendor capture further entrenches inefficiencies: proprietary case‑management systems create lock‑in, inflate costs, and block innovative entrants, while agencies have no benchmarks to assess value or accuracy claims.

The paper’s proposed research agenda offers a roadmap to remedy these flaws. Independent audits of AI assistants, systematic post‑mortems of failed deployments, and rigorous gap analyses of agency capacity can generate the hard data needed for evidence‑based policy. By bringing economists, antitrust scholars, and cybersecurity experts into the conversation, the sector can develop people‑centered metrics, enforce interoperability standards, and create competitive procurement frameworks. Implementing this agenda could transform justice tech from a series of well‑intentioned pilots into a proven engine for equitable legal outcomes.