Abstract

Current proposals in AI governance emphasize mechanisms such as system identification, auditing, and incident reporting. While valuable, these approaches are typically designed as discrete interventions. This creates a structural gap: the absence of a unified framework that links authorization, resource allocation, deployment, and observed outcomes. This paper argues that effective AI accountability requires not isolated traceability points, but an integrated traceability chain. Drawing on established practice in public sector transparency, particularly open contracting and procurement traceability, it outlines a model for linking governance functions across institutional boundaries, and illustrates the cost of fragmented traceability through two documented cases of automated public sector decision making failure. The paper concludes by identifying public sector AI procurement as a tractable starting point for implementing such a chain.

1. Introduction

Recent work in AI governance has proposed a range of mechanisms to improve oversight of advanced systems, including unique identifiers for AI systems, monitoring and visibility tools for deployed agents, third party auditing frameworks, and infrastructure for flaw disclosure. GovAI’s own research programme illustrates this well: its work on system identification, IDs for AI Systems, proposes methods for recognizing and verifying AI systems; its work on agent oversight, Visibility into AI Agents, proposes monitoring tools for systems that act with limited supervision; its work on Frontier AI Auditing proposes third party verification of safety and security claims; and its work on In-House Evaluation Is Not Enough argues for more robust third party flaw disclosure infrastructure for general purpose AI.

Each of these mechanisms addresses a distinct component of the governance problem. However, they are generally developed and implemented independently. As a result, they do not, by default, enable end to end accountability. A basic governance question remains difficult to answer in practice: who authorized a given AI deployment, with what resources, and what were its real world effects?

This paper argues that the difficulty of answering this question is not due to the absence of individual governance tools, but to the absence of a system that connects them.

2. The Limits of Point-Based Traceability

Existing proposals can be understood as forms of point based traceability: system identification enables recognition of a model or deployment; audits evaluate specific claims about safety or performance; monitoring tools provide visibility into system behavior; incident disclosure infrastructure records harms after they occur.

These interventions improve transparency at specific stages of a system’s lifecycle. However, they do not necessarily establish relationships between stages. A system may possess a unique identifier, pass a safety audit, be subject to monitoring, and have a disclosed incident history, while still lacking a coherent account linking the authority that approved deployment, the resources used to build and operate the system, the context in which it was deployed, and the outcomes it produced.

Two documented cases of automated public sector decision making illustrate the cost of this fragmentation.

Australia’s Robodebt scheme (2016 to 2019). The Australian government’s automated welfare debt recovery system compared annualized income data against fortnightly welfare payments and issued automated debt notices where a discrepancy was flagged. The underlying assumption, that recipients had stable and consistent employment, applied to only a small fraction of welfare recipients, which produced systematically inflated debt calculations. A subsequent Royal Commission found the scheme was sustained by a broader institutional failure: the calculation method itself was not transparent to those affected, and the burden of disproving a debt fell on individuals rather than the government having to justify it. Crucially, no single point of accountability existed connecting the ministerial decision to proceed, the department’s calculation methodology, and the outcomes experienced by the roughly 400,000 people wrongly pursued for debts. Each institutional layer could, and did, point elsewhere when the scheme was challenged.

The Dutch childcare benefits scandal and SyRI (2013 to 2021). Dutch tax authorities used a risk scoring algorithm to flag childcare benefit recipients for fraud investigation; a related system, SyRI, aggregated data across agencies to profile welfare fraud risk more broadly. The risk scoring tool used criteria that led to discriminatory targeting based on nationality, ethnicity, and income, and the resulting Childcare Benefit Scandal led to the resignation of the entire Dutch cabinet in 2021 after a parliamentary committee found that fundamental principles of the rule of law had been violated. As in the Robodebt case, the components existed in isolation: a scoring algorithm, a separate flagging registry, and tax administration decisions, without a linked chain that would have let an external reviewer trace a single family’s outcome back to the specific authorization and data inputs that produced it. It took a parliamentary inquiry, not a governance system, to reconstruct that chain after the fact.

Both cases show the same structural failure: the individual components of an accountability system existed, but no shared identifier or institutional linkage connected who authorized the system, what resources and data powered it, and what it did to specific people.

3.Traceability Chains in Public Procurement Governance

Public sector transparency systems have addressed a structurally similar problem in a different domain: linking authority, resources, and outcomes across institutional layers in public spending. The clearest working example is the Open Contracting Data Standard, developed by the Open Contracting Partnership.

The standard is built around the principle that a single contracting process, from planning and tender through award, contract, and implementation, should be traceable end to end through one unique identifier. Intermediate elements of the standard specifically allow publishers to connect procurement data to budget and project records, and to real world outcomes such as locations and delivery milestones. The standard has been adopted by dozens of governments and is endorsed by the G20, with implementations ranging from the EU’s Tender Electronic Daily to Ukraine’s ProZorro and Colombia’s SECOP.

The design features that make the Open Contracting Data Standard work are directly relevant to AI governance:

• A persistent unique identifier that follows a single process across every institutional stage, rather than each agency maintaining a separate, disconnected record.
• Publicly accessible data at intermediate stages, not only at the start (tender) or end (delivery), but at each transition in between.
• Deliberate linkage to project and outcome level data, so spending can be connected to what was actually delivered, not just what was authorized.
• Multi-stakeholder verification, with civil society and independent researchers able to check disclosed data against real world outcomes rather than relying on self reported agency claims.

This is precisely the structural feature missing from the Robodebt and Dutch childcare cases above: a chain that could be followed by an outside party from authorization to outcome, using a shared reference point.

4. Translating Traceability Chains to AI Governance