Governments are buying AI faster than they are regulating it. The purchase is where public purpose gets written in or left out.
Last week ended at the UN Working Group on Data Governance, and the Commission on Science & Technology for Development (CSTD), where ‘multi-stakeholders’ debated data governance language line by line. This week on Monday, the World Intellectual Property Organization (WIPO) opened its first Global Forum on Intellectual Property and Artificial Intelligence, two days (with a third on AI Infrastructure Exchange) on who owns what AI makes. Next week, to Paris for the Council of Europe’s HUDERIA Local Academy, where officials from twenty local and regional authorities will test a human rights method on AI their own administrations are using.
The Geneva rooms argue over rules and ownership. The Paris room will be about purchase orders, which is where most AI enters public life.
A city does not adopt a model. It signs a contract, accepts a software update, or switches on a feature bundled into the case management platform it bought five years ago. Each of those moments decides what a system will do to the people who depend on a public service, and almost none of them is governed by anyone thinking about rights. That is why procurement is the most underused lever in AI governance. It is where the question of what we are regulating gets answered in practice, whether or not anyone asked it.
Consider a public employment service buying a tool to match jobseekers to vacancies. The system reads a long, unbroken record of paid work as evidence of suitability. A woman returning after years of unpaid care receives recommendations below her qualifications. Nobody wrote a rule against her. The model learned from records that never counted what she did, and the authority bought it without asking. Repeated across a caseload, one assumption becomes the shape of the service.
This is the harm our risk frameworks are worst at seeing. They are built to catch discrete failures, the wrong decision in a specifiable case. They miss the slow accumulation of slightly worse options offered to people whose lives were never well documented, which describes most people on earth. A complaints process will not surface it either, because she never learns which jobs she was not shown.
Procurement can. Before a tender exists, an authority decides what the service is meant to achieve, and it can make that decision with frontline staff and with the applicants who gave up halfway through the form. Once the purpose is clear, it becomes a requirement. Suppliers must show how their system treats a career break, using cases built to expose the failure rather than demonstrations run on tidy records. The contract then carries those claims into use. It secures the records needed to investigate a disputed result and keeps the authority’s right to suspend the tool while the service keeps running.
None of this is exotic. EIGE and the European Investment Bank have published guidance on gender-responsive public procurement. The Council of Europe’s Framework Convention requires attention to risk and remedy across the lifecycle, and HUDERIA offers a method for doing it. The EU has model contractual clauses for AI. What has been missing is the connection between these instruments and the moment a public body decides what to buy.
Our new policy paper, From Principles to Procurement, sets out to make that connection. Published with the support of the Friedrich Naumann Foundation for Freedom’s Human Rights Hub, it follows a purchase from the first conversation about public need through to the years of use that follow the contract. It builds on the procurement research developed by Emma Kallina, Professor Jatinder Singh and colleagues at Cambridge, (receiving an Honourable Mention at ACM’s CHI Conference on Human Factors in Computing Systems this year). The central test should be uncontroversial: a public AI purchase succeeds when the service gets better for the people who need it. Faster processing counts only when it serves that end.
The gender lens does particular work here. Women are half the population, yet the records feeding public systems still describe a working life organised around continuous paid employment. A system that cannot read a career break or irregular income will also fail the migrant whose qualifications were never recognised and the disabled applicant who needs another route to correct her file. Testing for gender becomes a test of whether a supposedly general-purpose system serves the public it claims to serve.
There is a market argument too. Public bodies are among the largest buyers of technology in any economy. When they require suppliers to demonstrate that a system recognises overlooked capability and supports real correction, they create demand for products that do. Suppliers build what they are paid to deliver. Governments have been paying for speed.
This is why the local level matters. The municipalities coming to Paris are not waiting for a global settlement on AI. They are already buying and switching things on. Most will never have in-house teams able to interrogate a model. They need shared expertise and common requirements they can adapt, along with the backing to stop a pilot when the evidence says stop. That is a question of political will more than technology.
Alongside the paper, the procurement section of our AI & Equality Workspace turns these questions into working steps for teams with a decision ahead.
At WIPO this week the question is who owns what AI produces. The harder question for public institutions is why and what they are paying AI to produce, and for whose benefit. That answer is written into the tender long before any regulator arrives.