A public authority should not be able to make a consequential decision and then explain that the reasoning belongs to its contractor.
The authority chose to rely on the system. It remains responsible for what it does with the result. Contracting out a technical task should not contract out the public’s ability to ask why a decision was made.
That principle is straightforward. Applying it requires attention to the less visible parts of a purchase: who retains the records, who can inspect an output, what survives a software update, and what happens when the supplier leaves.
The issue is not whether every public employee should understand every line of code. It is whether the institution can give an intelligible, contestable account of its own exercise of power.
A decision has more than one author
Consider a hypothetical agency using a commercial system to identify applications for additional scrutiny. The supplier may design the tool, the agency may choose the threshold, and a staff member may make the final decision. Responsibility can be distributed across that chain without disappearing from it.
An explanation should identify which parts of the result came from the tool and which came from human judgment. “A person reviewed it” is not enough when no one can say what the person reviewed or what authority they had to disagree.
Nor is the name of a product an explanation. A person affected by the decision needs to understand the relevant basis, not merely learn which company sold the machinery.
This is a proposal for accountable administration. It is not an assertion that every existing system lacks those safeguards, or that every adverse result produced with commercial assistance is wrong.
Buy the ability to answer
The necessary arrangements should be established before the contract is signed. A public body should know whether it can retain the information needed to understand a result, obtain an explanation from the supplier, and arrange appropriate independent examination when a serious question arises.
Those arrangements cost money. That cost belongs in the decision to purchase the system. An offer that appears inexpensive because it omits future scrutiny may simply move the expense onto the person forced to challenge an error.
The contract should also address continuity. Can the authority still explain a decision after the product changes? Can it obtain usable records if the supplier is replaced? Who is responsible when the company no longer supports the version that produced the disputed result?
A public agency should not discover the answer to those questions only after someone’s livelihood, liberty or access to an essential service depends on it.
Commercial confidentiality is not the end of the discussion
There are legitimate reasons not to publish every technical detail. Security, personal privacy and genuinely proprietary material can require protection. But the existence of those interests does not eliminate the responsibility to design a workable form of scrutiny.
Controlled examination, limited disclosure and an explanation focused on the relevant decision may sometimes reconcile competing interests. The arrangement must be adequate to the question, not merely convenient for the supplier.
Where a product cannot support meaningful accountability at an acceptable cost, the public body should consider whether it is suitable for the proposed use. A system that is acceptable for a low-stakes administrative task may be inappropriate where an unexplained mistake carries a much greater consequence.
This is a political choice about the conditions of public power. It should not be hidden inside procurement language that few affected people will ever see.
Do not sell the correction back to the public
A person challenging a decision should not have to become the contractor’s customer. The public authority should receive the complaint, identify its responsibilities and explain the available route to review.
If technical work is needed to answer a properly supported question, the authority should not automatically shift the entire burden onto the claimant. A process that requires an expensive private expert at the first step risks making a remedy available only to people already able to purchase influence.
That does not mean funding every speculative objection. It means adopting a transparent threshold for obtaining a limited explanation or examination, with reasons when a request is refused. A weak claim can be rejected without making the process itself inscrutable.
The person should also be told what was corrected and what was not. A revised score, a changed record and a reversed decision are different outcomes. Each should be described accurately.
Keep the institution responsible
Dissident Voice’s discussion of AI and social power raises the larger question of who benefits from technological change. Public contracting adds another question: who must answer when the machinery produces a contested result?
A supplier may have contractual responsibilities. A professional may have duties associated with a particular task. The public authority still has to account for its own decision. Dividing work should make the chain of responsibility clearer, not create a series of offices that all point somewhere else.
Workers inside the institution also need an honest route to raise concerns. A person who notices that a tool is unsuitable should not be left choosing between silent compliance and an unauthorised disclosure of private information. The system should provide a protected, accountable way to question its use.
A government cannot purchase legitimacy with a software licence. It must earn confidence through decisions people can understand and, where appropriate, contest. The contractor may own a product. It should not own the only explanation for the public power exercised through it.
*****
Author’s note: Prepared with substantial assistance from ChatGPT (OpenAI) in research, drafting and editing. No interviews or original empirical research are claimed.