Source: EN – Geneva Solutions — Read the original article.
States meeting in Geneva next month will decide whether to begin formal negotiations to regulate lethal autonomous weapons. But without agreement on what meaningful human control requires in practice, a future treaty risks enshrining a principle open to wide interpretation, writes Gerald Mako, a research affiliate at Cambridge University.
What happens when a weapon can choose its own target and engage it? Until a few years ago, the idea of a machine making these decisions largely belonged to dystopian science fiction. In September, however, this once-futuristic question entered a new phase when Russian and US delegations in Geneva struck out a number of clauses from a text being formulated by the United Nations Group of Governmental Experts on Lethal Autonomous Weapons Systems (LAWS) under the Convention on Certain Conventional Weapons (CCW).
While the name may be a mouthful, the subject is much simpler. These weapons, like autonomous drones, can select and engage with targets with limited or no human input. The new AI age has laid the groundwork for new AI weapons.
Read also: Killer robots close to breaching moral red line, warn UN and ICRC amid renewed push for treaty
Among the provisions removed were a requirement that humans review AI-generated targets, as well as language on predictability, reliability, explainability, traceability and ethical considerations. The Group agreed on a set of elements, without prejudging its nature, preserving the phrase governments have repeated almost like a mantra for years: human judgement and control. However, much of the detail that could give those words practical force is gone.
This is Geneva’s odd harvest this autumn. The non-binding outcome produced by the Group may not sound like much, but it is the furthest the process has reached towards an agreement on LAWS. The principle remains, though; what “human control” requires in practice is less clear.
Why human judgment needs a definition
The US arguing for more flexible international language is far from being a rejection of safeguards. Washington has long preferred “appropriate levels of human judgment” to continuous human control, which it has deemed far too rigid. The Pentagon’s 2023 directive on autonomy already requires judgment, reliable performance, realistic testing, and compliance with the law of war.
This concern is perfectly intelligible, and other governments share it, including 11 European countries that argued in a 2025 working paper that the appropriate level of human judgment and control depends heavily on context. As such, reports suggesting that this is a uniquely US and Russian approach are plainly wrong. But understanding the case for flexibility does not make the difficulty presented by the non-binding text disappear. Ultimately, a consensus that keeps the principle without specifying its practical requirements risks leaving every capital to define the principle on its own terms.
A human in the chain is not the same as control
Human control is increasingly being treated like clay. Everyone agrees on the label but not on what form it should take. Indeed, every country can shape it into a form that suits its own weapon development, deployment plans, and doctrine. Fire up! The resulting formulations can look very much the same from a distance but differ substantially in what they can contain. One may require a commander to understand the system, intervene when necessary and halt an engagement, while another may regard a potentially routine decision made before deployment as sufficient. Yet both can be labelled “human control”.
Read also: ❝When algorithms dictate war and the promise of efficiency
The difference comes down to three tests. Did a human review the target set and understand the system’s limits, or did they merely accept a ranked list? Did an operator retain a genuine ability to interrupt or withhold an engagement after the system was activated, or did they only make a decision in a planning cell weeks earlier? Would the system leave a record complete enough to reconstruct responsibility when something failed? Without answers to these questions, control requires an organigram.
There is also a deeper question: when does human judgment have to happen? Before a mission? Before an individual engagement? Or only when circumstances change? The text does not provide an answer. A human can remain responsible on paper while exercising no meaningful judgment when force is actually used.
One phrase, different meanings
The power of ambiguity is enormous. Countries with vastly different approaches can merrily agree on the same broad language because it leaves the meaning of human judgment open. A force that preserves genuine intervention can endorse the same formula as one that places the decisive judgment much earlier. The final language is broad enough to accommodate countries with different views of what human judgment requires while leaving some of the hardest questions unanswered.
What happens in November
The cliffhanger will not last long. The issue returns to Geneva soon, from 16 to 20 November, when the CCW’s Seventh Review Conference will meet to decide the way forward, including what should happen with the expert group’s agreed set of elements.
All this is easier said than done. The words are on the page, but their real meaning will be decided only in practice: in doctrine, testing, rules of engagement, and the design of the weapons themselves. Until then, human control will remain a common phrase without a common meaning.
Gerald Mako is a research affiliate at Cambridge University and a strategic adviser working on AI governance, autonomous weapons and cybersecurity. His research focuses on human control and legal accountability for military AI, as well as the geopolitical and infrastructural implications of AI.
