Human-AI Collaboration

Most disappointing AI rollouts are not model failures. The model does roughly what it does everywhere else. What is missing is a design decision that nobody made: which parts of the work the system owns, which parts a person owns, and what happens at the boundary.

That boundary is where the value and the risk both live. Set it too far towards the human and you get an expensive review queue that erases the time savings — people retyping, rechecking and effectively redoing work that was already done. Set it too far towards the system and errors reach customers, and they reach them at machine speed and machine volume, which is a different kind of incident from a single person having a bad afternoon.

The unit of design is therefore not the tool and not the prompt. It is the task, and specifically three properties of it: how reversible the outcome is, how large the blast radius is if it is wrong, and how expensive it is to verify relative to how expensive it is to produce.

That last property is the one people consistently underestimate. For some work, checking is genuinely cheap: run the tests, execute the query, reconcile the total, click the citation. For other work, checking costs almost as much as producing — a strategy memo, a legal argument, a clinical summary, a research synthesis. Automation is straightforwardly valuable in the first category. In the second, it can produce the appearance of leverage while quietly transferring effort from writing to reading, and reading a plausible document critically is harder than writing an ordinary one.

There is also an asymmetry worth naming early. AI output tends to be fluent regardless of whether it is correct, and human reviewers use fluency as a proxy for quality because in human-produced work it usually is one. A confident, well-organised, correctly formatted wrong answer defeats the review instinct that decades of reading colleagues' work has trained. Designing around that is most of the job.

None of this is new territory. Human factors research has studied the division of labour between people and automated systems since long before machine learning, and the field's conclusions — about vigilance, skill decay and the shape of the residual human task — transfer almost intact.

Four failures account for most of what goes wrong, and all four are well documented outside AI.

Automation bias. People over-trust automated recommendations, accepting them when they are wrong and failing to act when the system stays silent. Parasuraman and Riley's framework for the use, misuse, disuse and abuse of automation set out the pattern in the 1990s, and the clinical decision support literature has since documented the same effect in medicine: a confident suggestion from a system shifts the human judgement it was supposed to check.

The ironies of automation. Lisanne Bainbridge's classic 1983 paper of that name argues that the better the automation, the worse the human backup becomes. Automate the routine cases and the human is left only with the exceptions — the hardest cases, encountered rarely, by someone who has lost practice because the system handles everything else. The residual job is harder than the original one, and it is given to someone less prepared for it.

Deskilling. The longer-term version of the same problem. A team that stops writing first drafts stops being good at writing first drafts, and eventually loses the ability to tell a good draft from a fluent one. This is invisible for months and then very visible during an incident.

Review theatre. An approval step that exists on the process diagram and does nothing in practice. It appears when review has no time budget, when the reviewer has no authority to reject, when the approval interface makes acceptance one click and rejection a form, or when the reviewer receives so many items that attention becomes impossible. A near-universal approval rate is not by itself evidence that the system is good, and it is usually the first thing to check when a review step is suspected of doing nothing.

Two structural conditions make all four worse. The first is volume: a person can meaningfully review a bounded number of items per day, and any design that assumes otherwise is designing a rubber stamp. The second is the absence of feedback: reviewers who never learn which of their approvals turned out to be wrong cannot calibrate, and calibration is the only thing that turns experience into judgement.

Naming these is not pessimism. Each has a known counter-design, and the rest of this material is those counter-designs.

Delegation should be explicit and written down, at the level of the decision rather than the tool. "The team uses AI for support replies" is not a decision right. "Draft replies are generated automatically; anything involving a refund, an account change or a legal complaint requires a named approver; everything else is sent after a spot check" is.

Three questions determine where a task sits.

How reversible is the outcome? Reversible actions — a draft, an internal note, a proposal, a suggested edit — tolerate far more autonomy than irreversible ones. Sending an email, moving money, deleting data, publishing, and anything that touches a customer's record are one-way doors, and one-way doors deserve gates.

How large is the blast radius? One wrong sentence in one document is a correction. The same error applied across an entire queue is an incident. Volume converts small error rates into large consequences, and it is the reason a system that is more accurate than a human can still be riskier than one.

What does verification cost? If checking is cheap and reliable, delegate generously and verify everything. If checking is expensive, either invest in making it cheaper — structured output, citations, a diff, an executable check — or keep the human upstream, where they set direction, rather than downstream, where they must reconstruct reasoning.

A delegation ladder is a useful artefact because it makes the middle ground visible:

1. Inform. The system surfaces information; the person does the work.
2. Suggest. The system proposes options; the person chooses.
3. Draft. The system produces work; the person edits and owns it.
4. Act with approval. The system prepares an action; a person releases it.
5. Act and report. The system acts within limits and reports afterwards; the person audits samples.
6. Act autonomously. The system acts within limits; humans review aggregates and exceptions only.

Taxonomies of this kind go back to the levels-of-automation work by Sheridan and Verplank, and the useful discipline is not the exact rung count but the requirement to place each task on a rung deliberately and to write down what would justify moving it. Rungs should be earned with evidence, not assumed at launch, and the path back down needs to be as easy as the path up.

Three oversight postures cover most designs, and confusing them is a common source of both wasted effort and unpleasant surprises.

Human in the loop. No action occurs without a person. The human is a blocking step. This is appropriate for irreversible, high-blast-radius decisions, and it is expensive — it caps throughput at human speed and creates a queue that becomes a bottleneck and then, predictably, becomes review theatre when the queue grows faster than the reviewers.

Human on the loop. The system acts, a person monitors, and there are defined conditions under which the person intervenes. Throughput is not capped by review capacity, but the design now depends entirely on the quality of the intervention triggers and on the human retaining enough situational awareness to act. Endsley's work on the out-of-the-loop performance problem is the reason monitoring roles need deliberate design rather than a dashboard.

Human out of the loop. No runtime human involvement. Control is entirely upfront — limits, permissions, validation — and after the fact through audit. This is legitimate for low-stakes, high-volume, well-bounded tasks, and it should be a considered decision rather than the state a system drifts into.

The interesting engineering is in the escalation rules, because they decide which items get scarce human attention. Useful triggers include low model confidence where confidence is actually calibrated, disagreement between two independent attempts, novelty relative to previously seen cases, anything crossing a monetary or permission threshold, membership of a protected or high-sensitivity category, and any action that is irreversible.

Two design rules matter more than the trigger list.

The first is that review capacity is a hard budget. Decide how many items per day a person can genuinely examine, then design the triggers to fit that number. Triggers written without a capacity constraint produce alert fatigue, and alert fatigue produces exactly the automatic approval the review step existed to prevent.

The second is that not every item needs the same depth. A tiered scheme — full review for triggered items, structured sampling of the rest, deep audit of a small random subset — gives you both coverage and a measurement of what the untriggered stream actually contains. Random sampling is what tells you whether your triggers are catching the right things, and dropping it because everything looks fine is how teams stop finding out that it is not.

The goal is not high trust or low trust. It is calibrated trust: reliance that tracks actual reliability, at the granularity of task type rather than of the tool as a whole. The same system can be dependable at summarising a document you supplied and unreliable at recalling a fact you did not, and a team that has one global opinion about whether the AI is good will be wrong in both directions.

Calibration requires evidence, which means someone has to look. The mechanism that works is unglamorous: keep a record of where output was wrong, categorised by task type, and revisit it. A team that can say "it is reliable for extraction from provided text, unreliable for anything requiring a citation we did not give it, and dangerous on arithmetic in tables" has knowledge. A team that says "it is pretty good" does not.

Interface design does a large amount of the work here, and a few patterns reliably improve review quality.

• Show the evidence, not just the conclusion. A claim with the source passage next to it can be checked in seconds. The same claim alone requires the reviewer to reconstruct the work.
• Make the diff the unit of review. Reviewing a change is tractable. Reviewing a regenerated whole document is not, and reviewers respond to it by skimming.
• Make rejection as cheap as approval. If accepting is one click and rejecting requires a written justification, the measured approval rate tells you about the form design, not about quality.
• Add friction in proportion to blast radius. A confirmation step for an irreversible action is not bad UX; uniform friction on everything is.
• Be careful with confidence displays. An uncalibrated confidence score can make judgement worse rather than better, because it looks like information and is treated as such. Either calibrate it against outcomes or show the underlying evidence instead.
• Preserve the human's independent opinion where it matters. Asking a reviewer to form a judgement before revealing the system's answer costs time and is the strongest available defence against anchoring. Reserve it for high-stakes decisions rather than applying it everywhere.

Verification protocols deserve to be written down per task type. What must be checked, in what order, and what evidence must exist before approval. A short explicit checklist beats a vague instruction to review carefully, because "review carefully" degrades under time pressure and a checklist degrades more slowly.

Most AI adoption metrics measure activity: seats, sessions, tokens, acceptance rate. None of them tell you whether the work got better, and acceptance rate in particular can rise for the worst possible reason.

Better instrumentation starts from outcomes and errors.

An error taxonomy. Categories with real teeth: fabricated fact, misread source, correct but off-brief, subtly wrong number, unsafe action, missed requirement. Aggregate error counts are almost useless because the categories have wildly different costs. A style miss is not a fabricated citation.

Escape rate. How many defective items got past review and reached the customer, the repository or the ledger. This is the number that matters, and it can only be obtained by looking downstream — from complaints, incidents, reconciliations and audits — rather than from the review step itself.

Catch rate. The complement: of the defects that existed, how many did review catch. Estimating it requires seeding or independently re-examining a sample. It is the only honest measure of whether a review step is functioning, and it is the number that exposes review theatre.

Time to verify. Track it separately from time to produce. If verification time is growing while production time falls, total effort may not have moved at all, and you now know why the promised savings are not appearing.

Reviewer disagreement. When two people review the same item, do they agree? Low agreement means the standard is not shared, and no amount of process will fix an undefined standard.

Auditability is the other half. For any consequential output you want to be able to reconstruct what the system was given, what it produced, what version and configuration were in use, who approved it, and when. This is what makes an incident investigable and a regulator answerable, and it is far easier to build in at the start than to retrofit after the first incident. Immutable logging, retention aligned to your privacy obligations, and access controls on the logs themselves are the standard components.

For teams operating under formal obligations, the risk-management framework published by NIST is a reasonable structure for organising all of this, and the European Union's AI Act sets explicit human-oversight requirements for systems it classifies as high risk — meaning that for some organisations the design questions in this material are not optional preferences but documented controls.

Shadow mode deserves a mention as the safest way to earn a rung on the delegation ladder. Run the system on live inputs, log what it would have done, and compare against what humans actually did. You get a real performance estimate on your own distribution with no exposure, and the disagreements are the most informative dataset you will get.

Rollouts fail for organisational reasons far more often than technical ones, and the causes repeat.

The most common is an unstated incentive conflict. If throughput is measured and quality is not, reviewers will approve quickly, because that is what the organisation asked for. Any oversight design that contradicts the incentive structure loses to the incentive structure. Fix the measurement before blaming the reviewers.

The second is missing accountability. When a human approves an AI-produced action, they own it — and that only works if they had the time, the information and the authority to refuse. Assigning responsibility without those three is a way of manufacturing blame rather than safety.

The third is training that covers the tool instead of the work. What people need is not a feature tour. It is the local knowledge of which tasks this system is reliable at, what its characteristic errors look like, how to verify each task type, and when to escalate. That content is specific to your process and cannot be bought.

A workable sequence looks like this: pick one workflow with a measurable output, define the delegation rung and the escalation triggers, instrument errors and verification time, run in shadow mode until you have a real error profile, then move one rung and watch the same numbers. Expanding to a second workflow before the first is measured is the most common way to end up with broad adoption and no evidence.

As for whether this material fits you — it does if you own or influence a process where AI output reaches something that matters, and you have already felt the gap between an impressive demonstration and a dependable workflow. It does not fit you if you are looking for model-building content, because none of this involves training or serving anything, or if you have not yet used these tools on real work, since the failure modes only become recognisable through contact with them.

What follows naturally from here depends on your direction. If you are supervising autonomous agents, the next thing to understand is permissions and blast-radius control at the tool level. If you are accountable for compliance, the next thing is audit logging and access control as concrete implementations rather than policies. If you are designing the interface, the next thing is evaluation, because you cannot design a review step without knowing what it is supposed to catch.