What the AIOP domains measure: a construct validity map

The AI Interaction Outcome Profile was built from clinical practice rather than from the literature. It measures what happens to a person across repeated interaction with an AI system: six bipolar domains, profile-based, assessed at intervals rather than once.

That origin is a strength and a liability. A strength because the domains describe what is actually observable on a ward. A liability because an instrument defended only by its author’s clinical reasoning invites a fair question: what is this measuring, and has anyone measured it before?

The answer for five of the six domains is yes, at length, in a literature that predates generative AI by decades. This page states the anchor, the reference and what the instrument adds. Where there is no anchor, it says so.

A dyad has two sides. Almost every instrument in current use measures one of them.

Domain anchors

Displacement ↔ supplementation

Anchored in the automation misuse literature. Parasuraman and Riley’s 1997 review defines misuse as over-reliance on automation resulting in failures of monitoring or decision biases — the same movement the AIOP measures as displacement of the person’s own judgement. Parasuraman and Manzey’s 2010 attentional integration extends it: automation complacency occurs under multiple-task load, when manual tasks compete with the automated task for the operator’s attention. A short-staffed ward at night is that condition by definition.

Reference. Parasuraman, R. & Riley, V. (1997) ‘Humans and automation: use, misuse, disuse, abuse’, Human Factors, 39(2), pp. 230–253. doi:10.1518/001872097778543886. Parasuraman, R. & Manzey, D. (2010) ‘Complacency and bias in human use of automation: an attentional integration’, Human Factors, 52(3), pp. 381–410. doi:10.1177/0018720810376055.

What the instrument adds. The existing literature measures the phenomenon in controlled tasks with a defined correct answer. The AIOP measures it as a bipolar axis in clinical use, where supplementation and displacement can present identically in a single observation and separate only over time.

Epistemic stance

Anchored in the trust-calibration literature. Lee and See frame the problem exactly as the domain does: automation is often problematic because people fail to rely upon it appropriately — the failure is miscalibration in either direction, not trust as such. Overtrust and undertrust are both defects.

Reference. Lee, J.D. & See, K.A. (2004) ‘Trust in automation: designing for appropriate reliance’, Human Factors, 46(1), pp. 50–80. doi:10.1518/hfes.46.1.50_30392.

What the instrument adds. Trust calibration is normally assessed against a system of known reliability. In clinical deployment the reliability is unknown, contested and drifting, so the AIOP records the stance the clinician holds rather than the accuracy of that stance.

Regulation in absence

Anchored in the out-of-the-loop performance problem, a named construct since 1995. Endsley and Kiris describe it as leaving operators of automated systems handicapped in their ability to take over manual operations in the event of automation failure, attributed to loss of skills and of situation awareness arising from a shift from active to passive information processing.

Reference. Endsley, M.R. & Kiris, E.O. (1995) ‘The out-of-the-loop performance problem and level of control in automation’, Human Factors, 37(2), pp. 381–394. doi:10.1518/001872095779064555.

What the instrument adds. The original construct is assessed by performance after a staged automation failure. That test is not available in clinical practice, where the failure is not scheduled. The AIOP asks the question prospectively instead.

Human alliance

Anchored in team psychological safety, and this is where the dyad connects to the team. Edmondson defines it as a shared belief held by members of a team that the team is safe for interpersonal risk taking — the condition under which a person raises a doubt rather than deferring to a more confident voice. The AIOP domain asks the same question with the machine as the confident voice.

Reference. Edmondson, A.C. (1999) ‘Psychological safety and learning behavior in work teams’, Administrative Science Quarterly, 44(2), pp. 350–383. doi:10.2307/2666999.

Limits of the anchor. Edmondson’s design is cross-sectional, single-company, and she declines to claim causality. The construct is well established; its transfer from human teams to human–AI dyads is an assumption this instrument makes and has not yet tested.

Agency

Partially anchored. The domain draws on self-efficacy, locus of control and diffusion of responsibility, each of which has a validated measurement history, but the AIOP does not currently commit to one of them. That is a gap in the instrument rather than in the literature, and naming it is more useful than choosing retrospectively.

Disclosure pattern

Not anchored in a psychological construct. The domain measures whether a clinician records that a system contributed to a decision, and to whom that is visible. It is a documentation and record-integrity concern with governance precedent rather than a measured behavioural phenomenon. It is retained because it is clinically load-bearing, not because the literature supports it.

The finding that matters most

One result from the automation literature bears directly on how AI is currently being governed in health and care. In the Parasuraman and Manzey review, automation complacency is found in both naive and expert participants and cannot be overcome with simple practice; automation bias occurs in both naive and expert participants and cannot be prevented by training or instructions.

If the failure mode cannot be trained away, then training cannot be the mitigation that carries it.

The policy direction runs the other way. The National Commission into the Regulation of AI in Healthcare warns that responsibility for errors may be transferred onto healthcare professionals, and the coordinating response is to build AI capability through education. Education is necessary. On this failure mode it is documented as insufficient, and a safety case that rests on it is resting on a control the evidence does not support.

That is the argument for measuring the person directly. An attestation record proves a named individual decided; it cannot distinguish an independent judgement from a rubber stamp. Override rate over time is the obvious proxy and it is confounded — a falling override rate is either a system improving or a clinician who has stopped checking, and the log cannot separate the two, though they demand opposite responses.

A testable implication

If displacement in a human–AI dyad is the same construct as deference in a human team, then scores on a validated team instrument should correlate with the AIOP displacement axis in the same population. That is falsifiable, it has a direction, and a null result would constrain the instrument’s scale-invariance claim rather than merely disappoint it. It is stated here as an implication and is not among the seven predictions currently registered in the estate.

Sources