A data scientist notices something small. A lending model has begun to drift. Its approvals are moving, slightly but steadily, in a way that tracks a characteristic it should not. She writes to her manager: this is concerning and the model needs recalibrating.
The manager has a launch in three weeks. He does not ignore the note. He reports it accurately but gently, as “an area to monitor”. The head of product turns the weekly reports into a slide for the executive committee. By then it is one bullet among twelve: “Model performance: ongoing monitoring”. The board pack shows a green tick.
Nobody lied. Each person made a reasonable summary for the person above. The warning did not survive the journey.
This is the second question of the AI Risk Dialogue: do the right conversations happen? It asks who holds the knowledge, who is allowed to challenge it, and whether a challenge can reach the board with its urgency intact.
There is a second way conversations fail, and it is quieter. People use the same word and mean different things.
Take “resilience”. To an infrastructure engineer it means uptime: redundant systems, failover, load balancing. To a security team it means surviving an attack without losing data. To a UK bank’s risk function it has a regulatory meaning: keeping important business services within impact tolerances through severe but plausible disruption. Three people can agree, sincerely, that a new AI service is resilient, and be agreeing to three different things. The risk sits in the gap between them, and nobody owns the gap.
“Safe”, “tested” and “aligned” behave the same way.
When I reported to the OECD on corporate risk management in 2009, one of my recommendations was that each institution should have a single meaning of risk management, agreed and communicated by the board, so that different parts and levels of the organisation could actually talk to one another. AI has simply added new words to disagree about.
Commercial pressure makes both failures worse. The push to say yes to AI is strong, visible and rewarded. Saying “not yet” tends to be treated as friction. Nobody needs to silence the person with doubts. The system simply carries yeses faster than noes.
None of this is new either. The same report found that in banks the balance between remuneration and ethical behaviour had stopped working. I urged boards to address the “dangerous silence” in which people cannot safely raise concerns, going well beyond traditional whistleblowing. AI does not change the problem. It speeds up the machinery around it.
What can a board actually do? A few things work.
- Ask for the original words. For any AI risk marked as monitored, ask what the person who raised it actually wrote.
- Define the terms once. Agree what “resilient”, “safe” and “tested” mean in the firm’s AI papers, and hold everyone to the same definition.
- Talk to people below the line. Directors who meet the people building and testing the systems hear things that never reach a slide.
- Name who can say stop. Ask who in the organisation understands the system well enough to tell the chief executive it should be switched off, and whether they would.
The last question is the uncomfortable one, because the right to raise a concern is not the power to stop. Most firms have escalation routes and whistleblowing policies. Far fewer have given anyone below the executive committee the authority to pause an AI system. If the board cannot name the person who could, its AI risk problem is not in the technology. It is in its own structure.
Richard Anderson
Founder, AndersonRisk