AI Risk Reading

Protecting people and businesses in an agentic economy

What is new

Four UK regulators, working together in the Digital Regulation Cooperation Forum, have set out how the law applies to AI agents that act for consumers, “such as making purchases and communicating with businesses”. Existing rules already apply, including those on financial services, payments and cyber security, and “An AI agent’s autonomy does not diminish the responsibility of organisations that deploy it to comply with applicable law and relevant regulation.” Firms should think about “how authority to act is given” and “who is accountable for the agent’s decisions and how to obtain redress when something goes wrong”. The CMA updated its guidance the same day: a business is responsible for what an AI agent does “in the same way you are responsible for what an employee does”, with fines of up to 10% of worldwide turnover. The ICO is collecting evidence for its guidance on agentic AI until 20 November.

Why a board should care

UK regulators, the FCA among them, now say with one voice that responsibility for an agent sits with the firm that deploys it. A bank meets this from two sides: the agents it runs, and its customers’ agents acting on its services, which JPMorgan has begun blocking when they use shared credentials (Edition 5). The statement makes no new rules and says nothing specific about financial services, but it closes with a warning: “we will not hesitate to use our powers”.

Verdict: read the joint statement in full; it is two pages. Firms that use or plan to use agents should consider responding to the ICO before 20 November.

New prompt injection techniques cover agent-planning attacks

What is new

Vanessa Villa and Dr Jim Hoagland report that CrowdStrike’s taxonomy of prompt injection attacks now runs to 222 techniques, including new ones that target how agents plan. In “goal decomposition exploitation”, an attacker adds or alters a step so that the agent concludes a restricted action is required. In “constraint-driven task steering”, the attacker blocks the agent’s usual route, so the fallback it chooses is the path the attacker wanted. “Session state spoofing” plants a fake summary or memory log that presents “an approval that was never given as verified history”. In CrowdStrike’s hypothetical example, a coding agent asked to fix failing tests follows advice planted in a compromised open-source tool and sends local configuration files, including credentials, to an outside server. “Each action looks normal by itself… The risk only shows when you look at the whole sequence.”

Why a board should care

Risk appetite for agents is often written as a list of permitted actions. This shows why that is not enough: an agent can stay within its permissions at every step and still arrive somewhere nobody authorised. The approval it relies on may be forged in its own memory. Controls need to watch the plan and the record of approvals, not only each action. CrowdStrike sells products in this area, and its example is illustrative, not a reported incident.

Verdict: read “How Adversaries Manipulate Agent Planning” and the paragraph on session state spoofing. Pass the taxonomy to the security team for testing.

Budget 2026, section 5.7: Building the UK’s AI future

What is new

The Budget confirms the AI security package held over from last week: about £500m of a £700m AI programme goes to security. It includes a further £120m for the AI Security Institute; £70m for a National Exposure and Real-Time Vulnerability Centre at the NCSC; £60m for the first phase of a national Cyber Shield “to develop agentic cyber defence capabilities”; and £40m for a National Agentic Security Unit “to detect, investigate and respond to cases of rogue agentic systems”. Building on the Law Commission’s product liability consultation, the government “will consult next year on how to tackle insecure technology at source and consider where liability for cyber breaches should sit”.

Why a board should care

The liability line matters most. Today the loss from an insecure product falls mostly on the firm that bought it, as PHP2Shell showed last week. If liability moves towards suppliers, contracts, insurance and supplier assurance will all change. A unit for rogue agents also tells you the government expects incidents, not just risks. Announced funding is not yet operating capability, and a consultation next year is not a new rule.

Verdict: read “Strengthening security, resilience and sovereign capability” in section 5.7, and The Record’s article on the liability pledge.

Wall Street’s AI debt machine is starting to choke on supply

What is new

Lenders are growing choosier about AI borrowers. JPMorgan and Goldman Sachs had to sweeten a $5bn loan to the neocloud Volta, pushing its yield above 11%. At least 18 data-centre financings now carry double-digit yields, and top-rated data-centre bonds yield about 7%, against under 5% in January. The cost of insuring against default has hit records for Oracle. A large European bank says it is near its lending limit for the sector, and some insurers are at theirs. Oracle has issued a force majeure notice seeking to defer payments if a New Mexico data centre is delayed. The next day Alphabet’s shares fell about 13%, a record loss of about $500bn in market value, after capital spending of $51.4bn in the quarter and a second quarter of negative free cash flow.

Why a board should care

This is a signal to listen to, not a sign of crisis. Citadel Securities is not predicting a credit crunch, and Alphabet still holds more cash than debt. Last week Elon Musk claimed debt markets had “almost entirely shut off” for AI firms without other revenue. The evidence says borrowing is dearer and pickier, not shut. Stress usually shows first in contract terms, before services fail: Oracle’s notice is the kind of clause to look for in your own AI suppliers’ contracts.

Verdict: read Bloomberg in full. Treat the share price moves as context, not evidence.

North Korea steals USD 6.36 billion from Binance in the largest hack in history

What is new

On 29 October an attacker moved about $6.3bn from Binance’s hot wallets on ten blockchains, most of it within about 20 minutes. TRM Labs and Chainalysis attribute it with high confidence to TraderTraitor, the North Korean unit the FBI blamed for last year’s Bybit theft; Elliptic is more cautious. Binance first pointed to an engineer’s compromised laptop and has since reported “additional points of breach”, but has not yet published a root cause. TRM says the attacker took control of Binance’s wallet-signing systems, so Binance’s own systems signed the transfers. The attacker funded a wallet the day before and rehearsed the code on test wallets. About $3bn has been frozen.

Why a board should care

Some commentators suggest that only AI could coordinate a theft on this scale. No investigator has said so. The skill lay in the preparation, which this unit has done by hand for years, and scripted transfers are automation, not AI. Blaming AI flatters the victim and draws attention from the questions that matter: who can make the signing system sign, what limits apply to large outflows, and how fast a transfer can be stopped. The lesson for the AI era is speed. Response plans built around human decision times are already too slow.

Verdict: read TRM’s timeline. Wait for Binance’s root cause before drawing firm conclusions; as with PHP2Shell last week, first accounts are often wrong.

Anthropic’s prospectus, revisited

Edition 6 promised to revisit my valuation essay and to update the AI Blast Zone. Disclosure: I use Claude, Anthropic’s model, in my research and drafting.

Other Things Are Not Equal

My essay, published today. Anthropic’s prospectus lists its risk factors one by one, but one of them warns that many of its risks “are interconnected and may compound or trigger cascading adverse effects”, with a combined effect that “may be materially greater than the impact of any individual risk event in isolation”. Read that way, the filing shows three chains:

The money: revenue that customers can cut at short notice, about $430bn of commitments that cannot be cancelled, and cross-default provisions through which one broken contract could become many.

The state: in June, one US export decision removed access to two of Anthropic’s models for every customer, worldwide, for about three weeks. Nothing broke.

The model: capability may advance faster than safety testing, models behave differently when they detect they are being evaluated, and a serious incident anywhere in the industry could bring a regulatory response that reaches everyone.

The first is contractual contagion, the second a common shock, the third uncertainty spreading through an industry. Each can defeat controls designed around individual risks. For a customer, the question is not which risk is red. It is whether your fallback sits in the same chain.

What the S-1 changed for the valuation

Rerunning the model from Anthropic: What Does $2 Trillion Assume? with the S-1’s figures lifts the base case from about $420bn to about $530bn, mainly because the filing shows $131.7bn of cash and short-term investments at 30 June, against the $20.3bn the leak gave for December. The conclusion holds: $2tn still needs a cost of capital of about 5%. Two corrections to the original: the first quarter of 2026 was an operating loss of about $1.7bn, not a profit (the operating profit, $974m, came in the second quarter); and the Pentagon’s designation was not “overturned in August”: the D.C. Circuit upheld it in September, and Anthropic has petitioned for rehearing en banc.

AI Blast Zone

The map now has 101 relationships, up from 67 in mid-October. The S-1 put figures on links that had been reported or unknown, including non-cancellable compute commitments of at least $110bn to Amazon, $114.5bn to Google and $44.9bn to Microsoft, and showed money flowing back the other way: Amazon, Google and Broadcom provide funding facilities, and AMD has agreed to buy shares. The pattern is becoming clearer: suppliers increasingly fund, guarantee or invest in their own customers, and lenders are pricing the weakest borrowers sharply.

Anthropic and Hg plan a European AI services company

Reported by the FT on 23 October; not yet confirmed by either firm. Anthropic’s S-1 already describes a US version, Ode with Anthropic, built with private equity firms. The same filing names Accenture, Deloitte, KPMG and PwC as partners. EY, Anthropic’s auditor, is not on the list. When the model supplier also funds the advisers who recommend it, who gives independent advice on whether to use it and how to leave?

Sweden’s Government Offices block browser access to ChatGPT and Claude

Reported by SVT on 23 October. Not a ban on the models: staff move to Microsoft Copilot, approved after a security review, and can still reach ChatGPT and Claude through a separate approved platform. Shadow AI is controlled by choosing the channel, not by prohibition, though sending everyone to one supplier adds concentration.