top of page
Search
Why Intelligence Agencies Fail When It Matters

Why Intelligence Agencies Fail When It Matters

Writer: John Fullerton
John Fullerton
Nov 14
5 min read

Russia's assumptions about Ukraine in 2022 offered a brutal answer to why intelligence agencies fail. A service can have files, intercepts, paid sources and formidable powers, yet still tell its political masters what they want to hear. The failure is rarely a shortage of secrets. More often, it is a shortage of candour.

The public tends to imagine intelligence failure as one missed coded message or one traitor slipping through a checkpoint. Reality is less tidy and more troubling. Agencies fail because institutions distort information, human sources deceive, leaders reject unwelcome assessments, and technology produces an abundance of data without supplying understanding.

Why intelligence agencies fail: the truth arrives unwelcome

Intelligence is meant to reduce uncertainty, not abolish it. Its central task is to give decision-makers a better grasp of what an adversary intends and can do. That distinction matters. Capability can often be photographed, intercepted or counted. Intention resides in human minds, where fear, pride, faction and miscalculation do their work.

The Russian Federal Security Service (FSB) appears to have misread Ukrainian resistance and the condition of its own networks. The problem was not simply poor collection. It was a system in which optimistic reporting travelled upwards more safely than sceptical reporting. When a ruler has made his preference clear, subordinates may turn analysis into reassurance.

This is not a peculiarly Russian vice. Democracies have their own pressures: ministerial deadlines, electoral calculation, departmental rivalry and the desire to avoid being blamed for an unpopular judgement. The most dangerous intelligence culture is not one that lacks clever people. It is one where clever people learn which conclusions are career-limiting.

Good assessment therefore needs protected dissent. Analysts must be able to say, plainly, that the evidence does not support the policy. Senior officers need to distinguish between a confident source and a reliable one. Politicians need the discipline to hear an answer they dislike. None of this is glamorous. It is the hard, unromantic craft that prevents catastrophe.

The seduction of the source

Human intelligence, usually shortened to HUMINT, remains indispensable because people reveal intentions that satellites and sensors cannot see. But people also lie. A double agent may feed a service carefully prepared nonsense for years. An illegal - an intelligence operative living under a constructed identity - may look wholly ordinary until the moment he does not. A mole inside an agency can expose methods, betray agents and quietly shape what leaders believe.

The case of Kim Philby remains the classic British lesson: access does not equal loyalty. Ben Macintyre's A Spy Among Friends captures the social confidence and institutional blindness that allowed a gifted traitor to flourish. Christopher Andrew and Vasili Mitrokhin's The Mitrokhin Archive shows the scale and patience of Soviet penetration. The point is not that every charming insider is a traitor. It is that familiarity is a poor test of trust.

A source may be truthful about one matter and deceptive about another. He may be reporting what he thinks the case officer wishes to hear. He may be frightened, greedy, vain, compromised or under another service's control. The professional question is never merely, ‘Do we believe him?’ It is, ‘What would he gain by making us believe this, and what evidence would prove us wrong?’

That question requires empathy. Not sentimentality, and certainly not friendship, but an informed sense of another person's motives, pressures and worldview. HUMINT collection fails when an officer treats a source as a container of facts rather than a complicated human being. The same is true when an analyst mistakes cultural assumptions for universal ones.

Strategic deception works because it gives us what we expect

The finest deception operations do not invent a fantasy from nothing. They exploit an opponent's existing belief. If a service expects an adversary to be weak, it will readily accept reports of weakness. If it expects aggression, every ambiguous movement can look like proof.

This is why confirmation bias is so corrosive. Analysts are trained to weigh evidence, yet the organisation around them may reward consensus. Separate reports from different sources can look persuasive when they all began with the same planted story. Technical collection can reinforce the error if it observes activity designed to be seen.

John Buchan understood the tension between appearance and motive. Eric Ambler made it central to his morally compromised characters. Len Deighton gave bureaucracies their own appetites and evasions. Mai Jia's fiction is particularly alert to the psychological cost of codes, secrets and competing loyalties. The best espionage fiction does not present intelligence as omniscience. It presents it as judgement under pressure, made by fallible people with partial sight.

More data can make failure easier

Signals intelligence - information drawn from communications and electronic emissions - has transformed the business. The United States National Security Agency (NSA), Britain's Government Communications Headquarters (GCHQ) and their partners can collect at extraordinary scale. Programmes publicly revealed over the past decade, including PRISM, XKeyscore, Stellar Wind and the abandoned Trailblazer project, demonstrate both the reach of modern collection and its political, legal and analytical burden.

The uncomfortable truth is that collection is not comprehension. A database can retrieve a name, a location trail or a pattern of contact. It cannot, by itself, explain whether a meeting was coercive, routine, deceptive or decisive. Analysts drown when systems create more leads than trained minds can test.

Artificial intelligence (AI) sharpens that dilemma. It can identify faces, translate material, sort imagery and flag anomalies at a speed no human team can match. It can also reproduce the prejudices of its training data, elevate coincidence into suspicion and create a false aura of certainty. Cyber operations add another complication: an adversary can manipulate data, disguise origins and force an agency to spend months chasing a technically plausible fiction.

AI is useful for triage. It is dangerous when its output is treated as a verdict. The more secretive the target, the more valuable human context becomes. A machine can identify that a person has changed his routine. It cannot reliably tell you whether he is preparing to betray his country, visiting an ill relative, or trying to avoid surveillance.

The institutional failure beneath the operational one

Agencies are built to protect secrets, but excessive compartmentation can prevent the right people seeing the whole picture. One department may hold a financial clue, another a cyber indicator, another a source report. Each may be properly classified and carefully handled. Together, they may reveal the threat. Separately, they are merely fragments.

There is a trade-off. Share too freely and sources, methods and innocent people may be exposed. Share too little and warning arrives too late. The answer is not indiscriminate access. It is disciplined collaboration, clear authority and leaders who insist that rivalry must stop at the point where national security begins.

Richard J. Aldrich's GCHQ, Keith Jeffery's MI6: The History of the Secret Intelligence Service, and Philip H. J. Davies's MI6 and the Machinery of Spying are useful correctives to the fantasy of a single all-seeing service. Intelligence is an ecosystem of collectors, analysts, handlers, technical specialists and ministers. Its strengths are also its seams.

Failure cannot be abolished, because adversaries adapt and uncertainty is real. But agencies can fail less expensively when they reward bad news, test cherished assumptions, protect dissent and remember that secret information is not the same thing as knowledge. For the reader of serious espionage, that is where the real tension lies: not in the gadget, but in the moment somebody must decide what to believe.

 
 
 

Comments


bottom of page