People, ideas, machines VII: 'The Wizard War' - lessons on technology, intelligence & organisation from World War II

RV Jones's classic Memoir... Science is cumulative, the lessons of politics are cyclical, endlessly forgotten and re-learned... Whitehall is in 'forgot & don't want to re-learn' mode...

Courage is rightly esteemed the first of human qualities because it is the quality that guarantees all others. Churchill

Despite any unpopularity, I survived because war is different from peace: in the latter fallacies can be covered up more or less indefinitely and criticism suppressed, but with the swift action of war the truth comes fairly quickly to light — as Churchill said, ‘In war you don’t have to be polite, you just have to be right!’ RVJ

The temptation to tell a Chief in a great position the things he most likes to hear is the commonest explanation of mistaken policy. Thus the outlook of the leader on whose decisions fateful events depend is usually far more sanguine than the brutal facts admit. Churchill

[UPDATED: 23/5: final thoughts and general lessons.

The war ends, RVJ goes to Aberdeen, Whitehall dismantles the successful system built during the war (like it dismantled the Vaccine Taskforce and other things built in 2020), committees replaced individual responsibility, analysts pulled away from the front line, talent pushed out of Whitehall, Alanabrooke’s COS system also vandalised — a prelude to decades and thousands of acts of Whitehall vandalism that has destroyed talent in public service and made our institutions pathological.

SW1 discusses all new ‘scandals’ and failures as if they’re ‘shocking’ and an effect of recent developments but they all trace back to the 19th Century, fundamental dynamics of big organisations from Google to the Pentagon, and a core issue — is a permanent bureaucracy that recruits overwhelmingly from an internal caste (A) a ‘Rolls Royce civil service’ or (B) ‘a programmed pathological disaster that will force stupidity even in an existential crisis and remorselessly deteriorate decade after decade, impervious to all discussion of reform’? SW1 thinks (A), I think (B)…

The successes in WW2 (e.g Alanbrooke’s COS system, RVJ and Scientific Intelligence) came from rare individuals who found ways to build things with effective-enough barriers against the entropy and pathologies of normal Whitehall, NOT from ‘normal Whitehall’ which was a constant source of entropy (NB. the Germans suffered from similar problems with committees, cf. V2 program). It’s only when the Northcote-Trevelyan system itself is replaced, as should have happened after 1945 and could have happened in the 12 months after April 2020, that we have a hope of revival. All ‘reform’ discussion is conducted in ignorance of (99%) or knowingly deflecting from (1%) the issues around the pathological dynamics of ALL big organisations.

Whatever happens in the election, 99.99% of the same people will stay running the country as now, Starmer will have the same attitude to the civil service actually running the country as Cameron and Sunak, and the situation since 2010 will largely continue: The government does not control the government, doesn’t want to, and couldn’t if it tried… Cf. Francis Crick’s plea as Whitehall wrecked Intelligence 1946: ‘It’s no use reorganising with just the same old gang’. He was ignored and he left for Cambridge.

I urge subscribers to ignore the election. It will be almost entirely clowns jabbering things not-even-wrong interpreted by hacks who’ve never built anything valuable in their lives and are anti-expert on how power works, how communication works, and how high performance organisations are created. Noise about noise. All the budget numbers will be fake because of the massive black budget horrorshows and corruption of the MOD. Starmer will be given these on yellow paper soon after he goes to No10 and he and others will say to themselves ‘un-fucking-believable’. Then, probably, punt-and-classify like Brown, Cameron, May, Boris, Truss, and Sunak. I’ve been talking to various people about what should be built after the 2010-24 clown show is over and the new clown show begins. As promised I’ll share here first.]

The Wizard War: British Scientific Intelligence 1939-1945 is the memoir of RV Jones, a legendary physicist who built and ran a new science and technology intelligence operation in World War II. Aged just 28 RVJ was summoned to No10 by Churchill himself in 1940 to discuss the ‘battle of the beams’, the offensive and defensive duel over radar and navigation that was central to the Battle of Britain. From 1939-1946, RVJ was Head of Scientific Intelligence on Britain’s Air Staff and Scientific Adviser to SIS (a crucial role that exists today and is greatly underrated and under-supported in Westminster). His operation played a vital role including in defeating the German air force in the Battle of Britain and many other critical battles of the war.

It is a wonderful, inspiring, fascinating tale. It has many deep lessons for today whether you’re in government, running an AI company, a VC investor etc. It also deepens one’s gloom and anger about how Westminster has failed year in year out since 1945 in so many ways, something RVJ refers to himself:

And in a Britain that has been drifting downstream ever since 1945, I hope this story will show one facet of what we could achieve no more than forty years ago, and what therefore — since a people cannot change its basic make-up so quickly — Britain could achieve again, if it could only replace the present mood of self seeking easement with a sense of purpose and service.

These words ring true and heavy for us.

Previous in this series:

VI: Alanbrooke diaries, incredibly relevant to today’s problems and what military ‘strategy’ really is

V: Colin Gray and defence planning

IV: Notes on The Kill Chain

III: More on fallacies of nuclear thinking / strategy / deterrence

II: Thinking about nuclear weapons

I: On innovation in militaries, when does it succeed/fail

Nothing I write about is of less interest to political people in Westminster than the intersection of war, politics, technology and management. Near zero MPs or political hacks are interested and the only discussion I have is with the odd deep state official, military people, and people outside Westminster where there is 100X more interest. This is not accidental but points to a general, important, and deeply bad feature of our political class.

For example, I’ve written on this blog many times since leaving No10 about something of the absolute first importance — the dangerous rot of our nuclear weapon infrastructure, the massive bills for this vast failure, the way these bills make all official budget numbers (including OBR numbers) fake, how this is covered up by more and more classification, how I started working with the deep state to sort it out in 2020, how much of this was kiboshed within hours of leaving No10 (when Boris encouraged senior officials to u-turn on facing the truth on budgets and instead double down on lies), how I insisted to Sunak in our two discussions on returning to it as a condition for helping him politically and so on.

And I’ve pointed out that the old media doesn’t cover the story and our MPs actively avoid thinking about such problems — a very large, important, and disastrous shift in our political system, one so important it is entirely invisible in the mainstream news and pundit-world. This week another hard-to-keep-secret symptom of the rot hit the media with the botched missile exercise incident. Nothing will change. Sunak decided to go with the Whitehall flow. Once Whitehall itself realises that the political leadership has no gumption and no desire to face reality, it will, acting naturally in accordance with the perpetual laws of large bureaucracies, hide, punt, and manage the theatre of power for the ministers without the truly important issues on the agenda. Our MPs will not, as they could, insist on uncovering this 20 year tale of dereliction of duty. Our media will not expose the problem just as they almost entirely ignore the day-to-day farce of procurement and Ukraine. (MPs really are needed here because it is illegal for those who know about it to discuss most of the nuclear issue in any detail unless the MPs create a process allowing people to speak legally, and it is not a subject that should be briefed/leaked as many others are treated.)

Starmer will be confronted with a symbol of this on his first day as PM when he talks to the deep state about the submarines and his letter. And the Cabinet Secretary will say something like: PM, not for now but we will have to discuss some important aspects of this subject soon… And Starmer will read (on yellow paper above Strap 3) the detail of these horrific budgets. And he will face the same choice Boris and Sunak faced: go public, blame his predecessor and face openly the vast financial (and other implications) or classify, punt and continue the charade that means the continuing cannibalising of the open budgets by the broken black budgets and their black holes. (An interesting question that will signal power will be: is Sue Gray allowed in the room for the submarine chat or not?)

These deep problems are behind the predictable and predicted in this blog disastrous failures in Westminster over Ukraine — while, of course, the MPs and hacks have told themselves an opposite story of bravery and success, a fake story now colliding with reality as always happens eventually in war. But, as with covid, our failures are also seen across the west and should be seen as only partly idiosyncratic and, most importantly, as part of a general rot of the West’s post-1945 regimes.

In the world of RV Jones, duty and public service were seen differently. Senior ministers took their highest duties seriously. As Bunk says to Omar, makes me sick how far we done fell…

Below are notes. I will tweet a link to each update so subscribers know.

---

Some subscribers of this blog are researchers and/or in the deep state working on some of these issues. What’s the very best stuff written on:

• What are the best examples of people doing safety well and drastically improving standards with extremely dangerous technologies? (E.g improving nuclear weapon safety after they started with 0000 pin numbers, the differences between the best bio-labs and those that have lab leaks.)

• What are the worst examples and why do they persist even in the most sensitive areas? (E.g repeated debacles with nuclear weapons constantly exposed by Red Teams that politicians do not act on.)

• What are important general lessons from security we can be confident in? (E.g the recurring problem of vulnerability to social attacks on key individuals, such as the KGB’s honey trap girls (some of whom I met in Moscow 30 years ago). Defending against this is a general social/management problem for everyone trying to maintain security regardless of the particular technology. If I were a state looking to evade sophisticated security in top AI firms, I would start with such tried and trusted (and relatively cheap) methods. Many recently suggested a ‘Manhattan Project for AGI’ to bring the top people together in a safe place. Leaving aside all other aspects of this idea, those suggesting it rarely seem aware that the original, conducted with a level of secrecy and security hard to imagine today, was so thoroughly penetrated by the Soviets that Stalin and Beria knew more about many critical aspects than Truman.)

• A lot of critical software now is open source which is more secure because it is open to scrutiny. Which systems are not open for good reasons? What lessons are there from software for nuclear, bio, AI? What’s the best exploration of the ‘open source is vital for software security’ versus ‘yes but this will break at a certain point with AI models’ argument? I’ve seen many speculative assertions about this but not detailed analysis. This has huge economic importance because vast investments are being made in OS, and many entrepreneurs are basing products on OS to avoid being vulnerable to events like the OpenAI coup or the crazed Google-like decisions evident this week, but many safety people argue OS models will have to be stopped/regulated ‘for safety’.

• What lessons are there for software/AI from biologists with the security mindset, such as Kevin Esvelt, who are pushing for large improvements in bio security? Some aspects of bio-defence must be decentralised and open but some he suggests should be closed and secret. What implications for AI? (NB. over the next few years the capability to build bio-weapons will be radically democratised with vast implications being ~100% ignored by western regimes as you’d expect from regimes that have allowed gain-of-function to continue after covid.)

• What are examples of governments and companies collaborating well together on complex systems problems of great significance to governments? (E.g a case study is TRW and Simon Ramo working with Schriever on ICBMs, after von Neumann’s Teapot Committee, developing the core ideas of ‘systems engineering’ and ‘systems management’. I wrote in 2017 how these lessons would be central to the next pandemic (and how EU procurement law would kill people) and so it turned out but a) the Pentagon systematically unlearned these lessons starting in the McNamara era, b) so did we, c) covid and Ukraine have shown how hostile our regimes now are to doing this properly — their revealed preference is ‘rather lose in Ukraine than face reality on procurement and industrial capacity’.)

• Which regulators have done a good/bad job in drawing lines defining banned/unbanned R&D? (E.g you can research X in nuclear physics but Y is illegal unless authorised by government; you can create CRISPR but you can’t engineer a more deadly smallpox.)

• What persistent patterns are there in how voters have thought about science/technology and safety/regulation? What’s the best things written on fears of automation and the cycle of such stories? A fundamental argument on AI now is a) ‘practically all fears and demands for regulation now are the same cycle we always see and almost all of this proves to be wrong and counterproductive and simply a way for those with power threatened by the new technology to defend themselves’ versus b) ‘superhuman capabilities really are “different this time” and we need to find ways for government to try to regulate better than it has ever regulated anything’. I’ve strongly recommended reading von Neumann’s essay Can We Survive Technology? for years and I think core ideas are relevant to this problem. (If like me you are worried about things like democratising bio-weapons capability and the next generation of AI/drones applied to war, you should also be thinking about how to end the dumb Ukraine war and de-escalate conflict over Taiwan and seek a new version of the post-1815 Concert of Powers, rather than continue current escalatory dynamics that our regimes are obviously incapable of handling.)

An RV Jones of today is likely to be working at the intersection of such questions.

Please leave links to documents you think could help figure out good ideas on these issues.

Separately, please post links to the best-vastly-underrated sources of analysis on the US election, public opinion, election models etc. Where is the best data on what question voters think the election is fundamentally about? Who is publicly tracking — this election is really about Biden’s senility / Trump’s criminality / the border / democracy etc?

A paper (Social class and wise reasoning about interpersonal conflicts across regions, persons and situations, 2017) suggests evidence that people from the lowest social class (therefore on average less educated and lower IQ) are more likely to use ‘wise reasoning’ than highest social class (more educated, higher IQ). Does anybody know the status of such claims given how little in social science stands up to scrutiny/replication? (I’m inclined to believe the result based on my own observations but…)

A paper (Individuals with greater science literacy and education have more polarized beliefs on controversial science topics, 2017) suggests that more knowledgeable people use scientific knowledge to defend positions that have become politically polarised (rather than use their knowledge to challenge their own beliefs). Does anybody know the status of such claims? (I’m very inclined to believe these results!)

A paper (Ideology, motivated reasoning, and cognitive reflection, Kahan 2013) suggests that those who scored highest in cognitive reflection were the most likely to display ideologically motivated cognition (which promotes individuals’ interests in forming and maintaining beliefs that signal loyalty to in-groups): i.e the disposition to engage in conscious and effortful System 2 information processing ‘actually magnifies the impact of motivated reasoning’ to signal loyalty. Other studies suggest that if looking at issues that have not been polarised politically (e.g skin rash treatment), people with better quant skills do better but if the issue has been polarised those people are even more polarised. Does anybody know the status of such claims? (I’m very inclined to believe these results!)

I wrote in November that Netanyahu is a disaster and the sooner he’s replaced the better. I’ve never followed Israeli politics in any detail. Does anybody know of particularly interesting work on predicting Israeli elections? (E.g This appears to have a particularly tricky problem — that there is a threshold between 3-4% that is very important for seats won in a tight election but is well within the margin of error in normal polls.)

Thanks for subscribing and if you’re not, you should — you’d have been way ahead of the game on Ukraine, Sunak’s collapse, Biden’s senility and Trump being ahead in the electoral college contra mainstream predictions since 2020, AI going mainstream in 2023, the covid inquiry farce, and much much more, £100 a year is a massive bargain…!