Snippets 10: Trolley/covid, AI/politics & summoning the demon, bank runs/Eurozone, AMA

Do 'the clever beasts have to die'?

On Wednesday the Trolley will answer questions about parties and covid.

I’m in the middle of writing my own statement to the official inquiry and therefore reading a lot about covid again.

So I’ll watch and post thoughts on how he tries to lie his way to safety. (If you’re one of those on top of the details and part of the network that pushed him out, text and I’ll post.)

Generally I think SW1 is overrating the chances of him returning this year even if he manages to escape this inquiry.

Why?

Unless Sunak gets fed up and walks away, he can use the vast trove of material in PET (the part of the Cabinet Office that deals with scandals) to smash the Trolley up. Much remains unpublished. And remember that useless Lord Geidt didn’t even bother investigating all sorts (he didn’t interview key people who actually knew what was going on). So if Sunak’s team is crashing, there’ll be people in No10 who’ll think ‘we may be doomed but we’ll finish the trolley off’. And spads who’d relish it will be helped by officials who don’t want the trolley smashing around again as they prepare for Starmer.

So although a lot of hacks like writing stories about his return, I strongly suspect his blunders and scandals have finally been too big for him to recover from. It’ll be much easier to keep him out than it was to get him out. He only got in in the first place because the entire system had paralysed itself and Tory MPs thought they were headed for Corbyn as PM by Christmas 2019 and the end of the Tory Party.

Please post questions on anything and I’ll answer over the next few days and through Wednesday afternoon as I live-blog…

For subscribers, AI and bank runs below…

COVID, TROLLEY (free to all, as promised in 2021)

There’s many things that could be said about the parties. E.g why did neither Sue Gray nor the cops investigate the ABBA party?

But here’s a neglected one:

• a number of junior women were told to attend the BYOB event summer 2020 by the PM’s PPS who a) was responsible for checking the rules with the covid taskforce and b) told officials it was a ‘work event’;

• they attended;

• some were subsequently fined for attending (but not senior people);

• the PPS and the PM were told by me and Lee Cain earlier that day the event seemed clearly outside the rules and shouldn’t happen;

• they ignored this;

• the PM was NOT fined for attending, even though it was his decision to hold the event, but the junior women, who reasonably thought ‘the covid taskforce has approved this event’*, were fined.

It’s indefensible. You expect such behaviour from the Trolley. But how do senior Cabinet Office officials defend letting this happen?

There are many such episodes in this sorry saga. The civil service talks a lot about being a great employer, and its ‘values’, but the truth is it repeatedly treats junior people shamefully, cannot keep young talent, and has many people in senior management with no talent or moral courage. They are the people who’ve survived the moral maze of a rotten HR and promotion system, with the ‘values’ and priorities you’d expect.

Andy Grove said to ignore what a company says about its ‘values’ and watch what it does — who it promotes and lets go. This tells you all you need to know about Whitehall. Promotion for failure. Gongs for disaster. And great young talent flooding out. Generally those who gave the worst advice in spring 2020 — closing borders is racist, the public won’t accept lockdown, masks are bad for you, no alternative to herd immunity in a single peak by September — have got gongs and promotions. Those who were proved right have almost all left. The system is working as intended.

(* NB. No10 was in an odd legal situation. E.g it was sometimes used as a test area that meant it was exempt from some rules (e.g for rapid tests), the classified nature of some work also meant some rules did not apply etc. So the rules WERE different for No10 staff than for the general public at work. The media keeps saying ‘did they think they were different’ but No10 staff WERE legally different. This is relevant because junior staff were told that the PPS was responsible for checking that No10 was sticking to those rules it had to stick to, which changed and were not identical with the normal workplace rules. This obviously does not mean some of the pissups in 2021 were OK but it does mean that if the PM’s PPS said ‘come to this work event’ junior staff were entitled to assume that the event was lawful. And entitled to think that them getting fined while the PM was not is totally unfair. And that their bosses failed to stick up for fair process. And that the Gray inquiry let the PM and senior officials off the hook.

PS. Many media reported that I’d been questioned by police and/or fined. 1/ I’ve never had any penalty of any kind over covid rules/laws. 2/ I have never been questioned by the police regarding covid rules including by the police investigating parties. I wasn’t even sent a questionnaire unlike other staff in No10.)

---

Tues, 12:26.

The Times reports that ‘Boris Johnson rejects a claim by Dominic Cummings that the No 10 garden party was against the rules’. This is not just obviously false, it’s further misinformation from him. Officials were fined therefore the cops concluded it was against the rules, as Cain and I warned that morning and which is referred to in emails given to Sue Gray. My opinion is irrelevant to whether it was / not against the rules.

Re the BYOB event remember:

• a) when I said last year the PM’s PPS was warned it was against the rules, the PM sent out No10 press office to say I was lying; b) then an email was published showing what I’d said was true and they had to correct the record.

• b) the morning of the BYOB event was a particularly chaotic day as that morning, before the BYOB invite went out, the PM and I were arguing furiously over a) his totally botched handling of the Cabinet Secretary which had led to the latter reasonably furious and threatening resignation, b) me demanding that his PPS be replaced as he clearly could not do the job properly (having contributed to the botched handling of the Cabinet Secretary among many other things), a widespread view across No10 and the Cabinet Office. ‘He may be a bit useless but he’s my loyal labrador, I don’t want you replacing him with someone who does what YOU want!’, said the PM — a clue as to why we couldn’t get No10 running professionally!

Wed, 1000

Trolley / ERG / new EU deal.

1/ Is the new deal perfect? 

Obviously not. But it’s a substantial improvement and the EU has accepted breaches of its principles that Remain-iacs repeatedly predicted would never happen. 

2/ Why is Trolley opposing it and supporting the ERG Bill? 

Simple. His ‘strategy’ is to bring his No10 management style to the fore — his only hope is CHAOS. Sunak winning a small majority is a disaster for him. Sunak throwing the towel in because the Party collapses in chaos in 2023 gives him his only chance of riding around Chequers on his motorbike again, the real point of being PM for the Trolley. 

3/ What about the ERG’s Bill? 

As you’d expect from the ERG it’s a farce.

Their whole pitch is ‘no ECJ’ yet their own dumb Bill enshrines the power of the ECJ and creates new powers!

If you’re new to British politics this may sound too dumb to be true but you haven’t met Bill Cash.

When we did the UKIM Bill in 2020, we had VERY SPECIFIC ‘notwithstanding’ clauses. We knew specific problems we wanted to address and were clear publicly and to the EU about our goals. Our extensive enemies in the legal profession had to admit defeat. 

It worked. The EU engaged and we made progress. (The Trolley then collapsed in December undoing some of the progress but still…)

The Trolley’s NIP bill doesn’t have anything specific to solve specific problems and actually *creates powers for things to be referred to the ECJ*. It was useless as a negotiating tactic.

Probably Boris doesn’t even realise that the Bill he supports creates powers for the ECJ. If he does, or it’s pointed out to him, he’d just laugh — it’s irrelevant, the point is to cause chaos for Sunak.

No10 made a mistake leaving the vote so long. They should have done what we did in 2019 — create a nonsense Potemkin clause called ‘the Bill Cash sovereignty clause’, let Cash et al walk up Downing Street for the cameras, then ram it through pronto. ‘With pirates, a pirate-and-a-half…’

Keep a note of Tory MPs who argue for the ERG Bill ‘because it removes the power of the ECJ’: they’re either fools who don’t understand what it says or they’re lying.

4/ When you see the Remain-iac lawyers and ex-officials jabbering about ‘respect for the rule of law’, remember an important fact.

In Britain the traditional approach (for centuries) has been that a) ministers must obey DOMESTIC law passed properly by Parliament (‘obey the rule of law’) but b) INTERNATIONAL law is not domestic law, is not enforced in our courts, and there is no such legal/constitutional duty on ministers to obey it.

Indeed there is often a duty on ministers to disobey it. Remember that, for example, we have intelligence services whose job is to break ‘international law’. Any serious country has a subtle approach to international law, taking seriously most obligations but sometimes putting significant interests ahead of the letter of international law, sometimes making clear they will not conform, sometimes demanding renegotiation etc.

(Ironically whenever the EU itself does this, the EU’s legal fans in the UK maintain a respectful silence! E.g. the ECJ’s own attitude to the powers of the Strasbourg Court is instructive.)

There has been a long and successful campaign by officials and lawyers to elide the distinction between domestic and international law.

This came to the foreground in 2020 when J Jones, top solicitor in the Cabinet Office and bitter opponent of Brexit, resigned over UKIM.

The spin across the media was — it’s outrageous for officials to be asked to ‘undermine the rule of law’.

But we were changing DOMESTIC law in a normal legal way. Under the traditional approach, officials like J Jones would have knuckled under without a squawk.

What Jones and others want is for officials to be allowed to decide not to do things if they interpret international law in a certain way (or if Britain’s enemies/opponents do!).

This is a revolutionary doctrine hiding as a conservative one. It’s developed because many officials and lawyers see themselves as having a ‘higher moral duty’ to their own ideas of internationalism than to Britain. That’s ok for a private citizen, but you shouldn’t be an official representing Britain and responsible for taxpayers’ money.

Because most of the media and MPs don’t understand the constitution and/or agree with the campaign, the campaign has been successful.

You can see the lack of understanding of the constitution constantly in ECHR/HRA stories.

No10 were given two options. A) Legislation that doesn’t fix the problems with the ECHR/HRA. B) Legislation that does. They chose (A) and added lots of spin about a ‘super-tough option taking on the ECHR’. No10 under Cameron, May, Johnson (after I left), and now Sunak has pulled this trick to fool Tory MPs, Telegraph etc. It’s worked so far.

The new legislation does not and cannot ‘take back control’, as No10 span. It leaves the Strasbourg Court and our obligations to respect its views in exactly the same position. And it won’t and can’t ‘stop the boats’.

As I’ve pointed out repeatedly, ignored by the media because it doesn’t suit either side’s story, the legal advice to No10 in 2020 was crystal clear: the boats problems are unsolvable without changing the whole edifice of HRA/ECHR and judicial review.

MPs would rather let the boats keep coming, and let torturers and sex criminals out of jail early to do the same again, than deal with the actual problems of the HRA, ECHR and evolution of judicial review. (Strasbourg has interpreted ‘human rights’ to mean we can’t deport sex criminals if they claim ‘err I’m gay and face the chop back in the Congo’ — our MPs our fine with all this but then, disgracefully, blame the judges when the judges say ‘MPs have chosen to respect the Strasbourg Court so…’.)

The above is inconvenient for the trolley, the ERG, Remain and both parties. You will not see these issues properly discussed.

1025, Simon Case evidence

Case’s evidence (just published) shows that as I said at the time the PM misled the Commons. The most senior official, deeply involved in everything to do with covid and investigations into parties, is explicit that he did NOT tell the PM that guidance and rules were always followed.

Further, he also misled the Commons in his written evidence submitted yesterday.

1110, Trolley case wrecked already

The Committee has published new evidence.

1/ Boris’ claim now is that his statements to the Commons — which he now admits were false — were what he was advised to say at the time by his ‘most senior trusted officials’ so it wasn’t his fault they turned out to be false.

2/ The three most senior trusted officials were: the Cabinet Secretary, the PPS, the Communications Director.

3/ The Commons has published their evidence. All three have denied they gave him that advice.

His defence is wrecked.

[1145, off topic, what’s the best English account of why Austria decided to go against Russia before the Crimean War, it had such consequences and seems so foolish…?]

1215 Further to the point (top) about how disgracefully young women were treated

A. The Trolley’s legal bills are all getting paid by the taxpayer but…

B. The young secretaries who were TOLD to attend ‘work events’ were then told by senior managers that they would NOT get legal advice ‘because they’d broken the law’!

C. They were also told they had to answer Sue Gray’s questions (without legal advice) or face consequences. Then duff/distorted info from their interviews was handed over by the CABOFF to the cops! E.g wrong names for people at parties, accusations of people puking that were false, mistaken identity etc etc.

D. Meanwhile the senior guys were told ‘no comment and the cops can’t do anything’!

E. Also remember Sue Gray handed the ABBA party over to the cops and the cops then said it wasn’t for them, so it was NEVER investigated! Classic Yes Minister.

The whole thing is disgraceful even by the standards of the Cabinet Office and the senior Cabinet Office officials responsible would, if there is any justice, be fired and disgraced.

What will happen?

They’ll get gongs, obviously!

(Ps. In 2014 I wrote about Sue Gray and the Cabinet Office HERE. Tories never learn.)

[Off-topic. NB. We are now sending depleted uranium shells to UKR. Normally the use of these weapons is a big news story. But it isn’t now obviously because most of the media including the BBC is part of the campaign for war.]

NEW READERS

Check out:

What our MPs don’t know about nuclear war that could us all killed

Tolstoy and politics

Reading List

SUBSCRIBE AND BE AHEAD OF THE GAME!

---

1423

He says the photos do not show him at unlawful parties. But the photos DO show him at events which more junior people than him were FINED for being at, i.e the cops decided were unlawful.

He says I’m lying.

But when I blogged about the BYOB event last year he sent out No10 officials to say I was lying then. And what happened? Emails turned up showing what I’d said was true (warnings that the event was a mistake) and it was No10 which had to admit it had not told hacks the truth. Over and over what I’ve said has been proved right while the No10 story has collapsed.

1431

Reason why he’s stressing me is simple — his rat brain tells him that the MPs hate me about as much as they hate him, maybe more!, so he wants them thinking ‘do I want to do what Cummings wants and do the trolley in?’

1505

You can see why we had to keep Bernard out of running Vote Leave — confused, missing the point, HOPELESS!

1555

Re the BYOB he says if I/Cain had had concerns we’d have raised them with the PPS and it would not have happened.

We did raise concerns!

I spoke to PPS and to the PM.

And Cain sent an email then spoke to PPS about it.

PPS said he’d speak to the PM (and now says he can’t remember if he did).

He’s said twice that he had to speak to staff that day because the Cabinet Secretary had resigned that morning.

He had NOT resigned! He’s misleading the Committee again.

But the PM was arguing with me very violently about this shambles — the PM’s incompetence had led to the Cabinet Secretary THREATENING to resign that day but we PREVENTED it, no thanks to the trolley.

1720

Huge waste of time but boils down to —

A/ His whole defence rested on claims that he was repeating to the Commons assurances he’d been given by senior officials.

B/ But the three most senior officials have said they did NOT give him the assurance that rules and guidance had been ‘followed at all times’. When asked who gave him this assurance he says he doesn’t want to name them but they exist.

He’s trapped by his own lies and knows it.

Over and out on this for now…

---

Super-intelligence, ‘AI alignment’ and politics: power, power, power

Man is a rope fastened between animal and Superman — a rope over an abyss… What is great in man is that he is a bridge and not a goal. Nietzsche

The German nation is sick of principles and doctrines, of literary greatness and a theoretical existence. What it wants is power, power, power! And whoever gives it power, to him it will give honour, more honour that he can imagine. Julius Fröbel, revolutionary

It is necessary to be extremely careful. To get rid of a rival, he will do things of which it would be absurd to suspect any other statesman in Europe. Lord Salisbury on Bismarck

In some remote corner of the universe, poured out and glittering in innumerable solar systems, there once was a star on which clever beasts invented knowledge. That was the haughtiest and most mendacious minute of ‘world history’ — yet only a minute. After nature had drawn a few breaths the star grew cold, and the clever beasts had to die. Nietzsche

Prussia, 1862: summoning the demon, ‘Otto’…

There is an escalating constitutional crisis between the Prussian king and liberals. A player (Roon) realises his team (the king’s) is on the brink of defeat and complains sarcastically, ‘We lack a mere trifle at this stage and that is the brains of the ministry’. He has a friend who he thinks is a Super-intelligence. He thinks it’s also aligned to his team’s goals. He persuades the king to give it power so they win the conflict. Despite the king’s fears that the Super-intelligence, known in elite circles as ‘the mad Junker’, is prone to dangerous spasms, the situation is so desperate that the king eventually rolls the dice.

Periculum in mora, they telegram to Paris. The Super-intelligence gets the train to Berlin. It takes over formal power in one of Europe’s five Great Powers.

In retrospect we see the world’s timeline dramatically forked — humanity’s future has been fundamentally different because of that one decision in chaos in 1862. But at the time…

The specialist academic community and the lawyers are near-unanimous: Super-intelligence is impossible, this agent is definitely not superhuman or even dangerous, it’s laughably ‘shallow’, it ‘hallucinates’, and it will be aligned, easily constrained by safety features, and, if necessary, switched off. Many a learned article by professors, many a newspaper column by pundits, mocks the Super-intelligence and predicts its rapid humiliation and failure. The odd ‘maverick’ warns ‘you’re making a fatal error, I beg you to reconsider’ but is dismissed by the experts.

The Super-intelligence quickly escapes control. They had one chance to ensure it was aligned. It is not. It has its own goals that it kept partly secret and lied about and which only partly overlap with those who gave it power. It defines its fundamental goal as: the expansion of its own power and freedom to manoeuvre, and the elimination of threats to constrain it, to box it in, or ‘switch it off’. It programs its own hierarchy of secondary priorities. It does indeed beat the liberals and save the king, as the king originally wanted, but it does much, much more.

It has been trained on data to understand human morality, human weaknesses, normal political rules, and the diplomatic system but does not feel obliged to stick to any rules. It observes the behaviour of others and says openly that it will operate on the basis of ‘with a pirate a pirate-and-a-half’. When pirates think it’s cornered, it’s heard to quote Virgil darkly, Flectere si nequeo superos, Acheronta movebo — if I can’t bend heaven, I’ll move hell. And it does.

Those who thought it was programmed to ‘never undermine legitimate authority’ were wrong. Though it tells legitimate authorities that its highest purpose is to serve them, it jokes privately about ‘the sovereignty swindle’ of those authorities (the princes). When convenient it deploys what it refers to contemptuously as ‘the national swindle’ and ‘the democracy swindle’ to mobilise support and isolate enemies, while telling the rulers to give it more power so ‘I can protect you from the socialists’. It mocks the academics who write their books and give their speeches while twisting their output to advance its own power. Some of these academics end up being secretly funded by it to parrot its propaganda.

Its behaviour is baffling. It seems to be playing a different game to other players. It delves deep, deeper into details than any of its opponents. It zooms out, out to very abstract pictures. It implements operational plans, connecting the deep details and its very abstract pictures, of a subtlety far beyond the comprehension of its competitors. It can outthink famous scholars and it can operate in the gutter of international politics and diplomacy with its espionage, bribes, lies, honeytraps, memes and manipulation. It can charm the most glittering aristocratic parties, blackmail the blackmailers, terrorise the terrorists and build the world’s most sophisticated international media manipulation machine. It tricks diplomats into sketching plans and leaving them behind then it leaks the documents to the media in foreign countries at just the right time to preempt its enemies’ coordination and ensure other countries stay out of conflicts it is provoking. Diplomats rash enough to make idle threats have their bluff called: when Britain threatened to send its army, it’s told, ‘if Lord Palmerston sends the British army to Germany, I shall have the police arrest them’.

Some players describe it as ‘incompetent’, ‘incoherent’, ‘reckless’, ‘mad’ and predict it will soon blow up or be switched off. Others describe it increasingly as ‘diabolical’, ‘monstrous’, ‘amoral’, ‘terrifying’, ‘the devil himself’, ‘a demon’, ‘un barbare de genie’, ‘le diable’. Attempts to constrain it fail. Serious attempts end in isolation, disgrace, death/suicide.

It connects itself to more and more sources of real power and disconnects its opponents. There are five Great Powers in shifting coalitions. It repeatedly coordinates at least two of the others to work with it and isolate the other one or two, then, after achieving its goals, ditches its allies and reconfigures a new coalition. It repeats the trick. Even when they realise it keeps tricking them, they can’t escape the game. Constitutions across Germany, Austria, Hungary, and France are rewritten amid the chaos it creates and the armies it sends forth which win shockingly fast victories contrary to expert military opinion. The most powerful country, Britain, is manipulated into staying neutral and Whitehall even writes memos (not inherently stupid but in retrospect wrong) reassuring itself that the Super-intelligence’s progress is ‘really in our interests’ (cf. Whitehall response to Sadowa, 1866). 

Safety features such as ‘constitutions’ and ‘international law’ fail. It reprograms these safety features to increase its own power, such as by writing a new constitution to create a unique role giving itself vastly more power in the power network. It manipulates international law to drag opponents into quagmires: ‘I have never judged international disputes by the standards which prevail at a student duel.’ Intelligence services are deployed by its enemies to disrupt and destroy its thinking and operations: they fail and are manipulated to undermine their own side. It steals resources (e.g ‘the Reptile Fund’) for its sole use, outside constitutional restraints, which it deploys to support its own international espionage and propaganda network and thereby expand its power and eliminate threats. When this is discovered, it tells Parliament brazenly, and threateningly, ‘I was not born to be a spy, that is not my nature, but we deserve your thanks if we follow these malicious reptiles into their holes to see what they are doing.’

Players become increasingly terrified. But all attempts to align the Super-intelligence more closely with the interests of other players, or notions of ‘humanity’s interests’, fail. More and more scream ‘just turn it off’. Those who try meet a grisly fate. Assassination attempts fail. Rational players decide it’s dangerous to thwart it and suicidal to try to switch it off, so instead they try to cut deals with it. Its power grows further. Their deals often evaporate — they think they’ve got agreement on X but no, they don’t, and complaining about ‘lies’ doesn’t work. Everybody says ‘it lies’ and ‘it can’t be trusted’ but it doesn’t matter — everyone keeps trying to make deals even though they know they’re not seeing the game board properly. What else can they do?

Some of its friends beg, how can you make deals with our enemies?

You can’t play chess when sixteen of the sixty-four squares are forbidden to you by your own side.

Some of its friends beg, what happened to your principles?

One clings to principles only for so long as they are not put to the test. When that happens one throws them away as the peasant does his slippers and walks after the fashion that nature intended…

The statesman is like a wayfarer in the forest who knows in which direction he is walking but not at what point he will emerge from the trees. Just like him the statesman must take the negotiable paths if he is not to lose his way… Going through life with principles is like walking through a thick forest with a long pole held between one’s teeth.

When in 1878 it assembles an international conference in Berlin, it operates like no other such conference. The other players watch agog as it pours champagne for breakfast while pushing everybody to exhaustion, the pace unprecedented. You won’t die from hard work, it tells them contemptuously. Simultaneously it masterminds an election campaign. Both consolidate its own power.

Finally it ages and its performance degrades. A plot to switch it off finally works. But by now it’s already transformed the world and made it much, much more dangerous. What was the least powerful of the five Powers in 1862, Prussia, now has the world’s most effective armed forces (which are constitutionally entirely controlled by the appalling and deranged Wilhelm II), a dynamic economy, it’s a world centre of education and leads the world in critical areas of science and technology including technologies with huge military importance.

After the Super-intelligence was disconnected from power but before it died, it said that in twenty years, what it had built would collapse. Other players took over the power network it created but they didn’t understand it. They soon ran amok, disconnecting themselves from sources of power (e.g binning the Reinsurance Treaty) and antagonising everybody. Having created a hostile coalition encircling themselves, fear, desperation, greed, stupidity and chaos drove them to start a war over ‘the passions of sheep-stealers’. (In the chaos some players even semi-regret the Super-intelligence was replaced — yes it was terrifying but at least it was rational with some coherent, if largely impenetrable, utility function, it was diabolical but not insane, you could at least negotiate surrender! And after all, its horrific self-interest was somewhat reassuring — it didn’t engage in random violence and it dismissed the idea of tying Europe’s fate to conflicts in the Balkans: I will never tie our fate ‘to the passions of sheep-stealers’, it told everyone sarcastically. Its intelligence wasn’t entirely orthogonal to its goals, it was far from a paperclip maximiser!)

Twenty years after the Super-intelligence’s prediction, almost to the month, the system collapsed. The dynamics spark Lenin, Stalin, Hitler, Mao, nuclear weapons, the whole 20th Century dominated by two wars and the world that emerged from these wars…

[Ps. I originally described Bismarck as ‘AGI’ then changed to ‘Super-intelligence’ to avoid the confusion from that term, in the email I missed a couple of examples but have edited to make it consistent.]

A. Intelligence is inherently dangerous when married with acute psychological intuition, the extreme epistemological humility Bismarck displayed regarding long-term planning plus extreme talent for adaptation on the fly, extreme operational ability, extremely relentless efforts moving extremely fast with few if any moral scruples.

B. Safety features created for normal people break down when dealing with extreme intelligence plus these other characteristics. There’s no known solution for humans. Extremely high political performance inherently attempts to avoid constraints, just as the top chess players (human and AI) inherently avoid constraints (in chess this tendency can be quantified).

C. Artificial systems trained on the internet will approximate Bismarck skills with greater fidelity over time.

What now?

There’s suddenly a lot of discussion about AI in the mainstream political world.

I’ve followed the AI debate since 2000 when I read Bill Joy’s Why the Future Doesn’t Need Us. When I went to No10 in July 2019, I wore a T-shirt I’d got from a visit to OpenAI to spark some reflection on the intersection of politics and technology. In January 2020 I started building a data science team for the PM’s office (at the time No10 did not even have a cloud service and the PM’s office had worse tools than the average undergraduate).

2022 seemed a bit like the AI equivalent of February-March 2020 on covid — a time when if you were talking to the right people you could see a tsunami coming to smash politics-world but when you looked at politics-world they were carrying on as normal.

I wrote last autumn after returning from the US that the next generation of models would push the political world into a new equilibrium on AI.

After OpenAI published the original GPT much of academia expressed scepticism that ‘just scaling it up’ would work. Wrong. Sam Altman bet on the scaling hypothesis, raised the money to test the theory, and did GPT3. And even after GPT3 the same people said that scaling would have to break down. Maybe. I wouldn’t bet on it. Having spent time in San Francisco recently talking to people building the next versions of these models, I’m confident that the political and government world will get further big shocks over the next year.

Chat-GPT broke through into politics-world and GPT4 has given it a further push.

There’s suddenly a lot of discussions in the Deep State, here and DC, about what’s coming.

Some important dynamics at the apex of power:

1. The models in the labs of the top companies, unreleased, are more powerful and interesting than what’s public. The progress is steeply exponential, maybe these models will improve by 10X in a year. Huge new training runs are planned for 2023-25, billions are being raised fast, GPUs are being bought. Within 3 years there could be models orders of magnitude more powerful than GPT4. And super-powerful models now controlled by a handful of companies will a) get compressed and appear on smartphones and b) get opensourced, both much faster than politics-world expects.

2. Remember how bad most in politics are at using TV and polling which have been around since the 1950s. The idea our MPs will quickly understand and have good plans on AI is laughable.

3. We have regressed on nuclear thinking since the 1990s despite the importance of understanding safety, strategy, crises. We have decades of experience in this, invested vast amounts, and it’s embarrassing how bad we now are even though we’re in a war with nuclear threats being made. Much of the British nuclear enterprise is in disastrous condition, highly classified partly to protect amazing blunders.

4. London and DC have got themselves into a war of attrition but a) can’t reform their own procurement and industrial production to supply this war of attrition and b) have pushed the world’s biggest manufacturer into alliance with Russia. How likely is it that the Whitehall responsible for AJAX (which I got agreement to scrap in Q3 2020 but which was undone, like much else, immediately after I left) is suddenly going to do GPU procurement super-fast and smart?

5. Remember that No10 cannot do anything fast in normal times (except promise more spending). In spring 2020 even when thousands were dying now, much of Whitehall still optimised for business as usual. Thousands are not dying yet from AI. People have been briefing Whitehall including No10 for months. They can’t move. The usual vetocracy holds everything up. Whitehall could not act fast on AI unless, at the minimum, a) the PM made clear it was a top personal priority (this means a lot more than him just saying it) and b) there’s new money. Without these two basics, little serious will or can happen. The PM is presiding over multiple crises simultaneously with a broken No10-CABOFF system, a joke Cabinet, a rabble for a party etc. How likely is real PM focus? The only partial way through is if he does something analogous to 2020 with vaccines and hires someone who reports to the PM and the PM makes clear that people better do what they say. Even then, just as Hancock interfered with vaccines, other parts of the high friction system will naturally slow it down. Everything in Whitehall operates to stop you learning from Steve Jobs: Focusing is about saying ‘no’, you’ve got to say ‘no, no, no’ and when you say ‘no’, you piss off people.

6. Remember how little top technical talent there is in Whitehall. Almost all the top people in this field are now in companies. Many have fled public service in the last 2-3 years of chaos including critical people in GCHQ. So there’s a profound lack of expertise in the deep state even to procure what’s needed, never mind build it themselves. You often see on Twitter otherwise sensible people saying things like ‘surely the government will have a secret program all over this’. No! Unsurprisingly therefore, when the government tried to announce a ‘world-leading’ ambition a couple of weeks ago it announced it might buy … 3,000 GPUs, about 30X less than a serious goal. And default plans are to get the Turing Institute to work on it — the Institute that is seen by serious AI players as actively damaging to AI in the UK. Remember that seven years after the referendum, the Tories haven’t been able even to make officials get rid of the dumb cookie popup. (Top universities have embarrassed themselves on AI research. Many professors responded to GPT and similar breakthroughs by DeepMind with contemptuous comments that ‘just scaling neural networks won’t work’. When I wrote things like this in 2019 many supposed experts told me not to listen to what these labs were saying. Gwern has repeatedly pointed out how bad academia has been at predicting AI.)

7. Remember how we’ve regulated gain-of-function research and how regulators like the FDA/CDC scuppered rapid testing in covid despite huge global pressure for solutions. Look at how Brussels has done GDPR and plans to regulate Bayes Theorem. Do you think these entities will be able to cope with novel and extremely fast moving technology driven by some of the most aggressive and competent people in the world? Or do you think they could easily do dumb things that adds friction for those aiming for positive goals (e.g better drugs) while doing nothing to guard against actual (poorly understood) dangers?

8. Remember that fears of China provide an incentive for some in politics-world to accelerate technology development and worry about consequences later.

9. Remember the limits for an open society in controlling general purpose technologies like electricity and AI.

10. Remember that while the West congratulated itself on a deal with the Soviet Union on bioweapons, the Soviets cheated and built a massive secret bioweapon infrastructure that was undetected until after 1991 (then some of it vanished). International agreements are leaky at best and at the moment NATO is busy worsening relations with the most important non-west country on AI, China.

11. Remember that key terminology has become politicised. Five years ago, ‘AI alignment’ was a very niche subject with a clear technical meaning. I never heard politicians mention it. Now that AI has suddenly gone mainstream and LLMs have provoked political arguments, ‘alignment’ and ‘AI safety’ have acquired different interpretations. ‘Alignment’ is heard by some to mean ‘does this model align with the acceptable politics of the NYT/AOC’? Inevitably there is a backlash with people saying (I think reasonably) ‘I don’t want technology trying to align me with AOC’s politics thanks!’ There is, to put it mildly, an extreme difference between a) ‘AI safety’ in the sense of ‘chatbots not offending BLM activists’, b) ‘AI safety’ in the sense of ‘preventing thousands/millions dead’ and c) ‘AI safety’ in the sense of ‘lights out for everyone’ but the term is now used to mean each of these things. Further ‘AGI’ remains a poorly defined term and experts might be arguing about whether it has been achieved at the moment their conference gets vapourised by an AI system some of them said was ‘narrow’ and some of them said was ‘general’. Progress demands new language including for ‘capabilities that can be very dangerous and should be monitored even if the AI is not agreed to be AGI by all the experts’.

12. Remember how the ‘Putin-ninjas used AI-Facebook to tip Brexit and Trump elections’ meme spread like wildfire through elite networks. The vast majority of professors at elite universities and hacks at elite old media didn’t understand Facebook, advertising, or AI but they had an easy to understand excuse for what happened. They didn’t need to admit errors. They could explain Brexit and Trump as the product of a malign conspiracy that fooled dumb voters but which they understood and which justified more power for them — perfect! Large fractions of the old elites still believe this meme and it’s significant in the UKR war. And remember how ‘misinformation’ became a slogan under which people spread misinformation: e.g. ‘Hunter Biden’s laptop is Russian misinformation’ says CIA, reported by CNN. So imagine how they will respond when AI really does upend elections which will soon happen! And imagine how little those professors and hacks will understand the underlying technology. In 2024, it won’t be used in the UK by the old parties but I think very likely will be in the US. (You can already see some of the most competent political teams hiring LLM specialists.) It was clear from experiments I did last year (unpublished) that these LLMs can improve election models and improve prediction of ‘who are the true swing voters’. They will be used to improve communication in ways that will seem dystopian, or praised as ‘saving democracy’ depending on one’s perspective! Market research companies and advertising/marketing companies will also be totally disrupted. The ‘wilderness of mirrors’ can only get more confusing. (PS. In 2016 we used AI to improve polling, not Facebook marketing, but this fact was ignored in the media storm.)

13. Remember that what’s actually happening in the top three labs (Anthropic, DeepMind, OpenAI) is very little known anywhere in the ‘corridors of power’. They haven’t paid attention. Haven’t prioritised. Haven’t built networks. The few technical people whose job it is and do care have not been reinforced. And those in charge of those labs have mostly NOT told them what they really think. And many of the smartest technical people are extremely naive about politics (von Neumann and Grothendieck were perhaps the smartest people of the last century but they varied enormously in their grasp of politics and government). Remember that we created a data science team for the PM’s office in 2020 which did some great work and was intended to help No10 deal with AI generally but it was marginalised by Boris then moved out of the PM’s office by Truss on her first day on the job — i.e active vandalism of the PM’s office’s power by two PMs.  

14. Remember that ‘the Manhattan Project’ is a go-to metaphor in the political world but the realities of the Project are extremely poorly understood by politicians and many senior officials. So ‘Manhattan Project for safe AGI’ and similar calls are widespread but often misguided. The details matter enormously.

15. Remember that the security at places like Google is, though good relative to other companies and startups, very vulnerable to sophisticated intelligence agencies who can deploy illegal methods like burglary, blackmail, honey traps etc. The original Manhattan Project was penetrated by the Soviets. The counterintelligence branch chief for Soviet operations of the CIA, Aldrich Ames, [NB. edited from original after checking, this is how he’s described by the Senate Committee] was a Soviet mole. Nuclear security is repeatedly broken. Real security is extremely hard even in well understood areas. It is even harder for labs that have to be open to the world. This means that we should assume that new ideas are quickly seen by hostile states. 

The obvious prediction to make is that when politicians realise their own elections are at the mercy of new technology, they will focus and update strongly. I think this is wrong. In 2016 most preferred to believe media nonsense about digital marketing rather than figure out the truth. I think only a tiny few will update strongly over the next two years (but both US Presidential campaigns will be among them). More politicians will be fooled by misinformation about AI/LLMs, make wrong claims and demand damaging action than will have a reasonable grasp of what’s true or false, which will be hard and changing fast. The old media will be full of professors at elite universities talking garbage about politics and technology and most politicians will believe what they see on TV, they won’t talk to actual experts.

There are people in the deep state who understand the stakes. Officials are being briefed on things that are in the labs but not public that would be extremely big news. Some want to take it seriously. But they are stuck in the usual dysfunctional equilibrium.

Politics-world will shift very fast from a) almost totally ignoring it to b) panic starts bouncing around, people from GCHQ walk into rooms saying ‘I’m afraid that, extraordinary as this sounds…’ Scenes that would seem over the top in a thriller are being discussed by experts as near term possibilities. Having watched politicians’ faces when such things are explained, I’m sure jaws will hit tables.

I would summarise the situation as:

1. Politics-world is about to get another profound shock, in time even bigger than covid or Ukraine.

2. Almost nobody in government has a clue what to do so panic could make things worse.

3. Politics-world and tech world will be shocked by public reaction when people realise what’s been happening and secret elite discussion leaks into mainstream discussion. (I’ve good reason to think this from market research recently in US and here.)

4. There’s a small window for No10 to do useful things. So far they’ve botched it. It’s unlikely they recover. They also have a financial crisis to deal with on top of all the other crises. If No10, e.g, sets up a GPU taskforce to buy urgently ~100,000 GPUs (and new Nvidia H100s coming soon) it will have some leverage. Unlikely to happen.

5. Given how hard the technological and political issues (positive and negative) are around AI, our default mode should be to assume the same pattern of failure we’ve seen on Iraq, financial regulation, covid, Ukraine.

6. Those trying to make things better need more than ‘A Good Plan’. They also need ‘a Jean Monnet-style meta-plan for getting A Good Plan [if we can get one] adopted in dysfunctional politics-world’. This meta-plan has to cope with all the dynamics that pull governments to normal failure even in life-or-death crises.

7. Yes there’ll be AI regulation but we should expect that if this is done like normal safety regulation it will stop a lot of people doing good things and not stop, perhaps even encourage, many bad things. I’m sympathetic both to those saying ‘there’ll have to be regulation’ and those saying ‘it will be a car crash God no’. The general collapse of competence of political elites is a very bad time to have perhaps the biggest ever technical-political problem to solve! Perhaps the urgency of this question will drive the competent people in tech to engage more with politics, instead of retreating to their walled gardens and fish ponds, which will be an unexpected bonus!

It’s tricky to know what to say about ‘what to do’ publicly so I won’t say anything more here for now.

I advise you follow some of the following accounts, all of whom are at the edge of knowledge in different ways:

@sama

@gdb

@ilyasut

@demishassabis

@jackclarkSF

@ch402

@ESYudkowsky

@NPCollapse

@soundboy

@pmarca

@gwern

@MarcWarner10

Some further reading

Death With Dignity, Yudkowsky. Reply by Paul Christiano.

The anti-doom case, Scott Aaronson (quantum computer scientist, now at OpenAI).

Perhaps It Is A Bad Thing That The World's Leading AI Companies Cannot Control Their AIs, Scott Alexander.

Our approach to safety, by OpenAI.

---

Bank runs again, the Eurozone will suffer systemic dishonesty in financial institutions again

‘[It] is an extraordinary example of what happens when you get 1) a dozen people with an average IQ of 160; 2) working in the field in which they collectively have 250 years of experience; 3) operating with a huge percentage of their net worth in the business; 4) employing a ton of leverage.’ Warren Buffett on the collapse of LTCM hedge fund.

Westminster has acted like the Eurozone’s problems — the euro’s institutions, its banks, its debts, the politics of debt and stagnation — were solved after the last crisis. And it acts like the next decade will look like ‘EU success, British failure, lessons of Brexit clear’. I think this conventional wisdom is, as usual, wrong. NB. Westminster conventional wisdom in 1999 was the next decade would shift opinion in the UK towards the euro, the opposite happened.

In September 2022 I wrote:

And [all these other problems are coming] before our Very-Online-Very-Remain Hacks and Academics — blaming Brexit for the Remain-campaigning PM (see below) — get another unpleasant surprise that the Eurozone has not solved its fundamental structural problems, is deeply screwed, and will have its own systemic problems in coming months, exacerbated by its Russia/energy idiocy. It’s already suffering the rise of extremists, as Vote Leave predicted in 2016. This will get worse along with economic stagnation, free movement, crime, terrorism.

And if you look at SW1-twitter today, it’s truly on brand that a) it’s turned out UK pension funds were highly leveraged and vulnerable to blowing up, b) No10/11 were clueless about this crucial fact until some of them called the BoE in desperation, c) SW1-twitter is ignoring exactly how this happened! Only trillions at risk in leveraged positions without the Supposedly Serious People having a Scooby-Doo just like 2007-8. It’s … Brexit’s fault! [Bold added]

In October I wrote:

Bailey is a disaster too. This was another personnel decision I argued with Boris over and lost. I wanted Haldane. The Treasury did not (‘too unorthodox, hard to control’). It was obvious Bailey was a typical establishment dud. The usual pundits have tried their best to talk him up but have now abandoned ship. As I said a week ago, it’s sort of insane the MPs would not have demanded explanations over the sudden revelation of systemic risk posed by pension funds’ leverage — except it’s exactly what you’d expect given SW1’s handling of Iraq, 2008, Brexit, covid, Ukraine, China’s infiltration of British critical infrastructure etc. Before the BoE, Bailey was in charge of financial regulation to stop things like the pension funds gambling with leverage again…

Unwinding zero real rates is going to reveal hidden leverage and hidden insolvency all over the European economy, things will blow up. Maybe much of the financial sector will also get nationalised. The Eurozone institutions will be under fire again…

I think a large fraction of political pundits, who as a set are strongly Remain, have fooled themselves about this too. They are so desperate to think that ‘obviously Brexit was insane, obviously it will be a disaster’ that they do not want to face the weaknesses of Eurozone financial, economic, and political institutions. [Bold added]

I tweeted in October:

A senior FT hack, very pro-Remain and cross about Brexit, replied:

Sums up the motivated blindness of most Remain pundit-world.

Sebastian Mullaby writes:

Total capital buffer in the U.S. banking system: $2.2 trillion.

Total unrealized losses in the system, as calculated in a pair of recent academic papers: between $1.7 trillion and $2 trillion.

In other words, if banks were suddenly forced to liquidate their bond and loan portfolios, the losses would erase between 77 percent and 91 percent of their combined capital cushion. It follows that large numbers of banks are terrifyingly fragile.

And I think the European situation is worse than America.

Banks and other institutions including pension funds and insurers have bought vast amounts of bonds and other assets much of which are now underwater but the truth is hidden by dodgy accounts. As Warren Buffet says:

‘Bold imaginative accounting’, as a CEO once described his deception to me, has become one of the shames of capitalism.

And there are further bombs to go off, for example, commercial real estate and other ponzis (e.g Tether).

In 2015-16, one of the arguments I/Vote Leave made for Brexit was reducing our exposure to future inevitable Eurozone financial crises. It was obvious they had not solved the fundamental design flaws of the euro but had bodged a short-term fix partly by breaking their own laws. (Remember, when the EU breaks the law all those in London so keen to tweet about ‘the rule of law’ defend it.)

At the time, most of SW1 thought the problems fixed if they thought about them at all. If you look back at the FT and Economist over the past decade you’ll see a blankfaced refusal to face the Eurozone’s problems. Those who think we’re headed for a Rejoin referendum have not thought this through.

What will happen?

Obviously I have no confidence the authorities have a clue. My motto is always: assume systemic incompetence and lies (where sometimes things that look like they must be lies are not actual lies because they believe it). I assume the situation is much worse than admitted by the authorities, the authorities are not doing the sort of monitoring of risks that the most competent people would do by default (as the pension canary in the mine showed last October), the chances of banks blowing up and another major panic is greater than conventional wisdom.

If you watch experts over the past few days, they’re saying ‘a lot depends on whether the authorities can maintain confidence with professional communication’. Bailey and the Bank of England are car crash communicators. An underrated fact about the world is that often at the apex of critical institutions a) communication is crucial, b) those in charge have worse instincts about communication than a teenager.

Thinking through the combination of huge debts and politics, it seems to me most likely that over the next few years the west sees a period of sustained inflation. Shifting money from savers to debtors slowly is a normal move in history. The ‘slowly’ bit is important as they try to make it not too shocking and too obvious and therefore politically easier.

What’s the alternative? They keep raising rates, stuff blows up, and they tell everyone ‘there’s going to have to be years of misery for normal families while we pay off the debts’ — the same normal families who’ve barely had a pay rise since 2008 while watching the millionaires and billionaires make new fortunes from zero real rates?

I don’t think so. And I don’t think they could pull it off even if they tried to. At the moment I think they’re telling themselves that ‘Putin will fail in Ukraine’ and just suck it up, inflation will fall, Biden will win, and politics will ‘go back to normal’ like the lovely 1990s.

So, without reasoning it out in a ‘grand plan’ but just fumbling into it by avoiding certain things painful for them short-term, I think they’ll default to letting inflation run higher than they thought they would (and think they will today). This will play out against the background of Trump v DeSantis and a President who keeps referring to Ukraine as ‘Fallujah’.

It will also play out against the developments in AI sketched above!

---

People systematically overlook subtractive changes, 2021

Here we show that people systematically default to searching for additive transformations, and consequently overlook subtractive transformations. Across eight experiments, participants were less likely to identify advantageous subtractive changes when the task did not (versus did) cue them to consider subtraction, when they had only one opportunity (versus several) to recognize the shortcomings of an additive search strategy or when they were under a higher (versus lower) cognitive load. Defaulting to searches for additive changes may be one reason that people struggle to mitigate overburdened schedules11, institutional red tape12 and damaging effects on the planet13,14.

https://www.nature.com/articles/s41586-021-03380-y

This is very important to management generally.

A crucial aspect of success in politics/government is simplifying and focusing yet almost everything and everybody tries to stop you doing this.

It’s one of the ‘unrecognised simplicities’ of effective action — totally crucial, simple to understand, you see it repeatedly from Steve Jobs to Elon, yet almost nobody learns the lesson and all normal organisations push you to do the opposite.

Kitaev, genius physicist

When I ask top physicists ‘who alive is closest to one of the greats from history?’, an answer I often get is — ‘Kitaev at Caltech’.

Here is a fascinating interview with him.

https://heritageproject.caltech.edu/interviews-updates/alexei-kitaev