80 Comments
User's avatar
CP's avatar

Worth saying that HMG IT systems have blocked ChatGPT, obviously. Too useful I presume.

Stu255's avatar

I've come to believe that people are like dogs in that the more often they hear things repeated the more closed minded they become. Everyone is trained by the things we read and hear to some greater or lesser extent.

Twitter is but a deafening circus of performing seals.

This is also why experts are usually the most closed minded people of all, and is why lay people who are blessed with ignorance are able to consider new information in a much more objective manner.

I think this is partly why generalists outperform specialists. It's very hard to become deeply knowledgeable in a particular domain without some ideas getting too sticky. Few domains have enough upheaval to keep them fresh, I think the pace of software advancement is actually a blessing for software companies and tech VC's because it churns the specialists as Schumpeter describes and stops sticky ideas really getting too attached.

I have used chat GPT extensively recently, I think it's amazing whilst also infuriating. What I really want is an offline version, one that learns about my own ideas, principles and beliefs about the world and that I can delegate some agency to (and clone to scale).

The mass market version is too caveated and is really buttoned down, too verbose with excuses, but this will not last for much longer. Functionality and capability is what the market demands. Not safety, niceness and inoffensive. I don't care if my AI is short with me. A players just want to get shit done. So the best AI (the one that wins) will be objective and direct and not cute and safe.

Competitive AI is now here and you can build lots of clever systems with it. This is going to change everything. Those rare 20x people can now scale to 1,000,000x. I can train a clever assistant and then clone that assistant 1,000 times over. That is what is coming this decade.

As an engineer if I want to get stuff done in the real world... I don't want a god, I want an army.

Whilst some people try to build god, they will get bulldozed by flash armies.

That's what nature did. It's the better system, until you eventually get to saturation.

We are in the early days of AI and r / K selection theory says the r strategy will win in the near term.

It's still not AGI though. AGI is not required. I would say most human's lack general intelligence. Even smart people are parrots. People who can solve broad problems and think profoundly are very rare.

What AI is though, is the power to raise an extremely scalable army of obedient agents.

There is enough open source stuff to replicate a lot of this and OpenAI has now proven the technology to a viable extent.

What does this mean for politics? Lots of change.

But really, what's best for Britain?

What kind of society do we want our children to live in? We need to figure this out very fast.

We (all of us) probably only have 2-3 years to influence how this pans out and then the momentum will make it unstoppable. Like some of the great wars, we will then be swept along by history.

The biggest hurdle to changing the world, is usually building a high performance team, consider that this hurdle might have vanished forever within 3 years. Very soon, individuals will be able to scale themselves without needing teamwork.

Who needs managers?

Are the best leaders of people also the best leaders of machines? No.

SysAdmin are the new Emperors.

Dominic Cummings's avatar

People are building your 'offline version' now - on e.g all their own emails/texts/cloud.

And endless other finetunings - e.g 'all science papers since Newton', or 'all energy reports since 19th Century'

Stu255's avatar

Yes, I am doing this.

I’ve got some beefy servers (£10k’s) and I’m doing this at home on hardware that I physically possess. I’m attempting to make a system that can clone proficient agents.

If it works, I can have a 50 man engineering department in my garage working 24/7.

I have no idea if this will work, but it’s worth trying.

Also, have you seen the main LLM players talking of the scaling wall already?

Sam Altman, OpenAI CEO

https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/

Amjad Masad, Replit CEO

https://twitter.com/amasad/status/1647612789517864963?s=46&t=sr2nsGVXpT6dWVRrg6nlsA

Think I’ve predicted exactly that a few times on this blogs comments section. But interesting to see these people saying it. I think for the foreseeable we mostly scale horizontally now that we have a competent agent, and not vertically up the IQ ladder.

Now I’m not saying I’m certain I am correct here, but I think my non-conventional hypothesis-strategy has a pretty good chance of bulldozing success. I think the small r of the r/K selection theory is the way to go for now. It’s usually the case in a new frontier.

TheKnowing's avatar

But will your program's be compatible with Older OS's?

Stu255's avatar

Why would I be concerned about that?

TheKnowing's avatar

Because it's important to run on as many systems as possible? Because deployment targets are bad? Because you want to be accessible? On my ios 11 device I cannot log into ChatGPT. This is an artificial barrier caused by improper CSS and JS. I know for a fact that one can use ChatGPT as a separate client using Java 8 on Windows 98. To not be thinking of all systems shows a level of dangerous intolerance and a wrong mindset.

Stu255's avatar

Dangerous intolerance? I don’t want it to be accessible. It will only run on my system, for me. It’s proprietary.

I’m building this in Linux using a CLI, it will be accessible to me and me only via a web dashboard / workspace that I build and host.

It’s basically an engineering company with up to 50 competent agents listening (24/7 = 168hrs a week). I can build this for £50k capex with £1,000’s pcm opex in API calls.

It seems a lot cheaper than using humans. Similar bandwidth of humans would cost me £12m a year in payroll.

I’m not making a this as a product/service for other people to use. That would be a legal nightmare.

JThorpe's avatar

Have you read The Righteous Mind by Jonathan Haidt?

His analysis would seem to support the claims of easy to fool, hard to shift intelligentsia. More rational people just seem to be better at creating post-hoc justification for their initial emotional response than the less rational.

This would also align with the idea that "less educated" voters may be more open, as they can't so easily work up excuses to dismiss criticism.

Dominic Cummings's avatar

Read some blogs by him but not the book

JThorpe's avatar

I would recommend it if you are curious about some of the psychological phenomena behind some of the trends you have observed.

It definitely clarified the reaction of many people to events such as Brexit for me, as many of these decisions are viewed as much as moral issues, as much as strategic, or economic ones.

jeremy1113's avatar

"f) The blatant way in which Insiders and the old media shift from one false-coordinated-line to the next has generated strong counter-forces, much of which are hidden from the mainstream Insider network (hidden because the latter don’t want to see it but also because people are keeping their views and roles hidden). This seems to be leading more and more previously quite mainstream elements of elite networks to views adjacent to ‘almost anything that throws a wrench in the DC/NYC madness-spiral, even Trump, is good’. It adds to the chances of weird dynamics helping Trump or DeSantis."

So, there are actually some 'adults in the room'. This is reassuring. I've been wondering when whatever sensible members still remain within the elite would start pushing back against the insanity. Maybe China's stealing a march on them internationally has induced a reactionary spasm. Better late than never.

τ′'s avatar

Let me know if you need any help with cloud services, AI infrastructure, etc. (also wrote you a message on Twitter).

TheKnowing's avatar

Did he actually reply on twitter?

Erich M Schwarz's avatar

There's four assumptions you're making here that I find questionable:

1. The Nov. 2020 election was conducted honestly.

Obviously, *if* that were true, then it would indeed make sense to "move on from 2020". The problem is that the same people who swore Hunter Biden's laptop was "Russian disinformation" are the same ones who kicked out GOP vote observers and then collected vast quantities of Biden-only ballots at 4 AM, Nov. 4, 2020. They are not, as the Russians like to say, "agreement capable" people. Worse, they're *still* running U.S. elections and are likely to repeat their banana republic crap in Nov. 2024. What sense does it make to "move on" from Nov. 2020, if Nov. 2020 was the first of what will be a long string of Potemkin elections?

2. Jared Kushner was a brilliant political strategist for Trump in 2020, and would be again in 2024.

I would suggest not taking Kushner at his own valuation on this point. *At least*, I would suggest talking with or reading Peter Navarro before settling conclusively on that idea. Navarro has written two books about his work in the Trump administration, including during the fateful year of 2020. His image of Kushner is not as positive as yours seems to be.

If that is too ridiculous a suggestion, at least consider this: Kushner was completely in charge of Trump's 2020 election campaign *and* of the official Trump anti-fraud efforts in 2020. He failed at both. The frantic last-minute efforts you observed by Giuliani were what happened after Kushner had botched 2020 and had collected a pile of last-minute donations from (mostly small-money) Trump supporters to "stop the steal", but then had languidly refused to spend any actual money or lift one of his fingers to actually, you know, *stop the steal*. Giuliani never got access to the funds that Kushner had collected, which among other things made it much harder for Giuliani to run a professional operation (which would have been difficult to do at the last possible opportunity and with zero notice anyway).

3. Trump was bad at picking lawyers to fight his legal battles over election fraud.

In Nov. 2020, lawyers who wanted to work for Trump were either threatened with firing by their firms or had threats made to their families. That probably didn't get a lot of news coverage in the UK (it certainly got a good leaving alone *here*) but it was real. It was a substantial factor making it difficult for Trump to get good legal help. Giuliani may not have been as good a lawyer as you would have preferred -- I'm sure that he himself would have preferred to either be better himself, or be replaced by somebody who was better -- but he was willing to die to try preventing the theft of a presidential election. In late 2020, this was a sad requirement for being a Trump lawyer.

4. America has not already degenerated into being a decaying banana republic.

If you think my taking exception to that statement is too strong, let me remind you: we have political prisoners in the D.C. jail now. They've been there for two years. One of them was blinded permanently in one eye through a beating by a guard. One of them is a grandmother dying of cancer. Several of them are being denied basic medical treatment. Their putative charges are misdemeanors, but they have been denied bail for two. straight. years. If this happened in some other country, the whitewashed tombs prancing around on our mainstream television programs would be bleating about it. But it's being done by the regime here, so, it's glorious.

In America, we used to have a tradition where people who had political grievances would peacefully demonstrate, quite often at the nation's capitol. After Jan. 6th, during which there were a remarkable number of people like Ray Epps trying to get people to "storm the Capitol!", nobody in serious disagreement with the current regime believes in doing such demonstrations. It's pointless. Any such formerly accepted exhibitions of popular dissent will get more Reichstag Fire treatments, and the people attending them in good faith have an excellent possibility of being beaten by antifa thugs or jailed by FBI thugs.

There is a growing, real literature of books and essays written by people who were serious about trying to defeat the rape of American democracy in 2020. They're worth reading, if you can bear to read them, and want to understand why many of us in America no longer believe that it is very likely Trump will be allowed to run and win an honest election in 2024. But Kushner's book is not part of that literature. The man has one thing he cares about, and it's Kushner.

In 2024, I don't think about whether Trump can win. I think about how much worse things here will get. I think about whether Trump will be convicted on Beria-style charges, and then "commit suicide" like Jeffrey Epstein in a prison cell, or be shot to death if he refuses arrest by his political opponents. I think about more 4 AM vote drops the morning after "Election Day".

Move on from 2020? How can I move on from 2020? I'm still *living* in 2020.

Dominic Cummings's avatar

Im not assuming 2020 was honest. As in all US elections I assume there was significant cheatring, as per 1960 for JFK organised by his dad and Daley.

But Trump blew the handling of this.

I dont think youre right re 2020 / Kushner.

Yes there was lots of pressure on lawyers but he was President - he shd have had all that sorted far in adavance. He didn't. He screwed it up & personally hired a load of duffers.

I agree there are serious prospects for disaster in 2024 including widespread political violence.

But fundamentally this requires the GOP to have a plan for 'this is what we WILL DO' - just going on TV arguing about 2020 will fail. Not even the most hardcore MAGA want to see this. They all want to know - ok, if you win what WILL YOU DO NEXT

jeremy1113's avatar

Excellent and important points. I think it hard for many to realise just how far the United States has slipped and continues to slip, courtesy of a truly vile set of elites.

Patrick Gorham's avatar

i think from Jared and Ivanka's point of view they soon worked out that the Trump Presidency wasnt going to do much , so they used it as a massive business opportunity . Ivanka's no fool she knows her father better than anyone .

Dominic Cummings's avatar

Disagree, they both spent huge time trying to get things to happen, e.g vaccines, Middle East negotiations

Patrick Gorham's avatar

Trouble is Trump cant help but get in his own way all the time. he talked about draining the swamp but brought a large part of it into Government .

misomiso's avatar

Interesting write up.

Alan Texas's avatar

I think hedge funds struggle to find any robust correlation between political outcome and market outcome. Which means it is hard to justify any loss to investors.

Moreover investors, HF managers, sell side banks etc... they are all insiders in this context.. tend to have the same political view.. thus they find it difficult to admit one's personal view (vote tory etc) might not be 'correct' and they then extrapolate 'if not my view then bad for markets'.

Recall Hillary-Trump; the street agreed rational people would vote for Hillary, and Trump would be 'bad' for markets. Not only did they get the outcome wrong on the night, but markets rallied: a double punch of getting it wrong. lol.

Recall the Greek bailout referendum 2015; sell side / hf wondering who on earth would turn down the EU punishment and vote 'no' since it would be 'really bad' if they did that. They voted no, the euro went up. lol^2.

The insiders are also the ones with the money and they gamble according to their views. Thus in betting markets Trump & Brexit in 2016, Greek 'No' in 2015 ... all had similarly slim odds broadly ~15% chance of happening. From these 'low odds' (odds reduced by their own gambling) insiders took great comfort these low odds were 'new' information (over and above 50:50 type opinion polls) thus further confirming how 'correct' their views were. Wonderful to be able to take money out of the echo chamber every so often.

Dominic Cummings's avatar

But isn't the hedge fund world supposed to reward people who take a contrarian view, figure things out, then are proved right?!

For single figure millions you cd get better data than anybody other than the two campaigns on who is likely to win - and if one of the campaigns screws up their polling like HC in 16, youd get better than the campaign!

Id have thought this wd be valuable - even if just to sell on to others including foreign intel agencies!

Ali's avatar

There is a saying in HF world "Dont piss on the political, it is not a good sell"

Name's avatar

I’d be interested in your analysis of Yudkowsky. He seems to be doing a remarkably poor job of presenting g his case and giving a simple, robust explanation of how AGI inevitably leads to the death of humanity. He obfuscates when challenged.

I’m increasingly reminded of how you described the old-school Brexiteers as being more interested in campaigning against brexit than actually willing the referendum

Alethios's avatar

He obfuscates when challenged because he has very little knowledge about how modern ML actually works, and consequently his arguments are built on sand. His preferred method these days is to block people who disagree with him so he can maintain the fiction that "nobody can refute him on the merits".

AGI-risk experts are hopelessly compromised by their situation. Just as the WHO or CDC is never going to admit that they fucked up and may have made matters worse, MIRI will never admit that the entire AI paradigm they were founded to investigate has failed to materialise. They face high financial incentives and extreme social pressures to elevate AGI concerns.

See what actual ML-engineers have to say about Yudkowsky's fever dreams:

https://www.lesswrong.com/posts/wAczufCpMdaamF9fy/my-objections-to-we-re-all-gonna-die-with-eliezer-yudkowsky

Or the accounts of extreme cult-like behaviour across AI-safety orgs:

https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=dLyEcki7dBdxFkvJd

Alethios's avatar

As I talked about on your last post, preliminary results of Tetlock's X-risk tournament are out.

Superforecasters give a 0.38% chance of AI catastophe. AI Experts 3.8%. Unlike with every other risk area which tended towards convergence between the two groups, the forecasts of AI Experts did not update following debate. To me, this suggests very strongly that the 'experts' (and to be clear, we're talking about AGI risk experts here, not actual ML engineers) are either behaving irrationally, or rationally in the sense that they understand that their salaries and prestige are tied very strongly to not updating.

https://twitter.com/PTetlock/status/1649749012029792256

TheKnowing's avatar

No Model for Bernie Sanders? He would have won in 2016 and 2020 if he'd made it past the Primary. Are we just going to assume he's past it for 2024?

TheKnowing's avatar

One year older than Biden. Five years older than Trump. Not really a big age gap.

anna c's avatar

true although AOC does what he does with lippy and heels on (sarcasm...before you tell me off).

Patrick Gorham's avatar

Great blog . i am amazed by P B numbers, i cant seem him having a chance if he gets in it because of the reality/brutality of the campaign. i think he would not last long. i maybe wrong of course ..

Dominic Cummings's avatar

As I say I think these numbers are obviously v noisy - and you have to think how a campaign wd play out. But almost all the main matchups wd be close cos of structural factors. Might the gay problem sink him? Yes. But Im not confiudent about ruling out much except - Trump beats Kamala and Newsom easier than Biden, and Kamala implodes!

hodag's avatar

Go on Red Scare. It would be interesting for all.

John R Ramsden's avatar

Speaking of AI and AGI (G in the latter standing for "general", i.e. not-yet-achieved super duper AI, in case anyone is still unfamiliar with the acronym) ..

It seems clear that at the borders of an AI's domain of expertise there is a kind of twilight zone where its prognostications are wrong. This is referred to as hallucinating, and it seems reasonable to assume the same vulnerability will affect AGI when the latter deals with edge cases. The problem then is how to guard against this potentially dangerous tendency.

An analogous situation occurs in optics when making lenses. A weak, almost flat, lens can be made with a single block of glass. But once one increases the curvature to make the magnification stronger, a vexing problem starts to appear - chromatic aberration, in which a coloured fringe appears round the edge of the image. This can be eliminated by gluing together a pair of back-to-back lenses made of glasses with different refractive indexes, the result being called an achromatic doublet.

So by analogy, perhaps an AI, and in due course AGI, system can be made safer by coupling a pair of them with differing abilities and aptitudes, like a Jack Russell sleeping beside a racehorse in its stable, each being content with and complementing the other in different ways.

Maybe a political parallel would be a minister and their PPS (with the former corresponding to the AI and the latter to the AGI!) :-)

Gruffydd's avatar

You can fix this by limiting the corpus to just a narrow set of very high quality resources, e.g. training a Doctor LLM on thousands of Medicine textbooks. Steve Hsu has mentioned this before, think he might even be helping run a startup working on it iirc.

John R Ramsden's avatar

That is fine for AI, which is specialised in its areas of expertise. But, by definition (hence the different acronym), an AGI will have no firmly delineated boundaries. So I still maintain it will need a sort of dual or mutually reflective design. (This feeds into the topic of consciousness, which to a large extent is the capability of self-reflection, but that is another matter.)

Mr Blah's avatar

If enough of a relevant corpus exists after limiting.

Gruffydd's avatar

I think what you do, and I don’t understand how it works but here goes, is have the language understanding, syntax, grammar, world model etc of a huge LLM like GPT-4, then emphasise the textbook corpus.

Mr Blah's avatar

You can indeed have it search a relevant knowledge base (e.g. added via a vector DB) but I am not sure if this prevents hallucination. I.e. I believe (but could be wrong) that hallucination is an artifact of training and that providing a specific corpus post training won't prevent hallucinations - merely constrain the space in which they take place.

Gruffydd's avatar

Yeah probably true. Definitely one of the big problems to solve in LLMs over the next few years!

Evan Parker's avatar

Can't say i can that interested in election prediction analysis especially re the basket democratic case across the pond. As i've said on several occasions we should be changing our democratic system of politics. Current system is only crudely democratic and forever diminished by corrupting influences at play. Im very dissapointed Dominic that you are drawn to the US when UK politics in such a mess. Stu suggested we need to decide what we want out of democratic politics, esp. with AI coming on stream ..I'll go with that but there seems a distinct lack of interest from the 46k subscribers to this blog in taking this on. Contemplating leaving.

Glyn Morgan's avatar

What’s the date of your polls and focus groups. I suspect that the Texas judge’s nationalization of the abortion ban will hurt the GOP in states like PA and WI. More recent polls suggest precisely this.

Dominic Cummings's avatar

From Nov thro til March.

Trump was closer to the median voter on abortion in the past - maybe he will be again, the DEM 'all legal' position is not popular.

It's another issue where campaign choices will be important - the GOP cd screw it up but cd also neutralise it

Blue Wave's avatar

Could MAGA voters be repelled from Trump by associating criticisms away/independent from the MSM?

How much is AI a threat to human-created art and entertainment?

Thoughts on MBS?

Dominic Cummings's avatar

dont understand 1.

Obviously massive. Will accelerate 'winner takes all' effects already powerful. Will push artists towards more focus on LIVE events. And therell be a crypto opportunity for proving identity. And obv the lawyers will get rich