I write about moral philosophy, artificial intelligence and game theory—in particular non-zero-sum games and their importance in solving the world’s problems. Most of my writing originates on my personal website nonzerosum.games.
I have admitted I am wrong at least 10 times on the internet.
That (deliberate grieving) was also an interesting read, yes, exactly.
I see, I think you’re right not to change it—it’s just provocative enough to be catchy.
Wow, that was quick. I mean, rather than scaffolding work that seems unproductive but is actually necessary, most creative time (for me at least) is wasted in resisting change (my number 3 point was about trying changes even if you don’t immediately agree with them).
Thanks for this, nice writing.
The idea of ‘thinking it faster’ is provocative, because it seems to be over-optimising for speed rather than other values, where as the way you’re implementing it is by generating more meaningful or efficient decisions which are underpinned by a meta-analysis of your process—which is actually about increasing the quality of your decision-making.
I think it’s worthwhile seeing where we’re wasting time. But often I find wasted time isn’t what you’d expect it to be. As someone who also works in the creative industry, criticism is a lot easier than creating something out of whole cloth. Your senior partner, doesn’t just have more experience, but is also a fresh pair of eyes looking at the product you’re creating from a macroscopic (user’s) perspective—this is much easier when you’re not mired in the minutiae. I get this feedback in my job (a documentary editor) not only from people more experienced than me, but also those less experienced.
There a two things I have learned from experience:
1. Blocking out a scene is useful, even though the scene will never be in that form—the boring form of the scene makes it easier to step back and see the more creative way to approach the scene. The time spent making the picture clearer isn’t wasted.
2. When working alone, step away and view your work from a fresh perspective (in my case the audience, in yours the user) to be your own director / senior partner.
That being said, I think it’s well worth meta-analysing your own process and that of your more experienced colleagues, another thing I’ve learned is...
3. When someone you trust gives you changes you don’t agree with, try them, they probably have a clearer perspective than you do.
Anyway, thanks for the post, I’m planning to implement your advice in my own job, it sounds like a worthwhile process. I actually think this third thing is likely to be a key lesson learned from meta-analysis, to not be stubborn and to pivot to the better solution more freely, what I call “back it up and break it”.
Thanks Hastings,
I think at that time you could reason much better if you could recognize that the separation between left and right was not natural.
I think you’re saying it was easier in the past to see unorthodox or contradictory views within parties because the wings were more clearly delineated. I’d agree, it was a divided time, but a less chaotic divided time.
The effective left right split is mono-factor: you are right exactly in proportion to your personal loyalty to one Donald J. Trump
Absolutely, it’s also bizarre regarding his tariff policy which is wholly anti-free market, that’s a point the left didn’t pick up on (because of the chaos I imagine) that was obvious to me. As a left-wing (pro-taxation) person myself who also believes in free markets, his approach is so anti-thetical to my own views, as if he took the last good idea on the right (free markets), and abandoned that in order to create a party based on all the bad ideas. This sort of contrarianism is something I’ve read Steven Pinker write about as a loyalty test (to despots and cult leaders)—the inducement to followers to knowingly lie or act contrary to their own interests as a statement of loyalty to each other through joint faith in the dear leader.
Thanks Mr Frege for clarifying your points. As I have mentioned (in other comments) I’ve conceded that I probably should have contextualised my own abandonment of both-sidesism before taking a partisan approach that makes my post appear more biased than it actually is, and probably colours the way it is read.
advocating that we should not consider the other side of the story
Okay, I definitely should have clarified that this is not my intention at all. Both-sidesism, as I’m referring to it, is creating a false equivalence between two issues and giving them equal weight regardless of their validity. I am strongly for considering the other side of the story. I think it is important to steel-man your opponent’s position, and your comment has revealed that I failed to do this in the post. I should have made a clear case for both-sidesism further than merely stating that it is “well intentioned”, before addressing the problems with it. Thank you for your feedback on those two points, which seem obvious to me in retrospect.
Both sides are prone to such anti-democratic behaviour, but the findings also suggest that one side is “slightly more willing to sacrifice democracy
This was interesting. It provides a counter to the Nature study referenced in the post, which makes sense when considering the different methodologies, specifically what they count as equally anti-democratic actions. I have some ideas about how one could interpret these results, but after writing them down they were pretty lengthy, and would invite a larger argument that I don’t really have time for. I think the study provides an important lesson to learn—the danger of accusing your opponent of something leading to justifying your own doing of that very thing. This is something I’ve called negative moral licensing.
The events of Nov 6⁄7, 2024 might support the argument that the original argument was indeed self-defeating; ie, an argument against the argument against both-sidesism—effectively, an argument for both-sidesism.
I’m afraid I can’t quite parse this point, I’m not sure if you’re saying the election results support my post, or the contrary, and either way, I’m not sure why the result would support either position.
Thanks again for clarifying your points. As you will see I’ve taken on board a few of your points. Hopefully this has been a worthwhile interaction for you, it has for me. Happy to hear your thoughts on the negative moral licensing post if you get around to reading it.
Hi notfnofn, thanks again for the well considered comment, and for responding to my edited response. I think you’ve made good points which have revealed clarifications I could have made within the post.
Okay Trump is president now. Hoping that things go well regardless.
Me too. And we’ll see if the right-wing and online media’s concern that Harris is an equal threat to democracy over the next couple of months. Because if she is an equal threat we shouldn’t expect to see a peaceful transfer of power, like when Trump lost. Although, she has already graciously conceded as would be expected of any political candidate except Trump (who has continued to lie about the result of the 2020 election and require his followers and compatriots to do the same) due to the fact that he is held to a different standard. Obviously no one seriously expects Harris to lead an insurrection on the capitol, but they have been convinced that both-sides are equally dangerous, giving a permission structure to vote for Trump.
It’s not necessarily that it was the worst issue, but the easiest target.
First of all, I am part of the majority that believe that trans-women shouldn’t be competing the women’s category in sport. It’s dangerous, and undermines the integrity of the category due to the natural physical advantages of being born male, particularly on the extremes.
But my point is, as you say “it’s not necessarily the worst issue” whereas the promise to “root out” the “enemy within” is the literally the worst issue. The radical left want to fight for the rights of trans-people in all areas, and unfortunately, I believe, have over-stepped in terms of sport—an entirely optional recreational activity of little to no consequence in my opinion. This is an issue that is adjudicated largely independently of government by international sports’ bodies, and I hope that over time a fair and consistent ruling will prevail.
Rooting out the enemy within, on the other hand, is not even considered radical on the right, it’s said out loud by a mainstream candidate with popular support. This is how far the centre has shifted.
I’m a bit concerned that you referred to cancel culture as “accountability culture”
I think this is a fair use of both-sidesism, if I’m going to use the loaded term ‘cancel culture’, I’m going to qualify that this is opposed by others who see this as ‘accountability culture’. I’m a believer in the free market of ideas, and my support for this principle doesn’t stop when a group of less powerful people collectively use their ideas to combat powerful individuals, I also think companies should be able to act so as to protect themselves from public backlash—I largely believe in free markets in general. Where there are instances of top-down cancelling, which as you mention happens on both sides, I’m opposed to this, and would happily call this cancel culture without qualification. But in my experience that’s a small proportion of what people call “cancel culture”.
Do you not see this as a false equivalence?—Yes...
Great.
Are you comparing the opinions of US politicians on the left with US politicians on the right?
I’m comparing activists on the left with activists on the right. Both the Democratic and Republican parties profess strong pro-Israel support.
How seriously have you investigated the claim that “Harris’s plan is based on what many top economists think is best” and not “Economists find Harris’ plan overall better than Trump’s, despite its many weaknesses”? Have you controlled for the likelihood that they have other reasons to prefer Kamala to Trump?
The first I’d heard of this was in the debate, as a claim of Harris’ that Goldman Sachs and the Wharton School supported her plan, and that 16 Nobel laureates had said that Trump’s plan would invite a recession and increase the deficit. This demonstrated her respect for those experts. Trump wasn’t able to make any similar claims. Since then I have tried to understand more about tariffs, looking to the Wall St Journal and their explanation of Tariffs, Trump’s own interview with John Micklethwait, where he claimed the room full of economists didn’t understand tariffs, and this interview with The Economist Editor in Chief Zanny Minton Beddoes where she underscores the strength of Harris’s plan relative to Trump’s.
These are all respected, relatively right-leaning sources who all agree with Harris, and who’s opinions are respected by Harris, as opposed to Trump who has shown disdain for the opinion of the majority of these experts, in favour of his own expertise, borne out of his experience going bankrupt 6 times. I expect that when developing their plans, this same respect for expertise was also at play. So, I think I’ve investigated this claim seriously enough to have a fair opinion on it.
I’d like to thank you again for this response. I believe you’ve raised important clarifications that I will consider making in the text itself. As you might know, this cross-posted from my blog, and the blog is actually a series of webpages that I edit continually comprising a growing philosophical framework, and I will likely attempt to make it more ever-green by relying less on a current event. Posting here helps guide my editing process by getting critical feedback from smart people like yourself, so I appreciate your time and efforts.
This was a fascinating, original idea as usual. I loved the notion of a brilliant, condescending sort of robot capable of doing a task perfectly who chooses (in order to demonstrate its own artistry) to predict and act out how we would get it wrong.
It did make me wonder though, whether when we reframe something like this for GPTs it’s also important to apply the reframing to our own human intelligence to determine if the claim is distinct; in this case asking the question “are we imitators, simulators or predictors?”. It might be possible to make the case that we are also predictors in as much as our consciousness projects an expectation of the results of our behaviour on to the world, an idea well explained by cognitive scientist Andy Clark.
I agree though, it would be remarkable if GPTs did end up thinking the way we do. And ironically, if they don’t think the way we do, and instead begin to do away with the inefficiencies of predicting and playing out human errors, that would put us in the position of doing the hard work of predicting what how they will act.
Hi Seth,
I share your concern that AGI comes with the potential for a unilateral first strike capability that, at present, no nuclear power has (which is vital to the maintenance of MAD), though I think, in game theoretical terms, this becomes more difficult the more self-interested (in survival) players there are. Like in open-source software, there is a level of protection against malicious code because bad players are outnumbered, even if they try to hide their code, there are many others who can find it. But I appreciate that 100s of coders finding malicious code within a single repository is much easier than finding something hidden in the real world, and I have to admit I’m not even sure how robust the open-source model is (I only know how it works in theory). I’m more pointing to the principle, not as an excuse for complacency but as a safety model on which to capitalise.
My point about the UN’s law against aggression wasn’t that in and of itself it is a deterrent, only that it gives a permission structure for any party to legitimately retaliate.
I also agree that RSI-capable AGI introduces a level of independence that we haven’t seen before in a threat. And I do understand inter-dependence is a key driver of cooperation. Another driver is confidence and my hope is that the more intelligent a system gets, the more confident it is, the better it is able to balance the autonomy of others with its goals, meaning it is able to “confide” in others—in the same way as the strongest kid in class was very rarely the bully, because they had nothing to prove. Collateral damage is still damage after all, a truly confident power doesn’t need these sorts of inefficiencies. I stress this is a hope, and not a cause for complacency. I recognise that in analogy, the strongest kid, the true class alpha, gets whatever they want with the willing complicity of the classroom. RSI-cabable AGI might get what it wants coercively in a way that makes us happy with our own subjugation, which is still a species of dystopia.
But if you’ve got a super-intelligent inventor on your side and a few resources, you can be pretty sure you and some immediate loved ones can survive and live in material comfort, while rebuilding a new society according to your preferences.
This sort of illustrates the contradiction here, if you’re pretty intelligent (as in you’re designing a super-intelligent AGI) you’re probably smart enough to know that the scenario outlined here has a near 100% chance of failure for you and your family, because you’ve created something more intelligent than you that is willing to hide its intentions and destroy billions of people, it doesn’t take much to realise that that intelligence isn’t going to think twice about also destroying you.
Now, I realise this sounds a lot like the situation humanity is in as a whole… so I agree with you that...
multipolar human-controlled AGI scenario will necessitate ubiquitous surveillance.
I’m just suggesting that the other AGI teams do (or can, leveraging the right incentives) provide a significant contribution to this surveillance.
Sorry, you’re right, I did misread that—I’ve edited my response, correcting for my mistake.
Thanks for your comment, the post itself is meant to challenge the reader to question what is really bias, and what is actually an even-handed view with apparent bias, due to the shifted centre. But I certainly take your point, beginning in a clearly partisan manner might not have been the best approach before putting it in context.
I do think there are defences that can be made of the points you raise.
You took one of the tamest aspects of the radical left here
I agree I have taken a tame aspect of the radical left, because there are only tame aspects available. This is my point. The claim you point to, that the left is involved in cancelling conservative voices (not arresting conservatives, as you’ve clarified this claim is not supported by evidence) isn’t any less tame than the accusation of pro-LGBTQ woke-ness. Cancel culture is just a naturally occurring aspect of the free market of ideas (people are free to boycott whatever they like and employers are free to protect their businesses from public backlash). People on the right who usually advocate for free markets should know this best.
The trans issue is a perennial touchstone that has been used as the consistent example of radical left woke-ness for years, and throughout this campaign.
There is antisemitism amongst the pro-Palestine crowd
I don’t doubt you are correct that anti-Israel sentiment can stray into anti-semitism. But the point is about motivations, one side is motivated by sympathy for a population with 10s of thousands of people being killed over a year, and millions being displaced and having their homes destroyed, the other is motivated by white supremacy. Do you not see this as a false equivalence?
In short, I think pronouns and Palestine were fair comparisons. ‘Cancel’ (or ‘accountability’) culture could well be counted as another valid comparison, with a similar tameness to the examples I did use. The reason they sound so tame is because they are tame, they are not comparable, which is the point I’m making—it is the assumption of both-sidesism that leads people to draw the false equivalence.
But, again I agree that I should have spent some time explaining the problem of both-sidesism and the shifted centre before acting in accordance with the principles with which the post concludes.
I don’t know nearly enough about economics to know for sure.
I’m in the same position. It’s at these points where I defer to experts, which is what I have advised in the post.
Thanks again for your comment. I hope my comment hasn’t been too argumentative, it’s meant to explain as an extension of the post.
I agree, it seems as though the incentives aren’t aligned that way, so it ends up incumbent upon the audience to distill nuance out of binary messaging, and to recognise the value of those who do present unique perspectives.
This made me think about how this will come about, whether we we have multiple discrete systems for different functions; language, image recognition, physical balance, executive functions etc working interdependently, communicating through compressed-bandwidth conduits, or whether at some point we can/need-to literally chuck all the raw data from all the systems into one learning system, and let that sort it out (likely creating its own virtual semi-independent systems).
The nuclear MAD standoff with nonproliferation agreements is fairly similar to the scenario I’ve described. We’ve survived that so far- but with only nine participants to date.
I wonder if there’s a clue in this. When you say “only” nine participants it suggests that more would introduce more risk, but that’s not what we’ve seen with MAD. The greater the number becomes, the bigger the deterrent gets. If, for a minute we forgo alliances, there is a natural alliance of “everyone else” at play when it comes to an aggressor. Military aggression is, after all, illegal. So, the greater the number of players, the smaller advantage any one aggressive player has against the natural coalition of all other peaceful players. If we take into account alliances, then this simply returns to a more binary question and the number of players makes no difference.
So, what happens if we apply this to an AGI scenario?
First I want to admit I’m immediately skeptical when anyone mentions a non-iterated Prisoner’s Dilemma playing out in the real world, because a Prisoner’s Dilemma requires extremely confined parameters, and ignores externalities that are present even in an actual prisoner’s dilemma (between two actual prisoners) in the real world. The world is a continuous game, and as such almost all games are iterated games.
If we take the AGI situation, we have an increasing number of players (as you mention “and N increasing”); different AGIs, different humans teams, and mixtures of AGI and human teams, all of which want to survive, some of which may want to dominate or eliminate all other teams. There is a natural coalition of teams that want to survive and don’t want to eliminate all other teams, and that coalition will always be larger and more distributed than the nefarious team that seeks to destroy them. We can observe such robustness in many distributed systems, that seem, on the face of it, vulnerable. This dynamic makes it increasingly difficult for the nefarious team to hide their activities, meanwhile the others are able to capitalise on the benefits of cooperation.
I think we discount the benefit of cooperation, because it’s so ubiquitous in our modern world. This ubiquity of cooperation is a product of a tendency in intelligent systems to evolve toward greater non-zero-sumness. While I share many reservations about AGI, when I remember this fact, I am somewhat reassured that, as our capability to destroy everything gets greater, this capacity is born out of our greater interconnectedness. It is our intelligence and rationality that allows us to harness the benefits of greater cooperation. So, I don’t see why greater rationality on the part of AGI should suddenly reverse this trend.
I don’t want to suggest that this is a non-problem, rather that an acknowledgement of these advantages might allow us to capitalise on them.
Developing an idea about complexity and emergence which looks at the stages of an emergent cycle—that being how a substrate gives rise to an emergent phenomenon, which reaches equilibrium providing the substrate for a the next phenomenon. The way I see it, it goes something like this:
quantum randomness > is predictable at a certain scale > reaches equilibrium > becomes base + randomness (as a byproduct)
or this
substrate + free energy > patterns emerge (disturbances in the uniformity of the free energy) > equilibrium reached > substrate + free energy
This echoes Hegel’s cycle regarding history...
thesis > antithesis > synthesis (thesis—the substrate for further development)
But it’s cumulative. Like a spiral (so is Hegel’s actually, as it refers to History which moves forward so cycles don’t fold back on themselves)
Karl Popper has a related cycle related to intellectual discovery...
Problem 1 > Tentative Theory > Error Elimination (equilibrium) > Problem 2 (the byproduct left out of the solution to P1)
Popper suggests that this is analogous to inorganic physics, biology (using the example of an amoeba responding to heat) and intellectual discovery. Popper refers to organisms as problem-solving structures (to my mind the problem being solved is how to serve entropy probably, organisms are said to be dissipative structures, that while being ordered themselves increase entropy more efficiently than if they weren’t there).
My sense is that all creative or emergent processes follow this pattern. substrate + randomness, patterning (un-uniforming), equilibrium, substrate + randomness.
I’d be interested if anyone else has criticism, or better codifications of this, or elements I’ve missed in this very rough outline, before I solidify this kernel of an idea into a proper post (probably with pictures or interactives).