I was thinking about some of the unique features of Less Wrong’s cosmology—its large-scale model of reality—and decided to ask Bing, “Do you think it’s likely that the multiverse is dominated by an acausal trading equilibrium between grabby squiggle maximizers?”
The resulting dialogue may be seen here. (I normally post dialogues with Bing at Pastebin, but for some reason Pastebin’s filters deemed this one to be “potentially offensive or questionable”.)
I was impressed that Bing grasped the scenario right away, and also that it judged it to be unlikely because it’s based on many independent assumptions. It was also pretty good at suggesting problems faced by those constituent assumptions.
I am unimpressed. I’ve had conversations with people before that went very similarly to this. If this had been a transcript of your conversation with a human, I would have said that human was not engaging with the subject on the gears / object level and didn’t really understand it, but rather had a shallow understanding of the topic, used the anti-weirdness heuristic combined with some misunderstandings to conclude the whole thing was bogus, and then filled in the blanks to produce the rest of the text. Or, to put it differently, BingChat’s writing here reads like a typical essay in a college philosophy 101 class. (Admittedly it’s a bit above-average based on my experience at UNC Chapel Hill, I probably would have given this at least a B)
That said, at the current rate of improvement, I’d expect GPT-5 to be significantly better.
I don’t know, I feel like the day that an AI can do significantly better than this, will be close to the final day of human supremacy. In my experience, we’re still in a stage where the AIs can’t really form or analyze complex structured thoughts on their own—where I mean thoughts with, say, the complexity of a good essay. To generate complex structured thoughts, you have to help them a bit, and when they analyze something complex and structured, they can make out parts of it, but they don’t form a comprehensive overall model of meaning that they can then consult at will.
I don’t have a well-thought-out theory of the further tiers of intellectual accomplishment, these are just my rough impressions. But I can imagine that GPT-4 coupled with chain-of-thought, or perhaps Claude with its enormous, book-length context size, can attain that next level of competence I’ve roughly described as autonomous reading and writing, at the level of essays and journal articles.
I see this as a reason to have one’s best formula for a friendly outcome, ready now (or if you’re into pivotal acts, your best specification of your best proposal for halting the AI race). For me, I guess that’s still June Ku’s version of CEV, filtered through as much reflective virtue as you can manage… The point being that once AIs at that “essay level” of cognition start talking to themselves or each other, you have the ingredients for a real runaway to occur, so you want to be ready to seed it with the best initial conditions you can supply.
Oh I totally agree with everything you say here, especially your first sentence. My timelines median for intelligence explosion (conditional on no significant government-enforced slowdown) is 2027.
So maybe I was misleading when I said I was unimpressed.
Bing writes that one of the premises is that the AIs “can somehow detect or affect each other across vast distances and dimensions”, which seems to indicate that it’s misunderstanding the scenario.
I was thinking about some of the unique features of Less Wrong’s cosmology—its large-scale model of reality—and decided to ask Bing, “Do you think it’s likely that the multiverse is dominated by an acausal trading equilibrium between grabby squiggle maximizers?”
The resulting dialogue may be seen here. (I normally post dialogues with Bing at Pastebin, but for some reason Pastebin’s filters deemed this one to be “potentially offensive or questionable”.)
I was impressed that Bing grasped the scenario right away, and also that it judged it to be unlikely because it’s based on many independent assumptions. It was also pretty good at suggesting problems faced by those constituent assumptions.
I am unimpressed. I’ve had conversations with people before that went very similarly to this. If this had been a transcript of your conversation with a human, I would have said that human was not engaging with the subject on the gears / object level and didn’t really understand it, but rather had a shallow understanding of the topic, used the anti-weirdness heuristic combined with some misunderstandings to conclude the whole thing was bogus, and then filled in the blanks to produce the rest of the text. Or, to put it differently, BingChat’s writing here reads like a typical essay in a college philosophy 101 class. (Admittedly it’s a bit above-average based on my experience at UNC Chapel Hill, I probably would have given this at least a B)
That said, at the current rate of improvement, I’d expect GPT-5 to be significantly better.
I don’t know, I feel like the day that an AI can do significantly better than this, will be close to the final day of human supremacy. In my experience, we’re still in a stage where the AIs can’t really form or analyze complex structured thoughts on their own—where I mean thoughts with, say, the complexity of a good essay. To generate complex structured thoughts, you have to help them a bit, and when they analyze something complex and structured, they can make out parts of it, but they don’t form a comprehensive overall model of meaning that they can then consult at will.
I don’t have a well-thought-out theory of the further tiers of intellectual accomplishment, these are just my rough impressions. But I can imagine that GPT-4 coupled with chain-of-thought, or perhaps Claude with its enormous, book-length context size, can attain that next level of competence I’ve roughly described as autonomous reading and writing, at the level of essays and journal articles.
I see this as a reason to have one’s best formula for a friendly outcome, ready now (or if you’re into pivotal acts, your best specification of your best proposal for halting the AI race). For me, I guess that’s still June Ku’s version of CEV, filtered through as much reflective virtue as you can manage… The point being that once AIs at that “essay level” of cognition start talking to themselves or each other, you have the ingredients for a real runaway to occur, so you want to be ready to seed it with the best initial conditions you can supply.
Oh I totally agree with everything you say here, especially your first sentence. My timelines median for intelligence explosion (conditional on no significant government-enforced slowdown) is 2027.
So maybe I was misleading when I said I was unimpressed.
Bing writes that one of the premises is that the AIs “can somehow detect or affect each other across vast distances and dimensions”, which seems to indicate that it’s misunderstanding the scenario.