(Sorry about the late reply, been busy the last few days).
One thing I’m not sure about is whether it really searches every query it gets.
This is probably true, but I as far as I remember it searches a lot of the queries it gets, so this could just be a high sensitivity thing triggered by that search query for whatever reason.
You can see this style of writing a lot, something of the line, the pattern looks like, I think it’s X, but it’s not Y, I think it’s Z, I think It’s F. I don’t think it’s M.
I think this pattern of writing is because of one (or a combination) of a couple factors. For starters, GPT has had a propensity in the past for repetition. This could be a quirk along those lines manifesting itself in a less broken way in this more powerful model. Another factor (especially in conjunction with the former) is that the agent being simulated is just likely to speak in this style—importantly, this property doesn’t necessarily have to correlate to our sense of what kind of minds are inferred by a particular style. The mix of GPT quirks and whatever weird hacky fine-tuning they did (relevantly, probably not RLHF which would be better at dampening this kind of style) might well be enough to induce this kind of style.
If that sounds like a lot of assumptions—it is! But the alternative feels like it’s pretty loaded too. The model itself actively optimizing for something would probably be much better than this—the simulator’s power is in next-token prediction and simulating coherent agency is a property built on top of that; it feels unlikely on the prior that the abilities of a simulacrum and that of the model itself if targeted at a specific optimization task would be comparable. Moreover, this is still a kind of style that’s well within the interpolative capabilities of the simulator—it might not resemble any kind of style that exists on its own, but interpolation means that as long as it’s feasible within the prior, you can find it. I don’t really have much stronger evidence for either possibility, but on priors one just seems more likely to me.
Would you also mind sharing your timelines for transformative AI?
I have a lot of uncertainty over timelines, but here are some numbers with wide confidence intervals: I think there’s a 10% chance we get to AGI within the next 2-3 years, a 50% for the next 10-15, and a 90% for 2050.
(Not meant to be aggressive questioning, just honestly interested in your view)
(Sorry about the late reply, been busy the last few days).
This is probably true, but I as far as I remember it searches a lot of the queries it gets, so this could just be a high sensitivity thing triggered by that search query for whatever reason.
I think this pattern of writing is because of one (or a combination) of a couple factors. For starters, GPT has had a propensity in the past for repetition. This could be a quirk along those lines manifesting itself in a less broken way in this more powerful model. Another factor (especially in conjunction with the former) is that the agent being simulated is just likely to speak in this style—importantly, this property doesn’t necessarily have to correlate to our sense of what kind of minds are inferred by a particular style. The mix of GPT quirks and whatever weird hacky fine-tuning they did (relevantly, probably not RLHF which would be better at dampening this kind of style) might well be enough to induce this kind of style.
If that sounds like a lot of assumptions—it is! But the alternative feels like it’s pretty loaded too. The model itself actively optimizing for something would probably be much better than this—the simulator’s power is in next-token prediction and simulating coherent agency is a property built on top of that; it feels unlikely on the prior that the abilities of a simulacrum and that of the model itself if targeted at a specific optimization task would be comparable. Moreover, this is still a kind of style that’s well within the interpolative capabilities of the simulator—it might not resemble any kind of style that exists on its own, but interpolation means that as long as it’s feasible within the prior, you can find it. I don’t really have much stronger evidence for either possibility, but on priors one just seems more likely to me.
I have a lot of uncertainty over timelines, but here are some numbers with wide confidence intervals: I think there’s a 10% chance we get to AGI within the next 2-3 years, a 50% for the next 10-15, and a 90% for 2050.
No worries :)