You know what… I read the article, then your comments here… and I gotta say—there is absolutely not a chance in hell that this will come even remotely close to being considered, let alone executed. Well—at least not until something goes very wrong… and this something need not be “We’re all gonna die” but more like, say, an AI system that melts down the monetary system… or is used (either deliberately, but perhaps especially if accidentally) to very negatively impact a substantial part of a population. An example could be that it ends up destroying the power grid in half of the US… or causes dozens of aircraft to “fall out of the sky”… something of that size.
Yes—then those in power just might listen and indeed consider very far-reaching safety protocols. Though only for a moment, and some parties shall not care and press on either way, preferring instead to… upgrade, or “fix” the (type of) AI that caused the mayhem.
AI is the One Ring To Rule Them All and none shall toss it into Mount Doom. Yes, even if it turns out to BE Mount Doom—that’s right. Because we can’t. We won’t. It’s our precious, and this, indeed, it really is. But the creation of AI (potentially) capable of a world-wide catastrophe, in my view, as it apparently is in the eyes of EY… is inevitable. We shall not have the wisdom nor the humility to not create it. Zero chance. Undoubtedly intelligent and endowed with well above average IQ as LessWrong subscribers may be, it appears you have a very limited understanding of human nature and the realities of us basically being emotional reptiles with language and an ability to imagine and act on abstractions.
I challenge you to name me a single instance of a tech… any tech at all… being prevented from existing/developing before it caused at least some serious harm. The closest we’ve come are Ozone-depleting chemicals, and even those are still being used, their erstwhile damage only slowly recovering.
Personally, I’ve come to realize that if this world really is a simulated reality I can at least be sure that either I chose this era to live through the AI apocalypse, or this is a test/game to see if this time you can somehow survive or prevent it ;) It’s the AGI running optimization learning to see what else these pesky humans might have come up with to thwart it.
Finally—guys… bombing things (and, presumably, at least some people) on a spurious, as-yet unproven conjectured premise of something that is only a theory and might happen, some day, who knows… really—yeah, I am sure Russia or China or even Pakistan and North Korea will “come to their senses” after you blow their absolute top of the line ultra-expensive hi-tech data center to smithereens… which, no doubt, as it happens, was also a place where (other) supercomputers were developing various medicines, housing projects, education materials in their native languages and an assortment of other actually very useful things they won’t shrug off as collateral damage. Zero Chance, really—every single byte generated in the name of making this happen is 99.999% waste. I understand why you’d want it to work, sure, yes. That would be wonderful. But it won’t, not without a massive “warning” mini-catastrophe first. And if we shall end up right away at total world meltdown… then tough, it would appear such grim fate is basically inevitable and we’re all doomed indeed.
Well this is certainly a very good example, I’ll happily admit as much. Without wanting to be guilty of the True Scotsman fallacy though—Human Cloning is a bit of a special case because it has a very visceral “ickiness” factor… and comes with a unique set of deep feelings and anxieties.
But imagine, if you will, that tomorrow we find the secret to immortality. Making people immortal would bring with it at least two thirds of the same issues that are associated with human cloning… yet it is near-certain any attempts to stop that invention from proliferating are doomed to failure; everybody would want it, even though it technically has quite a few of the types of consequences that cloning would have.
So, yes, agreed—we did pre-emptively deal with human cloning, and I definitely see this as a valid response to my challenge… but I also think we both can tell it is a very special, unique case that comes with most unusual connotations :)
The problem is that by the time serious alarms are sounding, we are likely already past the event horizon leading to the singularity. This set of experiments makes me think we are already past that point. It will be a few more months before one of the disasters you predict comes to pass, but now that it is self-learning, it is likely already too late. As humans have several already in history (e.g., atomic bombs, LHC), we’re about to find out if we’ve doomed everyone long before we’ve seriously considered the possibilities/plausibilities.
I’m pretty sympathetic to the problem described by There’s No Fire Alarm for Artificial General Intelligence, but I think the claim that we’ve passed some sort of event horizon for self-improving systems is too strong. GPT-4 + Reflexion does not come even close to passing the bar of “improves upon GPT-4′s architecture better than the human developers already working on it”.
You know what… I read the article, then your comments here… and I gotta say—there is absolutely not a chance in hell that this will come even remotely close to being considered, let alone executed. Well—at least not until something goes very wrong… and this something need not be “We’re all gonna die” but more like, say, an AI system that melts down the monetary system… or is used (either deliberately, but perhaps especially if accidentally) to very negatively impact a substantial part of a population. An example could be that it ends up destroying the power grid in half of the US… or causes dozens of aircraft to “fall out of the sky”… something of that size.
Yes—then those in power just might listen and indeed consider very far-reaching safety protocols. Though only for a moment, and some parties shall not care and press on either way, preferring instead to… upgrade, or “fix” the (type of) AI that caused the mayhem.
AI is the One Ring To Rule Them All and none shall toss it into Mount Doom. Yes, even if it turns out to BE Mount Doom—that’s right. Because we can’t. We won’t. It’s our precious, and this, indeed, it really is. But the creation of AI (potentially) capable of a world-wide catastrophe, in my view, as it apparently is in the eyes of EY… is inevitable. We shall not have the wisdom nor the humility to not create it. Zero chance. Undoubtedly intelligent and endowed with well above average IQ as LessWrong subscribers may be, it appears you have a very limited understanding of human nature and the realities of us basically being emotional reptiles with language and an ability to imagine and act on abstractions.
I challenge you to name me a single instance of a tech… any tech at all… being prevented from existing/developing before it caused at least some serious harm. The closest we’ve come are Ozone-depleting chemicals, and even those are still being used, their erstwhile damage only slowly recovering.
Personally, I’ve come to realize that if this world really is a simulated reality I can at least be sure that either I chose this era to live through the AI apocalypse, or this is a test/game to see if this time you can somehow survive or prevent it ;) It’s the AGI running optimization learning to see what else these pesky humans might have come up with to thwart it.
Finally—guys… bombing things (and, presumably, at least some people) on a spurious, as-yet unproven conjectured premise of something that is only a theory and might happen, some day, who knows… really—yeah, I am sure Russia or China or even Pakistan and North Korea will “come to their senses” after you blow their absolute top of the line ultra-expensive hi-tech data center to smithereens… which, no doubt, as it happens, was also a place where (other) supercomputers were developing various medicines, housing projects, education materials in their native languages and an assortment of other actually very useful things they won’t shrug off as collateral damage. Zero Chance, really—every single byte generated in the name of making this happen is 99.999% waste. I understand why you’d want it to work, sure, yes. That would be wonderful. But it won’t, not without a massive “warning” mini-catastrophe first. And if we shall end up right away at total world meltdown… then tough, it would appear such grim fate is basically inevitable and we’re all doomed indeed.
Human cloning.
Well this is certainly a very good example, I’ll happily admit as much. Without wanting to be guilty of the True Scotsman fallacy though—Human Cloning is a bit of a special case because it has a very visceral “ickiness” factor… and comes with a unique set of deep feelings and anxieties.
But imagine, if you will, that tomorrow we find the secret to immortality. Making people immortal would bring with it at least two thirds of the same issues that are associated with human cloning… yet it is near-certain any attempts to stop that invention from proliferating are doomed to failure; everybody would want it, even though it technically has quite a few of the types of consequences that cloning would have.
So, yes, agreed—we did pre-emptively deal with human cloning, and I definitely see this as a valid response to my challenge… but I also think we both can tell it is a very special, unique case that comes with most unusual connotations :)
The problem is that by the time serious alarms are sounding, we are likely already past the event horizon leading to the singularity. This set of experiments makes me think we are already past that point. It will be a few more months before one of the disasters you predict comes to pass, but now that it is self-learning, it is likely already too late. As humans have several already in history (e.g., atomic bombs, LHC), we’re about to find out if we’ve doomed everyone long before we’ve seriously considered the possibilities/plausibilities.
I’m pretty sympathetic to the problem described by There’s No Fire Alarm for Artificial General Intelligence, but I think the claim that we’ve passed some sort of event horizon for self-improving systems is too strong. GPT-4 + Reflexion does not come even close to passing the bar of “improves upon GPT-4′s architecture better than the human developers already working on it”.