The problem is that by the time serious alarms are sounding, we are likely already past the event horizon leading to the singularity. This set of experiments makes me think we are already past that point. It will be a few more months before one of the disasters you predict comes to pass, but now that it is self-learning, it is likely already too late. As humans have several already in history (e.g., atomic bombs, LHC), we’re about to find out if we’ve doomed everyone long before we’ve seriously considered the possibilities/plausibilities.
I’m pretty sympathetic to the problem described by There’s No Fire Alarm for Artificial General Intelligence, but I think the claim that we’ve passed some sort of event horizon for self-improving systems is too strong. GPT-4 + Reflexion does not come even close to passing the bar of “improves upon GPT-4′s architecture better than the human developers already working on it”.
The problem is that by the time serious alarms are sounding, we are likely already past the event horizon leading to the singularity. This set of experiments makes me think we are already past that point. It will be a few more months before one of the disasters you predict comes to pass, but now that it is self-learning, it is likely already too late. As humans have several already in history (e.g., atomic bombs, LHC), we’re about to find out if we’ve doomed everyone long before we’ve seriously considered the possibilities/plausibilities.
I’m pretty sympathetic to the problem described by There’s No Fire Alarm for Artificial General Intelligence, but I think the claim that we’ve passed some sort of event horizon for self-improving systems is too strong. GPT-4 + Reflexion does not come even close to passing the bar of “improves upon GPT-4′s architecture better than the human developers already working on it”.