I dislike this post. I think it does not give enough detail to evaluate whether the proposal is a good one and it doesn’t address most of the cruxes for whether this even viable. That said, I am glad it was posted and I look forward to reading the authors’ response to various questions people have.
The main idea:
“The goal of the CoEm agenda is to build predictably boundable systems, not directly aligned AGIs.”
Do logical (not physical) emulation of the functions carried out by human brains.
Minimize the amount of Magic (uninterpretable processes) going on
Be able to understand the capabilities of your system, it is bounded
Situation CoEm in the human capabilities regime so failures are human-like
Be re-targetable
“Once we have powerful systems that are bounded to the human regime, and can corrigibly be made to do tasks, we can leverage these systems to solve many of the hard problems necessary to exit the acute vulnerable period, such as by vastly accelerating the progress on epistemology and more formal alignment solutions that would be applicable to ASIs.”
My thoughts:
So rather than an a research agenda, this is more of a desiderata for AI safety.
Authors acknowledge that this may be slower than just aiming for AGI. It’s unclear why they think this might work anyway. To the extent that Conjecture wants CoEm to replace the current deep learning paradigm, it’s unclear why they think it will be competitive or why others will adopt it; those are key strategic cruxes.
The authors also don’t give enough details for a reader to tell if they stand a chance; they’re missing a “how”. I look forward to them responding to the many comments raising important questions.
I dislike this post. I think it does not give enough detail to evaluate whether the proposal is a good one and it doesn’t address most of the cruxes for whether this even viable. That said, I am glad it was posted and I look forward to reading the authors’ response to various questions people have.
The main idea:
“The goal of the CoEm agenda is to build predictably boundable systems, not directly aligned AGIs.”
Do logical (not physical) emulation of the functions carried out by human brains.
Minimize the amount of Magic (uninterpretable processes) going on
Be able to understand the capabilities of your system, it is bounded
Situation CoEm in the human capabilities regime so failures are human-like
Be re-targetable
“Once we have powerful systems that are bounded to the human regime, and can corrigibly be made to do tasks, we can leverage these systems to solve many of the hard problems necessary to exit the acute vulnerable period, such as by vastly accelerating the progress on epistemology and more formal alignment solutions that would be applicable to ASIs.”
My thoughts:
So rather than an a research agenda, this is more of a desiderata for AI safety.
Authors acknowledge that this may be slower than just aiming for AGI. It’s unclear why they think this might work anyway. To the extent that Conjecture wants CoEm to replace the current deep learning paradigm, it’s unclear why they think it will be competitive or why others will adopt it; those are key strategic cruxes.
The authors also don’t give enough details for a reader to tell if they stand a chance; they’re missing a “how”. I look forward to them responding to the many comments raising important questions.