ATA is extremely neglected. The field of ATA is at a very early stage, and currently there does not exist any research project dedicated to ATA. The present post argues that this lack of progress is dangerous and that this neglect is a serious mistake.
I agree it’s neglected, but there is in fact at least one researh project dedicated to at least designing alignment targets: the part of the formal alignment agenda dedicated to formal outer alignment, which is the design of math problems to which solutions would be world-saving. Our notable attempts at this are QACI and ESP (there was also some work on a QACI2, but it predates (and in-my-opinion is superceded by) ESP).
Those try to implement CEV in math. They only work for doing CEV of a single person or small group, but that’s fine: just do CEV of {a single person or small group} which values all of humanity/moral-patients/whatever getting their values satisfied instead of just that group’s values. If you want humanity’s values to be satisfied, then “satisfying humanity’s values” is not opposite to “satisfy your own values”, it’s merely the outcome of “satisfy your own values”.
Hi !
I agree it’s neglected, but there is in fact at least one researh project dedicated to at least designing alignment targets: the part of the formal alignment agenda dedicated to formal outer alignment, which is the design of math problems to which solutions would be world-saving. Our notable attempts at this are QACI and ESP (there was also some work on a QACI2, but it predates (and in-my-opinion is superceded by) ESP).
Those try to implement CEV in math. They only work for doing CEV of a single person or small group, but that’s fine: just do CEV of {a single person or small group} which values all of humanity/moral-patients/whatever getting their values satisfied instead of just that group’s values. If you want humanity’s values to be satisfied, then “satisfying humanity’s values” is not opposite to “satisfy your own values”, it’s merely the outcome of “satisfy your own values”.