I believe that the only sort of seed AI anyone should ever launch has the “transparency” property, namely, that it is very clear and obvious to its creators what the seed AI’s optimization target is.
When I wrote that 9 days ago, I was not as clear as I should have been. All I meant was that the optimization target should have a clear and unambiguous description or specification that is very well understood by the writers of the description or specification. My personal opinion is that it should consist entirely of formal mathematics and source code.
So for example, I would say that the CEV qualifies as having the “transparency” property because although the current version of the CEV document is not nearly unambiguous enough, it is possible that a team of AI programmers will be able to write a clear and unambiguous description of a human being and his or her volition along with a specification of how to resolve the inconsistencies in what the human wants and of how to extrapolate that.
When I wrote that 9 days ago, I was not as clear as I should have been. All I meant was that the optimization target should have a clear and unambiguous description or specification that is very well understood by the writers of the description or specification. My personal opinion is that it should consist entirely of formal mathematics and source code.
So for example, I would say that the CEV qualifies as having the “transparency” property because although the current version of the CEV document is not nearly unambiguous enough, it is possible that a team of AI programmers will be able to write a clear and unambiguous description of a human being and his or her volition along with a specification of how to resolve the inconsistencies in what the human wants and of how to extrapolate that.