Robin wrote: Having to have an answer now when it seems an likely problem is very expensive.
(I think you meant to write “unlikely” here instead of “likely”.)
Robin, what is your probability that eventually humanity will evolve into a singleton (i.e., not necessarily through Eliezer’s FOOM scenario)? It seems to me that competition is likely to be unstable, whereas a singleton by definition is. Competition can evolve into a singleton, but not vice versa. Given that negentropy increases as mass squared, most competitors have to remain in the center, and the possibility of a singleton emerging there can’t ever be completely closed off. BTW, a singleton might emerge from voluntary mergers, not just one competitor “winning” and “taking over”.
Another reason to try to answer now, instead of later, is that coming up with a good answer would persuade more people to work towards a singleton, so it’s not just a matter of planning for a contingency.
Robin wrote: Having to have an answer now when it seems an likely problem is very expensive.
(I think you meant to write “unlikely” here instead of “likely”.)
Robin, what is your probability that eventually humanity will evolve into a singleton (i.e., not necessarily through Eliezer’s FOOM scenario)? It seems to me that competition is likely to be unstable, whereas a singleton by definition is. Competition can evolve into a singleton, but not vice versa. Given that negentropy increases as mass squared, most competitors have to remain in the center, and the possibility of a singleton emerging there can’t ever be completely closed off. BTW, a singleton might emerge from voluntary mergers, not just one competitor “winning” and “taking over”.
Another reason to try to answer now, instead of later, is that coming up with a good answer would persuade more people to work towards a singleton, so it’s not just a matter of planning for a contingency.