Thanks, I think we’re getting closer. The surface level breaks down mostly into two objections here.
The minor objection is, do you expect your target audience to contain people unsure whether we’ll get AGI prior to Matrioshka brains, conditional on the latter otherwise happening? My expectation is zero people would both be reading and receptive-to the document, and also hold that uncertainty.
The major objection is, is this an important thing to know? Isn’t it wiser to leave that question until people start debating what to allocate their Dyson spheres’ resources towards? Are those people you are realistically able to causally affect, even if they were to exist eventually?
When I ask for an “important claim you can make”, there’s the assumption that resources are put into this because this document should somehow result in things going better somewhere. Eliezer’s critique is surely from this perspective, that he wants to do concrete things to help with probable futures.
Those two points weren’t really my deep objections, because I think both of us understand that this is a far enough out future that by that point we’ll have found something smarter to do, if it were at all possible, but I’m still a bit too unsure about how you intend this far-future scenario to be informative for those earlier events.
Thanks, I think we’re getting closer. The surface level breaks down mostly into two objections here.
The minor objection is, do you expect your target audience to contain people unsure whether we’ll get AGI prior to Matrioshka brains, conditional on the latter otherwise happening? My expectation is zero people would both be reading and receptive-to the document, and also hold that uncertainty.
The major objection is, is this an important thing to know? Isn’t it wiser to leave that question until people start debating what to allocate their Dyson spheres’ resources towards? Are those people you are realistically able to causally affect, even if they were to exist eventually?
When I ask for an “important claim you can make”, there’s the assumption that resources are put into this because this document should somehow result in things going better somewhere. Eliezer’s critique is surely from this perspective, that he wants to do concrete things to help with probable futures.
Those two points weren’t really my deep objections, because I think both of us understand that this is a far enough out future that by that point we’ll have found something smarter to do, if it were at all possible, but I’m still a bit too unsure about how you intend this far-future scenario to be informative for those earlier events.