Interesting, thanks. This makes sense to me. I do think strong-HCH can support the ”...more like a simulated society...” stuff in some sense—which is to say that it can be supported so long as we can rely on individual Hs to robustly implement the necessary pointer passing (which, to be fair, we can’t).
To add to your “tree of John Wentworths”, it’s worth noting that H doesn’t need to be an individual human—so we could have our H be e.g. {John Wentworth, Eliezer Yudkowsky, Paul Christiano, Wei Dai}, or whatever team would make you more optimistic about lack of memetic disaster. (we also wouldn’t need to use the same H at every level)
Yeah, at some point we’re basically simulating the alignment community (or possibly several copies thereof interacting with each other). There will probably be another post on that topic soonish.
Interesting, thanks. This makes sense to me.
I do think strong-HCH can support the ”...more like a simulated society...” stuff in some sense—which is to say that it can be supported so long as we can rely on individual Hs to robustly implement the necessary pointer passing (which, to be fair, we can’t).
To add to your “tree of John Wentworths”, it’s worth noting that H doesn’t need to be an individual human—so we could have our H be e.g. {John Wentworth, Eliezer Yudkowsky, Paul Christiano, Wei Dai}, or whatever team would make you more optimistic about lack of memetic disaster. (we also wouldn’t need to use the same H at every level)
Yeah, at some point we’re basically simulating the alignment community (or possibly several copies thereof interacting with each other). There will probably be another post on that topic soonish.