In the worlds where there’s not much future risk of a LWer someday posting a dangerous capabilities insight, there’s also less future benefit to LW posts, since we’re probably not generating many useful ideas in general (especially about AGI and AGI alignment).
This seems correct, though it’s still valuable to flesh out that it seems possible to have LW posts that are helpful for alignment but not for capabilities: namely, such posts that summarize insights from capabilities research that are known to ~all capabilities people while known to few alignment people.
The main reason I shifted more to your viewpoint now is that capabilities insights might influence people who do not yet know a lot about capabilities to work on that in the future, instead of working on alignment. Therefore, I’m also not sure if Marius’ heuristic “Has company-X-who-cares-mostly-about-capabilities likely thought about this already?” for deciding whether something is infohazardy is safe.
‘We should require a high bar before we’re willing to not-post potentially-world-destroying information to LW, because LW has a strong commitment to epistemic rationality’ seems like an obviously terrible argument to me. People should not post stuff to the public Internet that destroys the world just because the place they’re posting is a website that cares about Bayesianism and belief accuracy.
Yes, that seems correct (though I’m a bit unhappy about you bluntly straw-manning my position). I think after reflection I would phrase my point as follows: ”There is a conflict between Lesswrongs commitment to epistemic rationality on the one hand, and the commitment to restrict info hazards on the other hand. Lesswrong’s commitment to epistemic rationality exists for good reasons, and should not be given up lightly. Therefore, whenever we restrict discussion and information about certain topics, we should have thought about this with great care.”
I don’t yet have a fleshed-out view on this, but I did move a bit in Tom’s direction.
Thanks for your answer!
This seems correct, though it’s still valuable to flesh out that it seems possible to have LW posts that are helpful for alignment but not for capabilities: namely, such posts that summarize insights from capabilities research that are known to ~all capabilities people while known to few alignment people.
The main reason I shifted more to your viewpoint now is that capabilities insights might influence people who do not yet know a lot about capabilities to work on that in the future, instead of working on alignment. Therefore, I’m also not sure if Marius’ heuristic “Has company-X-who-cares-mostly-about-capabilities likely thought about this already?” for deciding whether something is infohazardy is safe.
Yes, that seems correct (though I’m a bit unhappy about you bluntly straw-manning my position). I think after reflection I would phrase my point as follows:
”There is a conflict between Lesswrongs commitment to epistemic rationality on the one hand, and the commitment to restrict info hazards on the other hand. Lesswrong’s commitment to epistemic rationality exists for good reasons, and should not be given up lightly. Therefore, whenever we restrict discussion and information about certain topics, we should have thought about this with great care.”
I don’t yet have a fleshed-out view on this, but I did move a bit in Tom’s direction.