I think Robby’s comment captures a lot of my thoughts on this post. This was the third time I read this post, and I think it was the first time that I started getting any grasp on the core concepts of the post. I think there are two primary reasons for this:
1. The concepts are indeed difficult, and are recursive and self-referential in a way that requires a good amount of unpacking and time to understand them
2. The focus of the post shifts very quickly from “here is a crash-course introduction to the wizard’s code” to “here is a crash-course introduction to meta-honesty” to “here is a discussion about whether meta-honesty is good” and “here is an introduction to the considerations around meta-meta-honesty”.
I think it’s good to have a post that tries to give some kind of overview over the considerations around meta-honesty and rational honesty in general, and that that is better than only having a single educational introductory post that can’t give a sense of the bigger picture.
But I do think that the next natural step after this high-level discussion, if we think meta-honesty is an important norm to have in our community, is for someone to write an actual “introduction to honesty” post, that takes an educational approach to the skills outlined in this post, maybe even with some exercises, that allows people to actually learn the skills necessary to be meta-honest, instead of just knowing about them in abstract. (With this comment, I am personally putting out a $50 bounty for such a post, if it includes at least one exercise that forces me to think for myself and that I find even remotely valuable)
And I think that step is also necessary if we don’t know yet whether these norms around meta-honesty are a good idea for us to adopt widely, because I think right now not enough people actually have any system-1 understanding of the principles Eliezer is pointing at, to have a real discussion about these ideas.
So overall, while I think this post doesn’t fully succeed in its aim to teach people how to be meta-honest, I do think it can serve as a good reference work for future authors writing more about this topic. And as such I do think it deserves to be curated.
Also: Succeeded at making the word “meta” lose all meaning for at least 10 minutes after reading this post. +1 would do again.
Yeah, my feeling on re-reading this post is that it would have worked well as a sequence, since it breaks down into a bunch of parts that are important to consider and digest in their own right. (And since it would have benefited from more background, motivation, exercises, examples, etc.)
Also, to give a personal +1 to bounties and to this particular goal, I’ll give another $40 to whoever collects Oliver’s bounty, as judged by Oliver.
I happened to be re-reading this and figured it might be nice to raise this to conscious attention, and (possibly?) raise the bounty if no one has made any progress on this. (Or, alternately, give people some affordance to state their happy price for doing it and see what the supply/demand tradeoffs are like)
Promoted to curated, here are my thoughts:
I think Robby’s comment captures a lot of my thoughts on this post. This was the third time I read this post, and I think it was the first time that I started getting any grasp on the core concepts of the post. I think there are two primary reasons for this:
1. The concepts are indeed difficult, and are recursive and self-referential in a way that requires a good amount of unpacking and time to understand them
2. The focus of the post shifts very quickly from “here is a crash-course introduction to the wizard’s code” to “here is a crash-course introduction to meta-honesty” to “here is a discussion about whether meta-honesty is good” and “here is an introduction to the considerations around meta-meta-honesty”.
I think it’s good to have a post that tries to give some kind of overview over the considerations around meta-honesty and rational honesty in general, and that that is better than only having a single educational introductory post that can’t give a sense of the bigger picture.
But I do think that the next natural step after this high-level discussion, if we think meta-honesty is an important norm to have in our community, is for someone to write an actual “introduction to honesty” post, that takes an educational approach to the skills outlined in this post, maybe even with some exercises, that allows people to actually learn the skills necessary to be meta-honest, instead of just knowing about them in abstract. (With this comment, I am personally putting out a $50 bounty for such a post, if it includes at least one exercise that forces me to think for myself and that I find even remotely valuable)
And I think that step is also necessary if we don’t know yet whether these norms around meta-honesty are a good idea for us to adopt widely, because I think right now not enough people actually have any system-1 understanding of the principles Eliezer is pointing at, to have a real discussion about these ideas.
So overall, while I think this post doesn’t fully succeed in its aim to teach people how to be meta-honest, I do think it can serve as a good reference work for future authors writing more about this topic. And as such I do think it deserves to be curated.
Also: Succeeded at making the word “meta” lose all meaning for at least 10 minutes after reading this post. +1 would do again.
Yeah, my feeling on re-reading this post is that it would have worked well as a sequence, since it breaks down into a bunch of parts that are important to consider and digest in their own right. (And since it would have benefited from more background, motivation, exercises, examples, etc.)
Also, to give a personal +1 to bounties and to this particular goal, I’ll give another $40 to whoever collects Oliver’s bounty, as judged by Oliver.
I happened to be re-reading this and figured it might be nice to raise this to conscious attention, and (possibly?) raise the bounty if no one has made any progress on this. (Or, alternately, give people some affordance to state their happy price for doing it and see what the supply/demand tradeoffs are like)