(Warning, far too long and slightly rambly comment incoming. Haven’t gotten fully enough sleep tonight)
I have two levels of comments on this:
1. Pedagogical thoughts:
I really like the pedagogical presentation, but stumbled in a few places. When you asked
“Okay. Before I draw the through-line, it’s worth pausing to check: did any of these spark anything in your own memory or experience? Do you have a similar story rising to the surface already? ”
I don’t think I was fully ready yet to understand what “similar” meant here. A variety of stories that came to mind, but the common pattern of the stories wasn’t clear enough that I could determine whether the stories that came to mind fit what you were asking for. Maybe it might make sense to ask the reader to come up with a potential through-line themselves, and then ask them whether they have a story fitting to that through-line? Not sure, but I did end up stumbling a bit here.
Another thing about the Bayes explanation: I think I would be even happier if you would make some slight references to the terms that are commonly used for the relevant mental/mathematical actions when explaining Bayes in other places. I.e. you use the term “condition on” once in the explanation, but are not putting it into context with how that is the key action in a bayesian update, and how it might be used in a more technical explanation of the subject, such as Eliezer’s intro to Bayes. I generally think that making small cross-references to existing content or alternative explanations has a very large positive effect, since it causes the reader to connect different mental buckets that they are currently separating.
I ended up skimming over the 5-step process on my first read, not fully sure why. I am a bit more tired than usual today, but even so, I feel that maybe a section break or something else to give the actual 5-step technique some more weight would have been good. Maybe another paragraph encouraging the reader to actually apply the process after heaving read through it.
The last thing is more of a bug in human visual processing, but we might still need a few decades to fix it, so I will leave the critique here until then. Humans seem to be really bad at comparing the size of areas, especially the areas of circles, and this causes people to both repeatedly misinterpret pie charts and to be confused by Venn diagrams. I am not really sure what to do about this, except to maybe err on the side of using square Venn diagrams and flow diagrams when possible.
2. Rationality thoughts:
I am not fully sure how much this post will actually help me get rid of bucket errors. I think the really hard question in the process is (2). Bucket errors usually aren’t tagged like bucket errors, and even if I realize that I am in a situation where I might be making a bucket error, the correct resolution of the error is often very hard.
I feel that a part of the process of fixing bucket errors has to be some kind of brainstorming process about alternative explanations which might carve reality differently than you currently do, or the creation of some kind of safe space where you allow yourself to realize that your reasoning might be flawed in this situation. Internal Double Crux goes a long way here (i.e. dialoging with your internal parts instead of punishing or threatening them), but even then I feel that something more in the space of the Fermi Modeling stuff I gave some talks on might be useful.
I am a bit conflicted about this and will think more. I don’t think I fully yet understand what the motion you are describing in step 5 is, and will see whether I can come to better grasp it after sleeping and thinking on it for a while.
I endorse basically all of your constructive critique, as well as your more vaguely voiced concerns. I went back through and addressed everything in the above comment at least in part, though for some things (like your final concern about the motion in step 5) “addressing” it looked like “admitting to and lampshading the handwaving.”
I do think that you pointing out the teachable moment on the phrase “conditioned on” was super high value, and I’m glad for the opportunity to edit that in.
I am not fully sure how much this post will actually help me get rid of bucket errors. I think the really hard question in the process is (2). Bucket errors usually aren’t tagged like bucket errors, and even if I realize that I am in a situation where I might be making a bucket error, the correct resolution of the error is often very hard.
I had the same problem when I ran the algorithm. I made a too-hasty decision about what the “question” was, and “resolving” it was unsatisfying.
Perhaps adding a step for something like aversion factoring, in order to tease out which implication is the emotionally relevant one?
Tried square Venn diagrams prior to habryka’s comment. Rejected them because they were worse pedagogically (despite being more accurate). Suspect rounded-corner quadrangles may actually be an optimal substitute.
Essentially, I found that they implied something much more specific and meaningful in the overlap. With circles, you’re just pushing centers together/overlapping radii, and the mind parses the overlap as perfect/symmetrical and therefore containing no semantic content other than size/area.
With the square or rectangular Venn diagrams, the creator starts making choices—should the overlap be center-to-center, center-to-corner, corner-to-corner, etc.? Should the overlap be “designed” to have a square aspect ratio, or to be golden, or to be long and narrow? If you’ve got a clear grid such that every background square is 1u^2, should you force the blocks to adhere evenly to that grid, or have them be off? What if your area doesn’t easily break down into X by X, or X-minus-a-little by X-plus-a-little? If your squares are e.g. 4x4 and 6x6 but you want the overlap to be 7 squares, what do?
I found that there was no “natural” answer to these questions; no Schelling layout that seemed zero-content. Instead, every arrangement invited the reader to try to actively parse it for additional meaning that wasn’t there, chewing up attention and bandwidth.
Here’s my take on a good Venn diagram layout that doesn’t try to convey extra information, and avoids the problems you and the parent mentioned: https://i.imgur.com/tKPzfLM.png. Make the rectangles full height, and give them rounded corners so it’s clear that these are subsets of a larger space and not just vertical bars (it’s unclear with square corners that there are 2 overlapping sets and not 3 adjacent). Only caveats are that this is not instantly recognizable like your standard Venn diagram, and is only really usable for 2 subsets.
Yeah. I hadn’t thought to go full column height, but this definitely resembles one of the options I thought was most promising. I think you may have identified the best option for square diagrams that match the use case in this post.
(Warning, far too long and slightly rambly comment incoming. Haven’t gotten fully enough sleep tonight)
I have two levels of comments on this:
1. Pedagogical thoughts:
I really like the pedagogical presentation, but stumbled in a few places. When you asked
I don’t think I was fully ready yet to understand what “similar” meant here. A variety of stories that came to mind, but the common pattern of the stories wasn’t clear enough that I could determine whether the stories that came to mind fit what you were asking for. Maybe it might make sense to ask the reader to come up with a potential through-line themselves, and then ask them whether they have a story fitting to that through-line? Not sure, but I did end up stumbling a bit here.
Another thing about the Bayes explanation: I think I would be even happier if you would make some slight references to the terms that are commonly used for the relevant mental/mathematical actions when explaining Bayes in other places. I.e. you use the term “condition on” once in the explanation, but are not putting it into context with how that is the key action in a bayesian update, and how it might be used in a more technical explanation of the subject, such as Eliezer’s intro to Bayes. I generally think that making small cross-references to existing content or alternative explanations has a very large positive effect, since it causes the reader to connect different mental buckets that they are currently separating.
I ended up skimming over the 5-step process on my first read, not fully sure why. I am a bit more tired than usual today, but even so, I feel that maybe a section break or something else to give the actual 5-step technique some more weight would have been good. Maybe another paragraph encouraging the reader to actually apply the process after heaving read through it.
The last thing is more of a bug in human visual processing, but we might still need a few decades to fix it, so I will leave the critique here until then. Humans seem to be really bad at comparing the size of areas, especially the areas of circles, and this causes people to both repeatedly misinterpret pie charts and to be confused by Venn diagrams. I am not really sure what to do about this, except to maybe err on the side of using square Venn diagrams and flow diagrams when possible.
2. Rationality thoughts:
I am not fully sure how much this post will actually help me get rid of bucket errors. I think the really hard question in the process is (2). Bucket errors usually aren’t tagged like bucket errors, and even if I realize that I am in a situation where I might be making a bucket error, the correct resolution of the error is often very hard.
I feel that a part of the process of fixing bucket errors has to be some kind of brainstorming process about alternative explanations which might carve reality differently than you currently do, or the creation of some kind of safe space where you allow yourself to realize that your reasoning might be flawed in this situation. Internal Double Crux goes a long way here (i.e. dialoging with your internal parts instead of punishing or threatening them), but even then I feel that something more in the space of the Fermi Modeling stuff I gave some talks on might be useful.
I am a bit conflicted about this and will think more. I don’t think I fully yet understand what the motion you are describing in step 5 is, and will see whether I can come to better grasp it after sleeping and thinking on it for a while.
In any case, great post!
I endorse basically all of your constructive critique, as well as your more vaguely voiced concerns. I went back through and addressed everything in the above comment at least in part, though for some things (like your final concern about the motion in step 5) “addressing” it looked like “admitting to and lampshading the handwaving.”
I do think that you pointing out the teachable moment on the phrase “conditioned on” was super high value, and I’m glad for the opportunity to edit that in.
I had the same problem when I ran the algorithm. I made a too-hasty decision about what the “question” was, and “resolving” it was unsatisfying.
Perhaps adding a step for something like aversion factoring, in order to tease out which implication is the emotionally relevant one?
> I am not really sure what to do about this, except to maybe err on the side of using square Venn diagrams and flow diagrams when possible.
Strong support for square venn-diagrams.
Tried square Venn diagrams prior to habryka’s comment. Rejected them because they were worse pedagogically (despite being more accurate). Suspect rounded-corner quadrangles may actually be an optimal substitute.
I’ve never actually tried square Venn Diagrams, so I would be interested in you unpacking the “they were worse pedagogically”.
Essentially, I found that they implied something much more specific and meaningful in the overlap. With circles, you’re just pushing centers together/overlapping radii, and the mind parses the overlap as perfect/symmetrical and therefore containing no semantic content other than size/area.
With the square or rectangular Venn diagrams, the creator starts making choices—should the overlap be center-to-center, center-to-corner, corner-to-corner, etc.? Should the overlap be “designed” to have a square aspect ratio, or to be golden, or to be long and narrow? If you’ve got a clear grid such that every background square is 1u^2, should you force the blocks to adhere evenly to that grid, or have them be off? What if your area doesn’t easily break down into X by X, or X-minus-a-little by X-plus-a-little? If your squares are e.g. 4x4 and 6x6 but you want the overlap to be 7 squares, what do?
I found that there was no “natural” answer to these questions; no Schelling layout that seemed zero-content. Instead, every arrangement invited the reader to try to actively parse it for additional meaning that wasn’t there, chewing up attention and bandwidth.
This was a great explanation, and now I want to try this out to see the same effect myself.
Alternative idea: Waterfall diagrams, which is what Eliezer uses sometimes in his latest Bayes Guide.
Here’s my take on a good Venn diagram layout that doesn’t try to convey extra information, and avoids the problems you and the parent mentioned: https://i.imgur.com/tKPzfLM.png. Make the rectangles full height, and give them rounded corners so it’s clear that these are subsets of a larger space and not just vertical bars (it’s unclear with square corners that there are 2 overlapping sets and not 3 adjacent). Only caveats are that this is not instantly recognizable like your standard Venn diagram, and is only really usable for 2 subsets.
Yeah. I hadn’t thought to go full column height, but this definitely resembles one of the options I thought was most promising. I think you may have identified the best option for square diagrams that match the use case in this post.