I originally asked people qualitatively what they think the role of different jobs in society are. Then based on that I made a survey with about 100 questions and found there to be about 5 major factors. I then qualitatively asked people about these factors, which lead to me finding additional items that I incorporated in additional surveys. Eventually I had a pool of around 1000 items covering beliefs in various domains, albeit with the same 5-factor structure as originally.
I suggested that 20 of the items from different factors should be included in the LW census, which allowed me to estimate where LW was in terms of those factors. These 24 new items were then selected from the items in the pool that are the most extreme correlates of the delta indicated by the original 20.
Obviously since this procedure is quite distinct from actual rationalism (but also related since it does incorporate LW’s answer to the 20), it’s quite likely that this is a baseless extrapolation that doesn’t actually generalize well. In fact this is specifically one of the things I want to test for, since it seems wise to not overgeneralize LW ideology from a sample of only 20 beliefs to a sample of more than 1000 beliefs. By taking the 24 most extreme correlates of LW’s mean out of the 1000 items, I am stress-testing the model and seeing just how extremely wrong it can get.
I originally asked people qualitatively what they think the role of different jobs in society are. Then based on that I made a survey with about 100 questions and found there to be about 5 major factors. I then qualitatively asked people about these factors, which lead to me finding additional items that I incorporated in additional surveys. Eventually I had a pool of around 1000 items covering beliefs in various domains, albeit with the same 5-factor structure as originally.
I suggested that 20 of the items from different factors should be included in the LW census, which allowed me to estimate where LW was in terms of those factors. These 24 new items were then selected from the items in the pool that are the most extreme correlates of the delta indicated by the original 20.
Obviously since this procedure is quite distinct from actual rationalism (but also related since it does incorporate LW’s answer to the 20), it’s quite likely that this is a baseless extrapolation that doesn’t actually generalize well. In fact this is specifically one of the things I want to test for, since it seems wise to not overgeneralize LW ideology from a sample of only 20 beliefs to a sample of more than 1000 beliefs. By taking the 24 most extreme correlates of LW’s mean out of the 1000 items, I am stress-testing the model and seeing just how extremely wrong it can get.