Find different sets of exercises that are as different as possible from Thinking Physics (i.e. requiring a pretty different set of skills, while still being feeling relevant to becoming a “generalist researcher”), that would make for a good followup to this exercise.
I think my idea of investigating a recent (alleged) poker cheating scandal is a good exercise in this vein. It’s certainly very different from Thinking Physics problems.
The main objections people had when I posted it were that it requires either already having or quickly absorbing a lot of background knowledge about the rules of poker and norms in the high stakes poker scene as a prerequisite, and that there is no way to know if you got the answer right. I continue to think these are not fatal flaws, and that if you’re willing to invest some hours in learning the relevant background (which is itself a good rationality skill to practice, especially if you try to do it under time pressure), the payoff in the quality of the mystery is worth it.
There are a myriad of plausible competing hypotheses and piles of publicly available (but somewhat complex-to-think-about) evidence that make this a good test of your ability to make Bayesian updates about a real world situation. Also, the fact that there is no public consensus is actually a benefit in some ways—the exercise is un-spoilable, and you can research freely without fear of accidentally running into a consensus-accepted definitive conclusion.
Looking into other unsolved mysteries (e.g. murder mysteries, heists, or other famous cold cases) might provide a similar kind of challenge, and if you compile enough cases you could form a set of exercises in the “mystery solving” genre. But it can be hard to find suitable candidates with lots of publicly available evidence of different types, especially cases that still have multiple competing hypotheses and no clear / trivially obvious conclusion. Essentially, you want something that is actually unsolved (not just legally unsolved), but still interesting and not a total dead end due to lack of evidence. I haven’t personally looked into it much, but the JonBenét Ramsey case (warning: gruesome murder / CSA case) comes to mind as one possibility that might suit.
I’m not sure how good this particular exercise is (hard to evaluate without having done it, and the comments in the other post seem to have some good points) but I do like the general idea.
I think my idea of investigating a recent (alleged) poker cheating scandal is a good exercise in this vein. It’s certainly very different from Thinking Physics problems.
The main objections people had when I posted it were that it requires either already having or quickly absorbing a lot of background knowledge about the rules of poker and norms in the high stakes poker scene as a prerequisite, and that there is no way to know if you got the answer right. I continue to think these are not fatal flaws, and that if you’re willing to invest some hours in learning the relevant background (which is itself a good rationality skill to practice, especially if you try to do it under time pressure), the payoff in the quality of the mystery is worth it.
There are a myriad of plausible competing hypotheses and piles of publicly available (but somewhat complex-to-think-about) evidence that make this a good test of your ability to make Bayesian updates about a real world situation. Also, the fact that there is no public consensus is actually a benefit in some ways—the exercise is un-spoilable, and you can research freely without fear of accidentally running into a consensus-accepted definitive conclusion.
Looking into other unsolved mysteries (e.g. murder mysteries, heists, or other famous cold cases) might provide a similar kind of challenge, and if you compile enough cases you could form a set of exercises in the “mystery solving” genre. But it can be hard to find suitable candidates with lots of publicly available evidence of different types, especially cases that still have multiple competing hypotheses and no clear / trivially obvious conclusion. Essentially, you want something that is actually unsolved (not just legally unsolved), but still interesting and not a total dead end due to lack of evidence. I haven’t personally looked into it much, but the JonBenét Ramsey case (warning: gruesome murder / CSA case) comes to mind as one possibility that might suit.
I’m not sure how good this particular exercise is (hard to evaluate without having done it, and the comments in the other post seem to have some good points) but I do like the general idea.