Experimental: D&D.Sci, with a consistent limit on time & resources used.
Organizational: D&D.Sci, with a consistent limit on time & resources used, using freshly-baked scenarios you know no-one has ever played before.
Limitations:
Takes several hours to play most scenarios.
Requires generic coding/spreadsheeting/data-science-ing skills in addition to Rationality; people who are good at those skills get an unfair(?) advantage.
Getting familiar with the genre gives an unfair(!) advantage.
Misc. addl. reflections on the topic:
Starting from zero is a valid approach, but looking at existing tests and thinking “okay but what if this was better/harder/about slightly different skills” is also sensible. Figuring out how clever and effective people are is a big industry! We should take inspiration from tests employers give job applicants, and any test any gatekeepers give anyone. (Especially if that means we get to subsidize development of rationality-tests by selling them to HR departments.)
. . . are there any ways to test rationality which don’t rely on complementary skills? Even written tests test your ability to read the questions.
Videogames could be so good for this if they weren’t optimized for fun and accessibility.
Reputational: D&D.Sci.
Experimental: D&D.Sci, with a consistent limit on time & resources used.
Organizational: D&D.Sci, with a consistent limit on time & resources used, using freshly-baked scenarios you know no-one has ever played before.
Limitations:
Takes several hours to play most scenarios.
Requires generic coding/spreadsheeting/data-science-ing skills in addition to Rationality; people who are good at those skills get an unfair(?) advantage.
Getting familiar with the genre gives an unfair(!) advantage.
Misc. addl. reflections on the topic:
Starting from zero is a valid approach, but looking at existing tests and thinking “okay but what if this was better/harder/about slightly different skills” is also sensible. Figuring out how clever and effective people are is a big industry! We should take inspiration from tests employers give job applicants, and any test any gatekeepers give anyone. (Especially if that means we get to subsidize development of rationality-tests by selling them to HR departments.)
. . . are there any ways to test rationality which don’t rely on complementary skills? Even written tests test your ability to read the questions.
Videogames could be so good for this if they weren’t optimized for fun and accessibility.