Frank Mager, in various books, including “Preparing Instructional Objectives”, suggests working backward from evidence that would make you conclude that someone is, e.g. a Bayesian Master Rationalist, to the tests (and instructional objectives) for a course of instruction intended to turn someone into a Bayesian Master Rationalist (or whatever you want to turn them into).
After skimming some of his stuff on Amazon, I bought the whole “Mager Six-Pack” and am eagerly devouring it. I can already tell it″s going to make a huge difference in the way I teach mind-hacking.
One of the first ones I read, Goal Analysis, is particularly relevant to LW discussions: how to turn “fuzzies” (abstract qualities, adjectives, and adverbs) into concrete, measurable specifications of behavior. One minor catch: goal analysis can’t make people magically agree on the True Meaning of a term, it can only expose the things they do or don’t agree on...
...which probably makes it an incredibly valuable Rationality Tool in its own right.
Anyway, thanks for mentioning Mager’s books—I’d never heard of them before your comment.
Telephone operators were supposed to have good “tone of service”. So then the education people asked “What does good tone of service mean? What evidence would help you conclude whether an operator has good tone of service?”
And drilling down, they found that there was an entire list of behaviors implicit in the phrase “tone of service”, like inflection as the operator reads the standardized phrases, such as “I’m sorry”. One of the behaviors amused me—no banging—that is, hitting the telephone handset against something, presumably in anger at a frustrating customer.
So you can test for “good tone of service” by testing the observable behaviors.
If your concept of a Master Rationalist includes an “aura of competence”, then probably we can break that down into concrete evidence that would cause you to conclude that someone has an “aura of competence”. The concrete items become instructional objectives. If evidence that someone failed a bias or calibration test would cause you to conclude that they’re NOT a Master Rationalist, then passing the bias or calibration test can be one of the instructional objectives.
Bearing in mind the human tendency tofavorauthority over quality given a choice between the two, I think it’s important when testing to distinguish between “aura of competence” and ability to achieve useful results, and after testing to connect the former to the latter.
Right. EY has mentioned a couple of times that he expects graduates of the hypothetical Rationality Dojo to exude their abilities, like Taking a Level in Badass, or his hedge-fund elites.
I want to clarify that I do not agree with this notion, and I suspect that individuals who exude preternatural skills are primarily good at exuding, not at performing. The example was just an example.
Bearing in mind the human tendency tofavorauthority over quality given a choice between the two, I think it’s important when testing to distinguish between “aura of competence” and ability to achieve useful results, and after testing to connect the former to the latter.
Frank Mager, in various books, including “Preparing Instructional Objectives”, suggests working backward from evidence that would make you conclude that someone is, e.g. a Bayesian Master Rationalist, to the tests (and instructional objectives) for a course of instruction intended to turn someone into a Bayesian Master Rationalist (or whatever you want to turn them into).
After skimming some of his stuff on Amazon, I bought the whole “Mager Six-Pack” and am eagerly devouring it. I can already tell it″s going to make a huge difference in the way I teach mind-hacking.
One of the first ones I read, Goal Analysis, is particularly relevant to LW discussions: how to turn “fuzzies” (abstract qualities, adjectives, and adverbs) into concrete, measurable specifications of behavior. One minor catch: goal analysis can’t make people magically agree on the True Meaning of a term, it can only expose the things they do or don’t agree on...
...which probably makes it an incredibly valuable Rationality Tool in its own right.
Anyway, thanks for mentioning Mager’s books—I’d never heard of them before your comment.
Example?
Telephone operators were supposed to have good “tone of service”. So then the education people asked “What does good tone of service mean? What evidence would help you conclude whether an operator has good tone of service?”
And drilling down, they found that there was an entire list of behaviors implicit in the phrase “tone of service”, like inflection as the operator reads the standardized phrases, such as “I’m sorry”. One of the behaviors amused me—no banging—that is, hitting the telephone handset against something, presumably in anger at a frustrating customer.
So you can test for “good tone of service” by testing the observable behaviors.
If your concept of a Master Rationalist includes an “aura of competence”, then probably we can break that down into concrete evidence that would cause you to conclude that someone has an “aura of competence”. The concrete items become instructional objectives. If evidence that someone failed a bias or calibration test would cause you to conclude that they’re NOT a Master Rationalist, then passing the bias or calibration test can be one of the instructional objectives.
Bearing in mind the human tendency to favor authority over quality given a choice between the two, I think it’s important when testing to distinguish between “aura of competence” and ability to achieve useful results, and after testing to connect the former to the latter.
Right. EY has mentioned a couple of times that he expects graduates of the hypothetical Rationality Dojo to exude their abilities, like Taking a Level in Badass, or his hedge-fund elites.
I want to clarify that I do not agree with this notion, and I suspect that individuals who exude preternatural skills are primarily good at exuding, not at performing. The example was just an example.
Bearing in mind the human tendency to favor authority over quality given a choice between the two, I think it’s important when testing to distinguish between “aura of competence” and ability to achieve useful results, and after testing to connect the former to the latter.