I’m trying to build my own art of rationality training, and I’ve started talking to various CFAR instructors about their experiences – things that might be important for me to know but which hadn’t been written up nicely before.
Perhaps off topic here, but I want to make sure you have my biggest update if you’re gonna try to build your own art of rationality training.
It is, basically: if you want actual good to result from your efforts, it is crucial to build from and enable consciousness and caring, rather than to try to mimic their functionality.
If you’re willing, I’d be quite into being interviewed about this one point for a whole post of this format, or for a whole dialog, or to talking about it with you in some other way, means, since I don’t know how to say it well and I think it’s crucial. But, to babble:
Let’s take math education as an analogy. There’s stuff you can figure out about numbers, and how to do things with numbers, when you understand what you’re doing. (e.g., I remember figuring out as a kid, in a blinding flash about rectangles, why 2*3 was 3*2, why it would always work). And other people can take these things you can figure out, and package them as symbol-manipulation rules that others can use to “get the same results” without the accompanying insights. But… it still isn’t the same things as understanding, and it won’t get your students the same kind of ability to build new math or to have discernment about which math is any good.
Humans are automatically strategic sometimes. Maybe not all the way, but a lot more deeply than we are in “far-mode” contexts. For example, if you take almost anybody and put them in a situation where they sufficiently badly need to pee, they will become strategic about how to find a restroom. We are all capable of wanting sometimes, and we are a lot closer to strategic at such times.
My original method of proceeding in CFAR, and some other staff members’ methods also, was something like:
Find a person, such as Richard Feynman or Elon Musk or someone a bit less cool than that but still very cool who is willing to let me interview them. Try to figure out what mental processes they use.
Turn these mental processes into known, described procedures that system two / far-mode can invoke on purpose, even when the vicera do not care about a given so-called “goal.”
(For example, we taught processes such as: “notice whether you viscerally expect to achieve your goal. If you don’t, ask why not, solve that problem, and iterate until you have a plan that you do viscerally anticipate will succeed.” (aka inner sim / murphyjitsu.))
My current take is that this is no good—it teaches non-conscious processes how to imitate some of the powers of consciousness, but in a way that lacks its full discernment, and that can lead to relatively capable non-conscious, non-caring processes doing a thing that no one who was actually awake-and-caring would want to do. (And can make it harder for conscious, caring, but ignorant processes, such as youths, to tell the difference between conscious/caring intent, and memetically hijacked processes in the thrall of institutional-preservation-forces or similar.) I think it’s crucial to more like start by helping wanting/caring/consciousness to become free and to become in charge. (An Allan Bloom quote that captures some but not all of what I have in mind: “There is no real education that does not respond to felt need. All else is trifling display.”)
Perhaps off topic here, but I want to make sure you have my biggest update if you’re gonna try to build your own art of rationality training.
It is, basically: if you want actual good to result from your efforts, it is crucial to build from and enable consciousness and caring, rather than to try to mimic their functionality.
If you’re willing, I’d be quite into being interviewed about this one point for a whole post of this format, or for a whole dialog, or to talking about it with you in some other way, means, since I don’t know how to say it well and I think it’s crucial. But, to babble:
Let’s take math education as an analogy. There’s stuff you can figure out about numbers, and how to do things with numbers, when you understand what you’re doing. (e.g., I remember figuring out as a kid, in a blinding flash about rectangles, why 2*3 was 3*2, why it would always work). And other people can take these things you can figure out, and package them as symbol-manipulation rules that others can use to “get the same results” without the accompanying insights. But… it still isn’t the same things as understanding, and it won’t get your students the same kind of ability to build new math or to have discernment about which math is any good.
Humans are automatically strategic sometimes. Maybe not all the way, but a lot more deeply than we are in “far-mode” contexts. For example, if you take almost anybody and put them in a situation where they sufficiently badly need to pee, they will become strategic about how to find a restroom. We are all capable of wanting sometimes, and we are a lot closer to strategic at such times.
My original method of proceeding in CFAR, and some other staff members’ methods also, was something like:
Find a person, such as Richard Feynman or Elon Musk or someone a bit less cool than that but still very cool who is willing to let me interview them. Try to figure out what mental processes they use.
Turn these mental processes into known, described procedures that system two / far-mode can invoke on purpose, even when the vicera do not care about a given so-called “goal.”
(For example, we taught processes such as: “notice whether you viscerally expect to achieve your goal. If you don’t, ask why not, solve that problem, and iterate until you have a plan that you do viscerally anticipate will succeed.” (aka inner sim / murphyjitsu.))
My current take is that this is no good—it teaches non-conscious processes how to imitate some of the powers of consciousness, but in a way that lacks its full discernment, and that can lead to relatively capable non-conscious, non-caring processes doing a thing that no one who was actually awake-and-caring would want to do. (And can make it harder for conscious, caring, but ignorant processes, such as youths, to tell the difference between conscious/caring intent, and memetically hijacked processes in the thrall of institutional-preservation-forces or similar.) I think it’s crucial to more like start by helping wanting/caring/consciousness to become free and to become in charge. (An Allan Bloom quote that captures some but not all of what I have in mind: “There is no real education that does not respond to felt need. All else is trifling display.”)