A few years ago I came across The Logic of Failure by Dietrich Doerner (previously mentioned on LW) which discusses cognitive failures in people dealing with “complex situations”.
One section (p.1 28) discusses a little simulation game, where participants are told to “steer” the temperature of a refrigerated storeroom with a defective thermostat, the exact equation governing how the thermostat setting affects actual temperature being unknown. Players control a dial with settings numbered 0 through 100, and can read actual temperature off a thermometer display. The only complications in this task are a) that there is a delay between changing the dial and the effects of the new setting; b) the possibility of “overshoot”.
I found the section’s title chilling as well as fascinating: “Twenty-eight is a good number.” Doerner says this statement is typical of what participants faced with this type of situation tend to say. People don’t just make ineffective use of the data they are presented with: they make up magical hypotheses, cling to superstitions or even call into question the very basis of the experiment, that there is a systematic link between thermostat setting and temperature.
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. We were all involved in one way or another with managing software projects, which are systems way more complex than the simple thermostat system; our interest was to confirm Doerner’s hypothesis that humans are generally inept at even simple management tasks. By the reports of all involved it was one of the most effective learning experiences they’d had. Since then, I have had a particular interest in this type of situation, which I have learned is sometimes called “experiential learning”.
As I conceive of it, experiential learning consists of setting up a problematic situation, in such a way that the students (“players”) should rely on their own wits to explore the situation, invent ways of dealing with it (sometimes by incorporating conceptual tools provided by an instructor), and test their newfound insights against the original problem—or against real-world situations. My preferred setting for experiential learning is a small-group format, with individual or group interaction with the situation, and group discussion for the debrief.
In experiential learning there is no “right” or “wrong” lesson to be taken from a game or simulation. Everything that happens, not just the ostensible game but also the myriad meta-games that accompany it, is fodder for observation and analysis. Neither is realism a requirement for experiential learning; it is an understood convention of the genre that such games present an abstraction of some “real world” situation that necessarily deviates from it in many respects.
The important part of an experiential learning situation is the debrief. In the debrief, you initially refrain from drawing conclusions about the experiment. The first thing you want from the session is data. A good question to ask is “What happened in this session that stood out for you?”
Because you want to map the game back to the real world, perhaps in unforeseen ways, another thing you want from the session is analogies. A good question to ask is “What did the experiences of this session remind you of?”
For learning to take place there should also be some puzzles arising from either the observations made during the game, or their transposition to real life. For instance, your preexisting mental model—derived from real life interactions—would have led you to different predictions about the game.
The intended outcome of experiential learning is for students (and, sometimes, teacher) to construct an updated mental model that resolves these tensions and can be transposed back to real world situations and applied there. A constructivist approach doesn’t expect students to draw exactly the same conclusions as the teacher, even when the teacher makes available the ingredients out of which students build their updated model. Knowledge obtained in that way is more truly a part of you—it sticks better than anything the teacher could have merely told you.
An experiential learning game focusing on the basics of Bayesian reasoning might be a valuable design goal for this community—and a game I’d definitely have an interest in playing. Such is my “hidden agenda” in publishing this post...
Better thinking through experiential games
A few years ago I came across The Logic of Failure by Dietrich Doerner (previously mentioned on LW) which discusses cognitive failures in people dealing with “complex situations”.
One section (p.1 28) discusses a little simulation game, where participants are told to “steer” the temperature of a refrigerated storeroom with a defective thermostat, the exact equation governing how the thermostat setting affects actual temperature being unknown. Players control a dial with settings numbered 0 through 100, and can read actual temperature off a thermometer display. The only complications in this task are a) that there is a delay between changing the dial and the effects of the new setting; b) the possibility of “overshoot”.
I found the section’s title chilling as well as fascinating: “Twenty-eight is a good number.” Doerner says this statement is typical of what participants faced with this type of situation tend to say. People don’t just make ineffective use of the data they are presented with: they make up magical hypotheses, cling to superstitions or even call into question the very basis of the experiment, that there is a systematic link between thermostat setting and temperature.
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. We were all involved in one way or another with managing software projects, which are systems way more complex than the simple thermostat system; our interest was to confirm Doerner’s hypothesis that humans are generally inept at even simple management tasks. By the reports of all involved it was one of the most effective learning experiences they’d had. Since then, I have had a particular interest in this type of situation, which I have learned is sometimes called “experiential learning”.
As I conceive of it, experiential learning consists of setting up a problematic situation, in such a way that the students (“players”) should rely on their own wits to explore the situation, invent ways of dealing with it (sometimes by incorporating conceptual tools provided by an instructor), and test their newfound insights against the original problem—or against real-world situations. My preferred setting for experiential learning is a small-group format, with individual or group interaction with the situation, and group discussion for the debrief.
In experiential learning there is no “right” or “wrong” lesson to be taken from a game or simulation. Everything that happens, not just the ostensible game but also the myriad meta-games that accompany it, is fodder for observation and analysis. Neither is realism a requirement for experiential learning; it is an understood convention of the genre that such games present an abstraction of some “real world” situation that necessarily deviates from it in many respects.
The important part of an experiential learning situation is the debrief. In the debrief, you initially refrain from drawing conclusions about the experiment. The first thing you want from the session is data. A good question to ask is “What happened in this session that stood out for you?”
Because you want to map the game back to the real world, perhaps in unforeseen ways, another thing you want from the session is analogies. A good question to ask is “What did the experiences of this session remind you of?”
For learning to take place there should also be some puzzles arising from either the observations made during the game, or their transposition to real life. For instance, your preexisting mental model—derived from real life interactions—would have led you to different predictions about the game.
The intended outcome of experiential learning is for students (and, sometimes, teacher) to construct an updated mental model that resolves these tensions and can be transposed back to real world situations and applied there. A constructivist approach doesn’t expect students to draw exactly the same conclusions as the teacher, even when the teacher makes available the ingredients out of which students build their updated model. Knowledge obtained in that way is more truly a part of you—it sticks better than anything the teacher could have merely told you.
An experiential learning game focusing on the basics of Bayesian reasoning might be a valuable design goal for this community—and a game I’d definitely have an interest in playing. Such is my “hidden agenda” in publishing this post...
Any takers ?