I’m planning on doing a presentation on cognitive biases and/or behavioral economics (Kahneman et al) in front of a group of university students (20-30 people). I want to start with a short experiment / demonstration (or two) that will demonstrate to the students that they are, in fact, subject to some bias or failure in decision making. I’m looking for suggestion on what experiment I can perform within 30 minutes (can be longer if it’s an interesting and engaging task, e.g. a game), the important thing is that the thing being demonstrated has to be relevant to most people’s everyday lives. Any ideas?
I also want to mention that I can get assistants for the experiment if needed.
Edit: Has anyone at CFAR or at rationality minicamps done something similar? Who can I contact to inquire about this?
90% might not be the best number for demonstrating the idea of a confidence interval. It’s too close to 100%. There’s not much room to be underconfident. What about 50% confidence intervals?
For something very brief, anchoring bias is easy to demonstrate and fairly dramatic. I tried this on a friend a couple weeks ago, anchoring her on 1 million people as the population of Ghana; she guessed 900,000. Turned out to be 25 million.
Well the thing is that people actually get this right in real life (e.g. with the rule ‘to drink you must be over 18’). I need something that occurs in real life and people fail at it.
They get it correct when it’s in an appropriate social context, not simply because it’s happening in real life. If it didn’t happen in real life, confirmation bias wouldn’t be a real thing.
Right, but I want to use a closer to real life situation or example that reduces to the wason selection task (and people fail at it) and use that as the demonstration, so that people can see themselves fail in a real life situation, rather than in a logical puzzle. People already realize they might not be very good at generalized logic/math, I’m trying to demonstrate that the general logic applies to real life as well.
Well the thing is that people actually get this right in real life (e.g. with the rule ‘to drink you must be over 18’). I need something that occurs in real life and people fail at it.
No, people are more likely to get it right in real life. Some fraction of your audience will get it wrong, even with ages and drinks.
Confirmation bias, the triplet number test where the rule is “Any triplet where the second number is greater than the first and the third greatet than the second”. Original credit (edit:for my exposure)to Eliezer in HPmoR but I thought of it because that was what Yvain did at a meetup I was at.
Confirmation bias, the triplet number test where the rule is “Any triplet where the second number is greater than the first and the third greatet than the second”. Original credit to Eliezer in HPmoR but I thought of it because that was what Yvain did at a meetup I was at.
To be clear, since reading this made me double-take, I think by “original credit” you mean “original credit for your personal exposure to the concept”.
I’m planning on doing a presentation on cognitive biases and/or behavioral economics (Kahneman et al) in front of a group of university students (20-30 people). I want to start with a short experiment / demonstration (or two) that will demonstrate to the students that they are, in fact, subject to some bias or failure in decision making. I’m looking for suggestion on what experiment I can perform within 30 minutes (can be longer if it’s an interesting and engaging task, e.g. a game), the important thing is that the thing being demonstrated has to be relevant to most people’s everyday lives. Any ideas?
I also want to mention that I can get assistants for the experiment if needed.
Edit: Has anyone at CFAR or at rationality minicamps done something similar? Who can I contact to inquire about this?
Get people to give 90% confidence intervals on 10 questions, and then at the end ask
“Ok, so who got all 10 within their intervals. 9? That’s what you should have got… ok, 8? Still no-one? Ok, how about 7?”
90% might not be the best number for demonstrating the idea of a confidence interval. It’s too close to 100%. There’s not much room to be underconfident. What about 50% confidence intervals?
Have you tried it? I have, and I can tell you most people I tried it on are over-confident when asked for 90% intervals.
I’ve done this in a few small groups, using:
the “this is an attention test” video (aka “how many passes does the team in white make”) - always a hit
a calibration exercise just as Larks suggests below, which works pretty well
the Wason card task—some people find it unconvincing “because it’s logic” (!)
a “learned helplessness” experiment using impossible anagrams—works so-so
a “priming” experiment where two groups have a phrase-building task with different priming words—my version of that sucks, need to rework it
I also show the Spinning Dancer often
For something very brief, anchoring bias is easy to demonstrate and fairly dramatic. I tried this on a friend a couple weeks ago, anchoring her on 1 million people as the population of Ghana; she guessed 900,000. Turned out to be 25 million.
The Wason selection task is a good go-to example of confirmation bias.
Well the thing is that people actually get this right in real life (e.g. with the rule ‘to drink you must be over 18’). I need something that occurs in real life and people fail at it.
They get it correct when it’s in an appropriate social context, not simply because it’s happening in real life. If it didn’t happen in real life, confirmation bias wouldn’t be a real thing.
Right, but I want to use a closer to real life situation or example that reduces to the wason selection task (and people fail at it) and use that as the demonstration, so that people can see themselves fail in a real life situation, rather than in a logical puzzle. People already realize they might not be very good at generalized logic/math, I’m trying to demonstrate that the general logic applies to real life as well.
No, people are more likely to get it right in real life. Some fraction of your audience will get it wrong, even with ages and drinks.
To a first approximation, people get it right in real life.
Confirmation bias, the triplet number test where the rule is “Any triplet where the second number is greater than the first and the third greatet than the second”. Original credit (edit:for my exposure)to Eliezer in HPmoR but I thought of it because that was what Yvain did at a meetup I was at.
To be clear, since reading this made me double-take, I think by “original credit” you mean “original credit for your personal exposure to the concept”.