[separate comment, so it can get downvoted separately if needed]
Regardless of reliability or accuracy of the distribution of results, it’s clear that many, perhaps most, living humans are not rationally competent, most of the time. A lot of us have some expertise or topics where we have instincts and reflective capacity to get good outcomes, but surprisingly few have the general capability of modeling and optimizing their behaviors and experiences.
I’d expect that this would be front-and-center of debates about long-term human flourishing, and what “alignment” even means. The fact that it’s mostly ignored is a puzzle to me.
If they’re rushing then they can enter $5 as specifically instructed by the question. They can also check that if fits as a correct solution, which it does. The fact that so many people aren’t doing that is interesting.
The report also details a study that tests instructions like “Be careful! Many people miss the following problem because they do not take the time to check their answer.” Along with a big yellow warning icon. Does the warning work? Kindof! mTurk response times go from 22 seconds to 33 seconds. eLab responses go from 21 to 37 seconds (almost doubling). Did scores improve with greater attention paid? Not really! eLab correct answers went from 43% to 52%, a rise of only 9%. mTurk correct answers dropped from 34% to 25%.
I really don’t think that they’re wrong because they’re rushing.
It’s quite rational to ignore what people are telling you to do, and do what’s good for you instead. People on MTurk usually do a bunch of it, and they optimize for payout, not for humoring the experimenter or doing a good job.
I think many of them are not paying attention.
There are probably checks on speed of response to weed out obvious gaming. So the smart thing would be to do something else at the same time, and occasionally enter an answer.
I did cognitive psychology and cog neurosci for a couple of decades, and never ceased to be amazed at the claims of “humans can’t do this” vs. “these people didn’t bother to do this because I didn’t bother to motivate them properly”.
This isn’t to say humans are smart. We’re not nearly as smart as we’d like to think. But we’re not this dumb, either.
That’s fair. How do you explain the in-person respondents? And more than that, the people who answered incorrectly and then were willing to bet money that they were right?
> 16 of the 22 Yale undergraduates who made the common error were sufficiently confident to bet on it, preferring to receive $2 for a correct response than $1 for sure.
That’s a very large proportion! People who are rushing and who know that they are rushing should take the $1. I guess you could claim that that’s not a large enough monetary motivation to induce truthful reflection but.. I think that that’s a big problem all by itself! People in their day-to-day jobs are almost never motivated to put extra thought into making correct decisions to hard problems like these. They’re motivated to put in enough effort to make their boss happy and not get fired and then go home to their spouse and kids. Very few people enjoy these types of problems but everyone encounters them. If someone makes the common error to the bat and ball problem in their daily life, they’ll Maybe lose $5. For most people, intuition is sufficient.
I don’t think that your assertion that humans aren’t this dumb is particularly useful here. Humans definitely do behave this dumbly in their day-to-day lives and that matters quite a lot.
Does it matter a lot? People are screwing up decisions that don’t matter so they can focus on what they actually care about. This has terrible consequences when we take a sum of opinions on complex issues, so I guess that matters.
They’re not screwing up only decisions that don’t matter, they’re also screwing up decisions that don’t matter to them. But many of those decisions matter quite a lot! The entire phenomenon of bike-shedding happens due to people focusing on the wrong things: things that they can trivially and lazily understand. Bike-shedding was first observed and described in the context of people designing nuclear power generating stations.
So in summary, some of the decisions people make are very important. Some of the people making those decisions are screwing them up because they are dumb, with rushing being a contributing but not exclusive factor, a factor that applies just as much in real life as it does in studies.
I feel like handling human irrationality is a major motivation for agent foundations style research. What is the type of human values? How do we model human (ir)rationality to disentangle (unstable) goals from (imperfect) decison making procedures when figuring out what humans want. What’s the thing which the human brain is an approximation of and how does it approximate that ideal? I know getting a model of human irrationality is important to Vanessa, and she thinks working on meta-cognition will make some progress on that. Scott Garrabrants work, of which geometric rationality is the latest example, makes some headway on these problems. I’d go even further and say that a lot of non-agent foundation agendas are motivated by this problem. For instance, Steve Byrnes agenda tackles the “modelling human rationality” part more directly than any other agenda I can think of.
But that’s the thing: there’s been minor progress at best and researchers who work on these problems are pessimistic about further progress. I certainly don’t know of anything that really looks like it would work. Maybe our perspective is wrong. I would guess that shard theorists believe shard-theory would agree with that claim, and would state that shard theory looks like a more promising route to answering these questions (p~0.4).
So whilst the problem isn’t explicitly written about that much, I think a lot of researchers would contest that the problem is being ignored.
[separate comment, so it can get downvoted separately if needed]
Regardless of reliability or accuracy of the distribution of results, it’s clear that many, perhaps most, living humans are not rationally competent, most of the time. A lot of us have some expertise or topics where we have instincts and reflective capacity to get good outcomes, but surprisingly few have the general capability of modeling and optimizing their behaviors and experiences.
I’d expect that this would be front-and-center of debates about long-term human flourishing, and what “alignment” even means. The fact that it’s mostly ignored is a puzzle to me.
Humans are dumb, but not this dumb. This is about people not bothering to answer questions since they’re rushing. See other comments.
If they’re rushing then they can enter $5 as specifically instructed by the question. They can also check that if fits as a correct solution, which it does. The fact that so many people aren’t doing that is interesting.
The report also details a study that tests instructions like “Be careful! Many people miss the following problem because they do not take the time to check their answer.” Along with a big yellow warning icon. Does the warning work? Kindof! mTurk response times go from 22 seconds to 33 seconds. eLab responses go from 21 to 37 seconds (almost doubling). Did scores improve with greater attention paid? Not really! eLab correct answers went from 43% to 52%, a rise of only 9%. mTurk correct answers dropped from 34% to 25%.
I really don’t think that they’re wrong because they’re rushing.
It’s quite rational to ignore what people are telling you to do, and do what’s good for you instead. People on MTurk usually do a bunch of it, and they optimize for payout, not for humoring the experimenter or doing a good job.
I think many of them are not paying attention.
There are probably checks on speed of response to weed out obvious gaming. So the smart thing would be to do something else at the same time, and occasionally enter an answer.
I did cognitive psychology and cog neurosci for a couple of decades, and never ceased to be amazed at the claims of “humans can’t do this” vs. “these people didn’t bother to do this because I didn’t bother to motivate them properly”.
This isn’t to say humans are smart. We’re not nearly as smart as we’d like to think. But we’re not this dumb, either.
That’s fair. How do you explain the in-person respondents? And more than that, the people who answered incorrectly and then were willing to bet money that they were right?
> 16 of the 22 Yale undergraduates who made the common error were sufficiently confident to bet on it, preferring to receive $2 for a correct response than $1 for sure.
That’s a very large proportion! People who are rushing and who know that they are rushing should take the $1. I guess you could claim that that’s not a large enough monetary motivation to induce truthful reflection but.. I think that that’s a big problem all by itself! People in their day-to-day jobs are almost never motivated to put extra thought into making correct decisions to hard problems like these. They’re motivated to put in enough effort to make their boss happy and not get fired and then go home to their spouse and kids. Very few people enjoy these types of problems but everyone encounters them.
If someone makes the common error to the bat and ball problem in their daily life, they’ll Maybe lose $5. For most people, intuition is sufficient.
I don’t think that your assertion that humans aren’t this dumb is particularly useful here. Humans definitely do behave this dumbly in their day-to-day lives and that matters quite a lot.
Does it matter a lot? People are screwing up decisions that don’t matter so they can focus on what they actually care about. This has terrible consequences when we take a sum of opinions on complex issues, so I guess that matters.
They’re not screwing up only decisions that don’t matter, they’re also screwing up decisions that don’t matter to them. But many of those decisions matter quite a lot! The entire phenomenon of bike-shedding happens due to people focusing on the wrong things: things that they can trivially and lazily understand. Bike-shedding was first observed and described in the context of people designing nuclear power generating stations.
So in summary, some of the decisions people make are very important. Some of the people making those decisions are screwing them up because they are dumb, with rushing being a contributing but not exclusive factor, a factor that applies just as much in real life as it does in studies.
I feel like handling human irrationality is a major motivation for agent foundations style research. What is the type of human values? How do we model human (ir)rationality to disentangle (unstable) goals from (imperfect) decison making procedures when figuring out what humans want. What’s the thing which the human brain is an approximation of and how does it approximate that ideal? I know getting a model of human irrationality is important to Vanessa, and she thinks working on meta-cognition will make some progress on that. Scott Garrabrants work, of which geometric rationality is the latest example, makes some headway on these problems. I’d go even further and say that a lot of non-agent foundation agendas are motivated by this problem. For instance, Steve Byrnes agenda tackles the “modelling human rationality” part more directly than any other agenda I can think of.
But that’s the thing: there’s been minor progress at best and researchers who work on these problems are pessimistic about further progress. I certainly don’t know of anything that really looks like it would work. Maybe our perspective is wrong. I would guess that shard theorists believe shard-theory would agree with that claim, and would state that shard theory looks like a more promising route to answering these questions (p~0.4).
So whilst the problem isn’t explicitly written about that much, I think a lot of researchers would contest that the problem is being ignored.