9 months ago, I designed something like a rationality test (as in biological rationality, although parts of it depend on prior knowledge of concepts like expected value). I’ll copy it here, I’m curious whether all my questions will get answered correctly. Some of the questions might be logically invalid, please tell me if they are and explain your arguments (I didn’t intend any question to be logically invalid). Also, certain bits might be vague—if you don’t understand it, it’s likely that it’s my fault. Feel free to skip any amount of questions and selectively answer only the ones you like the most. Needless to say, I’m not a psychometrician and I can’t guarantee the correlation between someone’s rationality and his answers to this test.
1: Assume that exactly 10% of the people in the world are left-handed. Also assume that there are absolutely no differences between left-handed and right-handed people (so that the only groups of people where the expected percentage of left-handers is different from 10% are ones where membership explicitly depends on handedness). For these examples, we are looking at a randomly picked class of 24 students from a randomly chosen school. Note: all the examples are independent and in no particular order.
a) If we randomly pick three students—turns out that all of them are left-handed—do we still expect the average number of left-handers among the remaining 21 students to be 10%?
b) We count 10 right-handers (assuming that we managed to find at least 10 right-handers—if we didn’t, we would have changed the group). Among the remaining 14 students, is the average number of left-handers still 10%?
c) We randomly count 10 students. Turns out that all of them are right-handed. Is the average number of left-handers for the rest of the class still 10%? If your answer to this question was different from your answer to b), please explain why was this the case.
d) You happen to know one of the students in this class. You met him one day from a meeting, which was attended by all of the students from three of the classes in the school (out of 13 classes, each class has 24 students) - on that meeting, you randomly picked a left-handed person, out of all the left-handers who were there. Does this fact mean that the average number of left-handers in the remaining 23 students is different from 10%?
e) Out of all the left-handers in the world (about 700 million), you pick one at random. He happens to be from that class. Does this affect the average number of left-handers out of the remaining 23 people? (note that even very low changes in probability count as changes)
2: You are in Bulgaria. The number plates there always have four random numbers from 0 to 9. A person next to you claims to have psychic abilities and he says that the next car that you’ll come across will have the number 1337. The next car you come across has the number 1307. You are amazed by this and think that he might have real powers. He was very close to the actual number—what is the probability for someone to guess a number as close as he did, if we assume that he only made a guess?
3: Assume that A and B are psychological factors, both significantly correlated with school grades. It’s still possible (but unlikely) to have a good grades with a low score on either of them, or even both of them. Also we assume that A and B are totally uncorrelated, and that the only criteria for acceptance in a university is grades.
a) If you are in a university of average quality (with correspondingly average requirements for grades), and you aim to find people who score as high on A as possible, is it a good strategy to place higher priority on people who score as high on B as possible, or should they score as low as possible? Why?
b) Does your answer to a) change if the university is of low quality? What if it’s of high quality? If yes, why?
4: You have to pick between a certain profit of 10$ or 35% chance of winning 30$. Assume you already are financially stable and have a lot of money. Which is the correct choice, if you want to have as much money as possible?? Why?
I find these questions unclearly written. For example, in the license plate case, what does “close” mean? Are 1337 and 1307 close because three digits are exactly the same and the fourth one doesn’t matter as long as it’s not perfect, or because the nonmatching digit is only 3 away, or because the numbers have a difference of 30 out of a possible difference of thousands, or what?
I meant to say, a close match to what the person said. And I’m not entirely confident that 2 makes sense, I’d like to clarify something but that would give out the answer. Please tell me of the other questions you don’t understand.
I meant to say, a close match to what the person said.
This still doesn’t clear up my confusion. I’ll clarify.
In case (a), 1307 is as close to 1337 as are the example numbers 7337, 1937, and 1330 (among others). The only way 1307 could be closer to 1337 is if it were exactly 1337.
In case (b), 1307 is as close to 1337 as are the example numbers 4337, 1037, and 1334 (among others). The found number could be closer to 1337 if it were instead 1347 or 1327 (among others).
In case (c), 1307 is as close to 1337 as is 1367. The found number could be closer to 1337 if it were 1338, or 1336 (among others).
assume that there are absolutely no differences between left-handed and right-handed people
This can’t be. If nothing else, the one group uses their left hand and the other uses their right. You need an “except” or “other than” clause.
We count 10 right-handers (assuming that we managed to find at least 10 right-handers).
Did it just happen to turn out that we found ten, so we can proceed, and if we didn’t find ten we’d skip this problem—or does this problem solely use classes that have ten and throw out other classes?
Is the average number of left-handers still 10%?
In the entire class? Because that’s not clear.
randomly picked a left-handed person
Went around shaking hands until locating a left-handed person, or grabbed the first person you saw and they were left-handed?
Does this affect the average left-handers out of the remaining 23 people?
This is a weird and misleading way to put it if we’re still assuming the people in the class are independent of each other. Yes, even with the word “average”; I’m talking about writing, not math.
Also we assume that A and B are totally uncorrelated
What, really? These are both heavily correlated with a third thing but not at all with each other? Are there real phenomena that act like that? It is unlikely to have good grades and a low score on either one, but they’re not correlated?
profit … winning
I’m just nitpicking here, but this made me wonder if a won $35 would be taxed where the $10 wouldn’t.
as much money as possible
This is bad wording if this is supposed to be an expected value question. The most money possible is just $35; you don’t even have to work out the expected value. If you take the ten dollars you are not getting as much as you could possibly have gotten.
In case (b), 1307 is as close to 1337 as are the example numbers 4337, 1037, and 1334 (among others). The found number could be closer to 1337 if it were instead 1347 or 1327 (among others).
This is the case I meant to (at least one that would be very close to what someone would use in real life). The point is to choose your own criteria for the example situation to determine whether that person is a real magician.
This can’t be. If nothing else, the one group uses their left hand and the other uses their right. You need an “except” or “other than” clause.
I know, but in real life, left-handers can be a subject of stereotyping and discrimination. So I wanted to omit factors like those, like everyone does in such questions. I could have said that some have gene A and others have gene B and only you can identify people and nobody else cares about it, because it has no effect on anything, but handedness seemed more intuitive to me, for this already quite abstract question.
Did it just happen to turn out that we found ten, so we can proceed, and if we didn’t find ten we’d skip this problem—or does this problem solely use classes that have ten and throw out other classes?
The problem only uses classes that have ten or more right-handers. I have edited this in the description.
In the entire class? Because that’s not clear.
I have clarified that. I don’t know why did I include this item, because it sort of duplicates a).
Went around shaking hands until locating a left-handed person, or grabbed the first person you saw and they were left-handed?
I have edited it to “randomly picked a left-handed person, out of all the left-handers who were there”.
What, really? These are both heavily correlated with a third thing but not at all with each other? Are there real phenomena that act like that? It is unlikely to have good grades and a low score on either one, but they’re not correlated?
Why not? The original was with IQ and concentration, but someone took it literally, so I decided to rename it. As far as I know, they + conscientiousness are all correlated with academic success, but not correlated with each other. Also, intelligence and social abilities are both correlated with social success.
I’m just nitpicking here, but this made me wonder if a won $35 would be taxed where the $10 wouldn’t.
What do you mean? There are no taxes in either case.
This is bad wording if this is supposed to be an expected value question. The most money possible is just $35; you don’t even have to work out the expected value. If you take the ten dollars you are not getting as much as you could possibly have gotten.
I think it’s fine this way and I can’t think of another way to word it. English isn’t my first language.
(note that even very low changes in probability count as changes)
And you tell me that now? I had been answering the previous questions assuming I was allowed to round numbers of the order of 1/(world population) down to zero...
1.A) Approximately. (Originally this was yes, until you stated that there were at least 700 million people on the planet. After that information, I updated this answer, because I realized that the problem had an additional assumption of a finite number of people, thus encountering any one left-handed person reduces the odds, very very marginally, of any different future person I encounter being left-handed, because the pool of people I’m drawing from now has slightly different odds.)
1.B) No. (Still.)
1.C) Approximately. Why the answer is different without resorting to math: In 1.B, we nonrandomly pull 10 right-handed students out of the group. In a pool of 24 10-sided die we’ve already rolled, we’ve pulled out 10 of them which did not roll 1; this does not alter the number which did roll 1, increasing their relative proportion. In this case, we’ve rolled the dice 10 times, and they never came up 1; the remaining 14 times remain fair dice rolls.
1.D) (Modified) Approximately.
1.E) Very very slightly.
2.) [Edited; apparently I screwed up when I added the possibility of an exact match] .41%, still assuming we’re not considering the proximity of 0 to 3, and including closer matches. (That is, only considering identical digit matches.)
3.ab) Supposing it’s more likely that a higher quality student is A than !A; it’s possible that it’s extremely unlikely for a person who isn’t high A to have high grades while still having more high grade students who aren’t A than are A, if the odds of A are substantially lower than the odds of being neither A nor B but still having high grades. So there’s not enough information.
Assuming it’s more likely you’re A and have high grades than ~A and have high grades, however, and assuming that this distribution holds for the grade average for each college (p(A|G) > .5 for all three G), you should in all cases favor low-B students, because the remaining pool of accepted students is more likely to be A than !A, because !B limits you to the pool of students who are either A or !A with high (enough) grades, and A was already assumed to be more likely.
But we don’t really know p(A|G), either for low, average, or high grade levels from the problem description, so I couldn’t actually say.
ETA: That should really be p(A|G!B), because, while A and B are independent variables, G is correlated with both. But I think everything still holds anyways.
4.) 35%; expected utility is p(A) times A, which leaves us with 1*10 and .35 times 35, or 10 and 12.somethingorother. We have expected returns of $12 for the 35% case, which is higher than the $10 case.
Why the answer is different: Because 1.C asks what are expectations are, and 1.B asks what the state of the class is
For b) and c), the questions were supposed to be the same—my bad, I have edited it. Please edit your answer accordingly.
Not all of your answers were correct (unsurprisingly, because I find some of the questions extremely hard—even I couldn’t answer them at first :D). I’ll wait for a few more replies and then I’ll post the correct answers plus explanations.
Oddly, my answers remained the same, but for different reasons. Also, I changed my answer to 1.D, and would recommend you change the wording to “Expected average” wherever you merely refer to the average.
9 months ago, I designed something like a rationality test (as in biological rationality, although parts of it depend on prior knowledge of concepts like expected value). I’ll copy it here, I’m curious whether all my questions will get answered correctly. Some of the questions might be logically invalid, please tell me if they are and explain your arguments (I didn’t intend any question to be logically invalid). Also, certain bits might be vague—if you don’t understand it, it’s likely that it’s my fault. Feel free to skip any amount of questions and selectively answer only the ones you like the most. Needless to say, I’m not a psychometrician and I can’t guarantee the correlation between someone’s rationality and his answers to this test.
1: Assume that exactly 10% of the people in the world are left-handed. Also assume that there are absolutely no differences between left-handed and right-handed people (so that the only groups of people where the expected percentage of left-handers is different from 10% are ones where membership explicitly depends on handedness). For these examples, we are looking at a randomly picked class of 24 students from a randomly chosen school. Note: all the examples are independent and in no particular order.
a) If we randomly pick three students—turns out that all of them are left-handed—do we still expect the average number of left-handers among the remaining 21 students to be 10%?
b) We count 10 right-handers (assuming that we managed to find at least 10 right-handers—if we didn’t, we would have changed the group). Among the remaining 14 students, is the average number of left-handers still 10%?
c) We randomly count 10 students. Turns out that all of them are right-handed. Is the average number of left-handers for the rest of the class still 10%? If your answer to this question was different from your answer to b), please explain why was this the case.
d) You happen to know one of the students in this class. You met him one day from a meeting, which was attended by all of the students from three of the classes in the school (out of 13 classes, each class has 24 students) - on that meeting, you randomly picked a left-handed person, out of all the left-handers who were there. Does this fact mean that the average number of left-handers in the remaining 23 students is different from 10%?
e) Out of all the left-handers in the world (about 700 million), you pick one at random. He happens to be from that class. Does this affect the average number of left-handers out of the remaining 23 people? (note that even very low changes in probability count as changes)
2: You are in Bulgaria. The number plates there always have four random numbers from 0 to 9. A person next to you claims to have psychic abilities and he says that the next car that you’ll come across will have the number 1337. The next car you come across has the number 1307. You are amazed by this and think that he might have real powers. He was very close to the actual number—what is the probability for someone to guess a number as close as he did, if we assume that he only made a guess?
3: Assume that A and B are psychological factors, both significantly correlated with school grades. It’s still possible (but unlikely) to have a good grades with a low score on either of them, or even both of them. Also we assume that A and B are totally uncorrelated, and that the only criteria for acceptance in a university is grades. a) If you are in a university of average quality (with correspondingly average requirements for grades), and you aim to find people who score as high on A as possible, is it a good strategy to place higher priority on people who score as high on B as possible, or should they score as low as possible? Why? b) Does your answer to a) change if the university is of low quality? What if it’s of high quality? If yes, why?
4: You have to pick between a certain profit of 10$ or 35% chance of winning 30$. Assume you already are financially stable and have a lot of money. Which is the correct choice, if you want to have as much money as possible?? Why?
I find these questions unclearly written. For example, in the license plate case, what does “close” mean? Are 1337 and 1307 close because three digits are exactly the same and the fourth one doesn’t matter as long as it’s not perfect, or because the nonmatching digit is only 3 away, or because the numbers have a difference of 30 out of a possible difference of thousands, or what?
I meant to say, a close match to what the person said. And I’m not entirely confident that 2 makes sense, I’d like to clarify something but that would give out the answer. Please tell me of the other questions you don’t understand.
This still doesn’t clear up my confusion. I’ll clarify.
In case (a), 1307 is as close to 1337 as are the example numbers 7337, 1937, and 1330 (among others). The only way 1307 could be closer to 1337 is if it were exactly 1337.
In case (b), 1307 is as close to 1337 as are the example numbers 4337, 1037, and 1334 (among others). The found number could be closer to 1337 if it were instead 1347 or 1327 (among others).
In case (c), 1307 is as close to 1337 as is 1367. The found number could be closer to 1337 if it were 1338, or 1336 (among others).
This can’t be. If nothing else, the one group uses their left hand and the other uses their right. You need an “except” or “other than” clause.
Did it just happen to turn out that we found ten, so we can proceed, and if we didn’t find ten we’d skip this problem—or does this problem solely use classes that have ten and throw out other classes?
In the entire class? Because that’s not clear.
Went around shaking hands until locating a left-handed person, or grabbed the first person you saw and they were left-handed?
This is a weird and misleading way to put it if we’re still assuming the people in the class are independent of each other. Yes, even with the word “average”; I’m talking about writing, not math.
What, really? These are both heavily correlated with a third thing but not at all with each other? Are there real phenomena that act like that? It is unlikely to have good grades and a low score on either one, but they’re not correlated?
I’m just nitpicking here, but this made me wonder if a won $35 would be taxed where the $10 wouldn’t.
This is bad wording if this is supposed to be an expected value question. The most money possible is just $35; you don’t even have to work out the expected value. If you take the ten dollars you are not getting as much as you could possibly have gotten.
This is the case I meant to (at least one that would be very close to what someone would use in real life). The point is to choose your own criteria for the example situation to determine whether that person is a real magician.
I know, but in real life, left-handers can be a subject of stereotyping and discrimination. So I wanted to omit factors like those, like everyone does in such questions. I could have said that some have gene A and others have gene B and only you can identify people and nobody else cares about it, because it has no effect on anything, but handedness seemed more intuitive to me, for this already quite abstract question.
The problem only uses classes that have ten or more right-handers. I have edited this in the description.
I have clarified that. I don’t know why did I include this item, because it sort of duplicates a).
I have edited it to “randomly picked a left-handed person, out of all the left-handers who were there”.
Why not? The original was with IQ and concentration, but someone took it literally, so I decided to rename it. As far as I know, they + conscientiousness are all correlated with academic success, but not correlated with each other. Also, intelligence and social abilities are both correlated with social success.
What do you mean? There are no taxes in either case.
I think it’s fine this way and I can’t think of another way to word it. English isn’t my first language.
And you tell me that now? I had been answering the previous questions assuming I was allowed to round numbers of the order of 1/(world population) down to zero...
1.A) Approximately. (Originally this was yes, until you stated that there were at least 700 million people on the planet. After that information, I updated this answer, because I realized that the problem had an additional assumption of a finite number of people, thus encountering any one left-handed person reduces the odds, very very marginally, of any different future person I encounter being left-handed, because the pool of people I’m drawing from now has slightly different odds.)
1.B) No. (Still.)
1.C) Approximately. Why the answer is different without resorting to math: In 1.B, we nonrandomly pull 10 right-handed students out of the group. In a pool of 24 10-sided die we’ve already rolled, we’ve pulled out 10 of them which did not roll 1; this does not alter the number which did roll 1, increasing their relative proportion. In this case, we’ve rolled the dice 10 times, and they never came up 1; the remaining 14 times remain fair dice rolls.
1.D) (Modified) Approximately.
1.E) Very very slightly.
2.) [Edited; apparently I screwed up when I added the possibility of an exact match] .41%, still assuming we’re not considering the proximity of 0 to 3, and including closer matches. (That is, only considering identical digit matches.)
3.ab) Supposing it’s more likely that a higher quality student is A than !A; it’s possible that it’s extremely unlikely for a person who isn’t high A to have high grades while still having more high grade students who aren’t A than are A, if the odds of A are substantially lower than the odds of being neither A nor B but still having high grades. So there’s not enough information.
Assuming it’s more likely you’re A and have high grades than ~A and have high grades, however, and assuming that this distribution holds for the grade average for each college (p(A|G) > .5 for all three G), you should in all cases favor low-B students, because the remaining pool of accepted students is more likely to be A than !A, because !B limits you to the pool of students who are either A or !A with high (enough) grades, and A was already assumed to be more likely.
But we don’t really know p(A|G), either for low, average, or high grade levels from the problem description, so I couldn’t actually say.
ETA: That should really be p(A|G!B), because, while A and B are independent variables, G is correlated with both. But I think everything still holds anyways.
4.) 35%; expected utility is p(A) times A, which leaves us with 1*10 and .35 times 35, or 10 and 12.somethingorother. We have expected returns of $12 for the 35% case, which is higher than the $10 case.
For b) and c), the questions were supposed to be the same—my bad, I have edited it. Please edit your answer accordingly.
Not all of your answers were correct (unsurprisingly, because I find some of the questions extremely hard—even I couldn’t answer them at first :D). I’ll wait for a few more replies and then I’ll post the correct answers plus explanations.
Oddly, my answers remained the same, but for different reasons. Also, I changed my answer to 1.D, and would recommend you change the wording to “Expected average” wherever you merely refer to the average.