Well, you asked for DUMB ideas, so here’s mine. It has the advantage that I’m sure no one else will suggest it. This is based on an accidental discovery (so far as I know, unpublished) that one can compare two arbitrary documents for similarity (even if they are in different word-processor formats) by running them both through a recognizer built out of a random state machine and comparing bit masks of all the states traversed. The more common they are, the more states will be traversed in both.
So, lets assume we have a panel of highly rational individuals which are our control group. We generate a random multiple-choice questionnaire consisting of nonsensical questions and answers. Things like:
1) How Green is the Smell of Bacon?
a) 7.5
b) Neon
c) Introspection
d) Larger
You then do a correlation over how your panel of experts chose their answers and see if there is a common pattern. You then score students who take the test based on how similar to the common pattern they are.
Assuming this idea works at all, the advantage of this is that it would be extremely difficult to game. The disadvantage would be that it would penalize those who are significantly more rational than the ‘norm’. It would probably also require the panel to be similar to each other in cognition. There is also the general problem of not knowing if you’re really testing for what you think you’re testing.
Frankly, I don’t know if I’d be more happy if this was tested and shown to be workable, or if it turned out to be a really stupid idea.
What I observed when doing an MMPI was that it seemed altogether gameable. I believe I have more than enough knowledge about psychology, including the type of metrics that MMPI uses, to more or less choose whatever result I desired.
I’ve actually proposed something like this to test for personality type. The main reason it never got implemented is there isn’t really a good, workable theory of persistent personality.
It sounds altogether too much like the famous beauty pagent, with a bit of “guess the teachers answer” and radomly generated poetry thrown in for good measure.
Frankly, I don’t know if I’d be more happy if this was tested and shown to be workable, or if it turned out to be a really stupid idea.
I know I’d be far happier if it was shown to be a really stupid idea. I have a hunch, however, that a correlation of the kind you hypothesis would exist. The part that scares me is that there could well be more than one style of thinking of equal merit, with one being far more common than the other. Naturally the suspicion that I’d end up in the minority and downgraded for it is troublesome. There is more than enough of that sort of bias in schools already!
Upvoted for being the right kind of idea and incidently my answer to the example question is a) 7.5. The other three make absolutely no sense while I acknowledge that there is a possibility (though it is improbable) that the way the brain functions could make a quantisation of said greeness at least have some meaning.
Well, you asked for DUMB ideas, so here’s mine. It has the advantage that I’m sure no one else will suggest it. This is based on an accidental discovery (so far as I know, unpublished) that one can compare two arbitrary documents for similarity (even if they are in different word-processor formats) by running them both through a recognizer built out of a random state machine and comparing bit masks of all the states traversed. The more common they are, the more states will be traversed in both.
So, lets assume we have a panel of highly rational individuals which are our control group. We generate a random multiple-choice questionnaire consisting of nonsensical questions and answers. Things like:
1) How Green is the Smell of Bacon?
a) 7.5
b) Neon
c) Introspection
d) Larger
You then do a correlation over how your panel of experts chose their answers and see if there is a common pattern. You then score students who take the test based on how similar to the common pattern they are.
Assuming this idea works at all, the advantage of this is that it would be extremely difficult to game. The disadvantage would be that it would penalize those who are significantly more rational than the ‘norm’. It would probably also require the panel to be similar to each other in cognition. There is also the general problem of not knowing if you’re really testing for what you think you’re testing.
Frankly, I don’t know if I’d be more happy if this was tested and shown to be workable, or if it turned out to be a really stupid idea.
NOT CRAZY ENOUGH! We need EVEN STUPIDER ideas!
(Voted up for being the best try so far, though.)
I think that this resembles the MMPI methodology. http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personality_Inventory
What is the MMPI supposed to test?
There are similarities.
What I observed when doing an MMPI was that it seemed altogether gameable. I believe I have more than enough knowledge about psychology, including the type of metrics that MMPI uses, to more or less choose whatever result I desired.
I’ve actually proposed something like this to test for personality type. The main reason it never got implemented is there isn’t really a good, workable theory of persistent personality.
That scares me!
It sounds altogether too much like the famous beauty pagent, with a bit of “guess the teachers answer” and radomly generated poetry thrown in for good measure.
I know I’d be far happier if it was shown to be a really stupid idea. I have a hunch, however, that a correlation of the kind you hypothesis would exist. The part that scares me is that there could well be more than one style of thinking of equal merit, with one being far more common than the other. Naturally the suspicion that I’d end up in the minority and downgraded for it is troublesome. There is more than enough of that sort of bias in schools already!
Upvoted for being the right kind of idea and incidently my answer to the example question is a) 7.5. The other three make absolutely no sense while I acknowledge that there is a possibility (though it is improbable) that the way the brain functions could make a quantisation of said greeness at least have some meaning.
When I look at my question there, the only answer that seems appropriate is ‘Introspection’ as that’s at least a step towards an answer.