How insane would it be to just not have a control group?
Pretty insane in my opinion. I can’t imagine anything I would grade more harshly than not having a control except ethics violations.
Besides, don’t most university psychology experiments with volunteers keep the protocol secret throughout the whole experiment and then debrief at the end? (Or sometimes even lie about the protocol to avoid skewing the results?)
Alternatively, have you thought about doing a crossover-style design?
Take group A and group B. Group A plays your game, and then takes the test. Group B either just takes the test or goes through some traditional education lesson (or whatever else you want for your control) and then takes the test. Next, group A does the traditional education, group B does the game, and both take part 2 of the test.
That way, everyone gets to play the game at least, though it means they’re there for twice as long. Do you think you could pitch this in a way that is better than the “Maybe you play a game, maybe you don’t” option?
You could potentially derive additional research value from this as well. If group A does better on Test Part 2, then your game would be shown to be a better way of acclimating people to traditional education on the subject or something like that (I’m sure you can draw a better conclusion or phrase this better).
Just some thoughts. Also, make sure you write up a grading rubric ahead of time (or ideally, have someone else do it) and then have someone who knows nothing (or as little as possible) about the experiment (and especially the subjects) grade the answers to avoid researcher bias.
Pretty insane in my opinion. I can’t imagine anything I would grade more harshly than not having a control except ethics violations.
I think there might be reasonable theoretical grounds for it in this case, though? If I was testing say a medical treatment or self-help technique, then yes, there should absolutely be a control group since some people might get better on their own or just do better for a while because the self-help technique gave them extra confidence.
But suppose I give people a pre-test, have them play for some minimum time, and then fill out the post-test when they’re done. I don’t see much in the way for random chance to confound things here: either they know the things needed for solving the tasks, or they don’t. If they didn’t know enough to solve the problems on the first try, they’re not going to suddenly acquire that knowledge in between.
Besides, don’t most university psychology experiments with volunteers keep the protocol secret throughout the whole experiment and then debrief at the end?
To some extent, but usually they still give some brief description of it beforehand, to attract people.
Alternatively, have you thought about doing a crossover-style design?
But suppose I give people a pre-test, have them play for some minimum time, and then fill out the post-test when they’re done. I don’t see much in the way for random chance to confound things here: either they know the things needed for solving the tasks, or they don’t. If they didn’t know enough to solve the problems on the first try, they’re not going to suddenly acquire that knowledge in between.
If I get a problem I can’t solve I can Google afterwards and read about how to solve the problem. Even if you lock me in a dark room, there the possibility that I recover forgotten knowledge if you give my brain a few hours.
The pretest itself also provides practice. You need a control group, but it would be possible to give the control group nothing to do.
Pretty insane in my opinion. I can’t imagine anything I would grade more harshly than not having a control except ethics violations.
Besides, don’t most university psychology experiments with volunteers keep the protocol secret throughout the whole experiment and then debrief at the end? (Or sometimes even lie about the protocol to avoid skewing the results?)
Alternatively, have you thought about doing a crossover-style design?
Take group A and group B. Group A plays your game, and then takes the test. Group B either just takes the test or goes through some traditional education lesson (or whatever else you want for your control) and then takes the test. Next, group A does the traditional education, group B does the game, and both take part 2 of the test.
That way, everyone gets to play the game at least, though it means they’re there for twice as long. Do you think you could pitch this in a way that is better than the “Maybe you play a game, maybe you don’t” option?
You could potentially derive additional research value from this as well. If group A does better on Test Part 2, then your game would be shown to be a better way of acclimating people to traditional education on the subject or something like that (I’m sure you can draw a better conclusion or phrase this better).
Just some thoughts. Also, make sure you write up a grading rubric ahead of time (or ideally, have someone else do it) and then have someone who knows nothing (or as little as possible) about the experiment (and especially the subjects) grade the answers to avoid researcher bias.
I think there might be reasonable theoretical grounds for it in this case, though? If I was testing say a medical treatment or self-help technique, then yes, there should absolutely be a control group since some people might get better on their own or just do better for a while because the self-help technique gave them extra confidence.
But suppose I give people a pre-test, have them play for some minimum time, and then fill out the post-test when they’re done. I don’t see much in the way for random chance to confound things here: either they know the things needed for solving the tasks, or they don’t. If they didn’t know enough to solve the problems on the first try, they’re not going to suddenly acquire that knowledge in between.
To some extent, but usually they still give some brief description of it beforehand, to attract people.
That’s a good idea, thanks.
If I get a problem I can’t solve I can Google afterwards and read about how to solve the problem. Even if you lock me in a dark room, there the possibility that I recover forgotten knowledge if you give my brain a few hours.
The pretest itself also provides practice. You need a control group, but it would be possible to give the control group nothing to do.