This is an interesting project! I like the part where you ask people for feedback. Perhaps this is an opportunity to do some A/B testing… for example, if you suspect that “multiple learning styles” is nonsense, you could make a version that includes it, a version that does not, and compare the user satisfaction. This kind of experiment is probably never done with human teachers, because they wouldn’t be so flexible at changing their pedagogical methods just because you told them to.
Teachers should use rewards and punishments to shape student behavior.
As an immediate reaction, I feel uncomfortable about the idea of an LLM using rewards and punishments to shape my behavior. Multiple reasons.
For starters, there are approaches even within behaviorism which say that punishments often do more harm than good. (Read Don’t shoot the dog for more details.) One reason is that the rewards and punishments always apply to several things at once: if you reward/punish someone for giving you a correct/wrong answer, are you really rewarding/punishing the answer, or the fact that they speak clearly, or the fact that they communicate with you at all, or something else? For example, a strict teacher may congratulate themselves for punishing all the wrong answers, failing to consider that as a side effect the students now hate the teacher, hate the subject, hate the school, and perhaps hate the idea of learning things in general… because all these things are now strongly associated with the punishments in their minds.
Second, I do not approve the idea of being punished by a machine. Because this does not feel compatible with alignment, but also because the LLMs sometimes hallucinate.
...uhm, let’s change it to “provide feedback” perhaps?
Social Constructivism [...] suggests that we learn best through working with others.
I guess there is not much the AI can do about this. This is something for the designers of the educational system to consider (and they could use the AI to advise them), but once the AI is talking 1:1 to the students, the idea that “it would be better to have more students there at the same time” is not actionable.
As a general remark, I would recommend making sure that we do not mistake the map for the territory. The educational theories are… well, theories. It is interesting to know how much the things the AI does align with the theories, but it is more important to know how much the things the AI does contribute to the actual learning. Perhaps the students taught by the AI should afterwards be tested and compared to alternative groups taught by an AI with a different prompt, by a human teacher, just reading a textbook alone, or doing nothing.
I also wonder whether trying all these principles at the same time might be too much. Like, if someone comes with an idea that scores high on some of those principles, but very low on one of them (for example: learning 1:1 with the AI tutor already scores low on the “learning in group with others” dimension), when should the idea be tried, and when should it be rejected? Maybe by requiring the solution to fit all the theories we discard some potentially good ideas? Perhaps we could separate the principles into “must be” and “nice to have”.
This is an interesting project! I like the part where you ask people for feedback. Perhaps this is an opportunity to do some A/B testing… for example, if you suspect that “multiple learning styles” is nonsense, you could make a version that includes it, a version that does not, and compare the user satisfaction. This kind of experiment is probably never done with human teachers, because they wouldn’t be so flexible at changing their pedagogical methods just because you told them to.
As an immediate reaction, I feel uncomfortable about the idea of an LLM using rewards and punishments to shape my behavior. Multiple reasons.
For starters, there are approaches even within behaviorism which say that punishments often do more harm than good. (Read Don’t shoot the dog for more details.) One reason is that the rewards and punishments always apply to several things at once: if you reward/punish someone for giving you a correct/wrong answer, are you really rewarding/punishing the answer, or the fact that they speak clearly, or the fact that they communicate with you at all, or something else? For example, a strict teacher may congratulate themselves for punishing all the wrong answers, failing to consider that as a side effect the students now hate the teacher, hate the subject, hate the school, and perhaps hate the idea of learning things in general… because all these things are now strongly associated with the punishments in their minds.
Second, I do not approve the idea of being punished by a machine. Because this does not feel compatible with alignment, but also because the LLMs sometimes hallucinate.
...uhm, let’s change it to “provide feedback” perhaps?
I guess there is not much the AI can do about this. This is something for the designers of the educational system to consider (and they could use the AI to advise them), but once the AI is talking 1:1 to the students, the idea that “it would be better to have more students there at the same time” is not actionable.
As a general remark, I would recommend making sure that we do not mistake the map for the territory. The educational theories are… well, theories. It is interesting to know how much the things the AI does align with the theories, but it is more important to know how much the things the AI does contribute to the actual learning. Perhaps the students taught by the AI should afterwards be tested and compared to alternative groups taught by an AI with a different prompt, by a human teacher, just reading a textbook alone, or doing nothing.
I also wonder whether trying all these principles at the same time might be too much. Like, if someone comes with an idea that scores high on some of those principles, but very low on one of them (for example: learning 1:1 with the AI tutor already scores low on the “learning in group with others” dimension), when should the idea be tried, and when should it be rejected? Maybe by requiring the solution to fit all the theories we discard some potentially good ideas? Perhaps we could separate the principles into “must be” and “nice to have”.