I am unsure how obvious this result is in-advance, my guess is there are at least some people who would have correctly predicted it and for the right reasons. However, I really love a lot of things about this post: first because you answer a question ML researchers do have disagreements over, second because you coded two small games for humans to play to help understand what’s going on, third because you collect novel data, and fourth because you present it so clearly and readably.
I gained a better understanding of how language-models work from reading this post and playing the games, and I’d love to see more posts answering open questions through this kind of simple experimental work.
Curated.
I am unsure how obvious this result is in-advance, my guess is there are at least some people who would have correctly predicted it and for the right reasons. However, I really love a lot of things about this post: first because you answer a question ML researchers do have disagreements over, second because you coded two small games for humans to play to help understand what’s going on, third because you collect novel data, and fourth because you present it so clearly and readably.
I gained a better understanding of how language-models work from reading this post and playing the games, and I’d love to see more posts answering open questions through this kind of simple experimental work.