It seems much more likely that GPT-4 is not correctly reasoning through the options to arrive at a decision by any coherent decision theory. It is just not reliable enough at carrying out this sort of reasoning. This can be seen by the multiple simple errors it makes during your test prompts, and also in many other logical and numerical reasoning tasks.
It seems much more likely that GPT-4 is not correctly reasoning through the options to arrive at a decision by any coherent decision theory. It is just not reliable enough at carrying out this sort of reasoning. This can be seen by the multiple simple errors it makes during your test prompts, and also in many other logical and numerical reasoning tasks.
Maybe. It just seemed weird that when they coincided, it gave perfect reasoning.