I asked GPT-4 to write a list of desiderata for a naturalistic (i.e., scientific) theory of ethics: https://chat.openai.com/share/1025a325-30e0-457c-a1ed-9d6b3f23eb5e. It made some mistakes but in other regards surprised me with the quality of its philosophy of science and meta-ethics.
The mistake that jumped out for me was “6. Robustness and Flexibility”:
The ethical theory should be robust and flexible, meaning it should be able to accommodate new information and adapt to different contexts and conditions. As our scientific knowledge evolves, the theory should be able to incorporate new insights without losing its coherence or practical applicability.
According to a Popperian criterion for a good scientific theory, falsifiability, GPT-4 should have made the opposite point: the scientific theory of ethics should fit with other theories (of rationality, evolution, consciousness, etc.) in such a way that an update in one of those theories requires a revision in the theory of ethics as well. So, all theories should be mutually “brittle”, which reflects falsifiability.
When I asked GPT-4 to criticise the desiderata for a scientific theory of ethics with Popperian criteria in mind, it found this problem with item 6, as well as several other (albeit smaller) problems that I didn’t think about, and probably wouldn’t come up with even if tried quite hard!
Then, when I asked GPT-4 to rewrite the desiderata considering the criticisms, the flip side of LLM’s self-consistency (Karpathy mentions it in the “State of GPT” talk: youtube.com/watch?v=bZQun8Y4L2A) has come up: the LLM was reluctant to turn 180 degrees upon recognising its own mistake and instead tried to “amend” all desiderata, preserving their original chich actually made them only worse). The supposed remedy for this problem is the “Tree of Thoughts” LMCA: arxiv.org/abs/2305.10601.
I asked GPT-4 to write a list of desiderata for a naturalistic (i.e., scientific) theory of ethics: https://chat.openai.com/share/1025a325-30e0-457c-a1ed-9d6b3f23eb5e. It made some mistakes but in other regards surprised me with the quality of its philosophy of science and meta-ethics.
The mistake that jumped out for me was “6. Robustness and Flexibility”:
According to a Popperian criterion for a good scientific theory, falsifiability, GPT-4 should have made the opposite point: the scientific theory of ethics should fit with other theories (of rationality, evolution, consciousness, etc.) in such a way that an update in one of those theories requires a revision in the theory of ethics as well. So, all theories should be mutually “brittle”, which reflects falsifiability.
When I asked GPT-4 to criticise the desiderata for a scientific theory of ethics with Popperian criteria in mind, it found this problem with item 6, as well as several other (albeit smaller) problems that I didn’t think about, and probably wouldn’t come up with even if tried quite hard!
Then, when I asked GPT-4 to rewrite the desiderata considering the criticisms, the flip side of LLM’s self-consistency (Karpathy mentions it in the “State of GPT” talk: youtube.com/watch?v=bZQun8Y4L2A) has come up: the LLM was reluctant to turn 180 degrees upon recognising its own mistake and instead tried to “amend” all desiderata, preserving their original chich actually made them only worse). The supposed remedy for this problem is the “Tree of Thoughts” LMCA: arxiv.org/abs/2305.10601.