I do worry that thought experiments involving Omega can lead decision theory research down wrong paths (for example by giving people misleading intuitions), and try to make sure the ones I create or pay attention to are not just arbitrary thought experiments but illuminate some aspect of real world decision making problems that we (or an AI) might face. Unfortunately, this relies largely on intuition and it’s hard to say what exactly is a valid or useful thought experiment and what isn’t, except maybe in retrospect.
I do worry that thought experiments involving Omega can lead decision theory research down wrong paths (for example by giving people misleading intuitions), and try to make sure the ones I create or pay attention to are not just arbitrary thought experiments but illuminate some aspect of real world decision making problems that we (or an AI) might face. Unfortunately, this relies largely on intuition and it’s hard to say what exactly is a valid or useful thought experiment and what isn’t, except maybe in retrospect.