Oracle-genie-sovereign is a really useful distinction that I think I (and probably many others) have avoided using mainly because “genie” sounds unprofessional/unacademic. This is a real shame, and a good lesson for future terminology.
After rereading the chapter in Superintelligence, it seems to me that “genie” captures something akin to act-based agents. Do you think that’s the main way to use this concept in the current state of the field, or do you have other applications in mind?
Ah, yeah, that’s a great point. Although I think act-based agents is a pretty bad name, since those agents may often carry out a whole bunch of acts in a row—in fact, I think that’s what made me overlook the fact that it’s pointing at the right concept. So not sure if I’m comfortable using it going forward, but thanks for pointing that out.
Perhaps the lesson is that terminology that is acceptable in one field (in this case philosophy) might not be suitable in another (in this case machine learning).
I don’t think that even philosophers take the “genie” terminology very seriously. I think the more general lesson is something like: it’s particularly important to spend your weirdness points wisely when you want others to copy you, because they may be less willing to spend weirdness points.
Oracle-genie-sovereign is a really useful distinction that I think I (and probably many others) have avoided using mainly because “genie” sounds unprofessional/unacademic. This is a real shame, and a good lesson for future terminology.
After rereading the chapter in Superintelligence, it seems to me that “genie” captures something akin to act-based agents. Do you think that’s the main way to use this concept in the current state of the field, or do you have other applications in mind?
Ah, yeah, that’s a great point. Although I think act-based agents is a pretty bad name, since those agents may often carry out a whole bunch of acts in a row—in fact, I think that’s what made me overlook the fact that it’s pointing at the right concept. So not sure if I’m comfortable using it going forward, but thanks for pointing that out.
Perhaps the lesson is that terminology that is acceptable in one field (in this case philosophy) might not be suitable in another (in this case machine learning).
I don’t think that even philosophers take the “genie” terminology very seriously. I think the more general lesson is something like: it’s particularly important to spend your weirdness points wisely when you want others to copy you, because they may be less willing to spend weirdness points.
Is that from Superintelligence? I googled it, and that was the most convincing result.
Yepp.