epistemic status: speculative, probably simplistic and ill defined
Someone asked me “What will I do once we have AGI?”
I generally define the AGI-era starting at the point where all economically valuable tasks can be performed by AIs at a lower cost than a human (at subsistance level, including buying any available augmentations for the human). This notably excludes:
1) any tasks that humans can do that still provide value at the margin (ie. the caloric cost of feeding that human while they’re working vs while they’re not working rather than while they’re not existing)
2) things that are not “tasks”, such as:
a) caring about the internal experience of the service provider (ex.: wanting a DJ that feels human emotions regardless of its actions) --> although, maybe you could include that in the AGI definition too. but what if you value having a DJ be exactly a human? then the best an AGI could do is 3D print a human or something like that. or maybe you’re even more specific, and you want a “pre-singulatarian natural human”, in which case AGI seems impossible by (very contrived) definition.
b) the value of the memories encoded in human brains
c) the value of doing scientific experiments on humans
For my answer to the question, I wanted to say something like, think about what I should do with my time for a long time, and keep my options open (ex.: avoid altering my mind in ways I don’t understand the consequences well). But then, that seems like something that might be economically useful to sell, so using the above definition, it seems like I should have AI system that are able to do that better/cheaper than me (unless I intrinsically didn’t want that, or something like that). So maybe I have AI systems computing that for me and keeping me posted with advice while I do whatever I want.
But maybe I can still do work that is useful at the margin, as per (1), and so would probably do that. But what if even that wasn’t worth the marginal caloric cost, and it was better to feed those calories into AI systems?
(2) is a bit complex, but probably(?) wouldn’t impact the answer to the initial question much.
So, what would I do? I don’t know. Main thing that comes to mind is observe how the worlds unfold (and listen to what the AGIs are telling me).
But maybe “AGI” shouldn’t be defined as “aligned AGI”. Maybe a better definition of AGI is like “outperforming humans at all games/tasks that are well defined” (ie. where humans don’t have a comparative advantage just by knowing what humans value). In which case, my answer would be “alignment research” (assuming it’s not “die”).
epistemic status: speculative, probably simplistic and ill defined
Someone asked me “What will I do once we have AGI?”
I generally define the AGI-era starting at the point where all economically valuable tasks can be performed by AIs at a lower cost than a human (at subsistance level, including buying any available augmentations for the human). This notably excludes:
1) any tasks that humans can do that still provide value at the margin (ie. the caloric cost of feeding that human while they’re working vs while they’re not working rather than while they’re not existing)
2) things that are not “tasks”, such as:
a) caring about the internal experience of the service provider (ex.: wanting a DJ that feels human emotions regardless of its actions) --> although, maybe you could include that in the AGI definition too. but what if you value having a DJ be exactly a human? then the best an AGI could do is 3D print a human or something like that. or maybe you’re even more specific, and you want a “pre-singulatarian natural human”, in which case AGI seems impossible by (very contrived) definition.
b) the value of the memories encoded in human brains
c) the value of doing scientific experiments on humans
For my answer to the question, I wanted to say something like, think about what I should do with my time for a long time, and keep my options open (ex.: avoid altering my mind in ways I don’t understand the consequences well). But then, that seems like something that might be economically useful to sell, so using the above definition, it seems like I should have AI system that are able to do that better/cheaper than me (unless I intrinsically didn’t want that, or something like that). So maybe I have AI systems computing that for me and keeping me posted with advice while I do whatever I want.
But maybe I can still do work that is useful at the margin, as per (1), and so would probably do that. But what if even that wasn’t worth the marginal caloric cost, and it was better to feed those calories into AI systems?
(2) is a bit complex, but probably(?) wouldn’t impact the answer to the initial question much.
So, what would I do? I don’t know. Main thing that comes to mind is observe how the worlds unfold (and listen to what the AGIs are telling me).
But maybe “AGI” shouldn’t be defined as “aligned AGI”. Maybe a better definition of AGI is like “outperforming humans at all games/tasks that are well defined” (ie. where humans don’t have a comparative advantage just by knowing what humans value). In which case, my answer would be “alignment research” (assuming it’s not “die”).