Very informative, lots I can agree with, and plenty to chew on. I love what I see as your intentions and values on a human level, and how those affect your approach, which we seem to be fundamentally “aligned” on.
During my initial read, I noted some differences in our perspectives — not disagreements per se, likely just a product of the inherent incompleteness of our respective “training environments”. Your perspective certainly helped fill in many gaps in my perspective. And while I don’t want to be presumptuous, my intuition is that our perspectives are potentially complementary, so a discussion could be fruitful. I’ll start with the item that seems to be of greatest consequence, at least potentially.
I think it would be in the right ballpark to say that your overarching objective is to ensure that AGI (and especially super-human variants) is developed, deployed, and managed in a way that is safe, responsible, and ultimately increases altruism. And that the primary risk that concerns you is that a malign AGI would be released into the world that cannot be controlled, whether due to carelessness or malign purposes. Which I believe is a worthy objective and a valid concern.
In my mind, there is another, but far greater existential threat which I am concerned about, and believe it poses immediate threats that will only grow, perhaps exponentially, as we approach AGI and beyond. Please hear me out. I’m not interested in being defeatist or alarmist. Like you, I am only interested in pragmatic solutions, and do have some potential ideas to that end.
Keeping that in mind, I believe the far greater threat is humans — specifically, the degree to which we are not “aligned” as you use that word. Obviously, there are those who are consciously and intentionally malicious, and that’s always a concern. However, that’s not my primary concern. It has become abundantly clear to me in recent years the gravest danger is posed by those who are unwittingly misaligned, or can be manipulated into being so, especially in large numbers. And actually, being more even-handed and accurate, this misalignment is something in which we all share blame and responsibility to some extent, and repairing it will require collective action from all of us.
Even at current levels of capability, AI can be used by humans for malicious purposes. And I am concerned about the impacts that will have before even getting close to AGI, and that those could severely jeopardize your ability to achieve your objectives.
Additionally, in my mind, it doesn’t matter how pure and noble the AI agents are that you develop, even if you somehow beat everyone else to the finish line. Because as long as humans remain as misaligned as we currently are, we will be too divided to protect ourselves, and the worst of us will almost certainly infiltrate and corrupt the AGI agent population for their malign purposes. So the fundamental threat we need to address urgently and before AGI is reducing the degree of misalignment among ourselves as human beings.
For the record, I do have some ideas about how we might achieve that. And at least one relatively concrete way that could help harness the power of AI for that purpose, that may at least be worth harvesting for parts if nothing else.
I’m open to discussing any of this further, but for now, I’d really love to hear your thoughts, and would consider that an honor.
Very informative, lots I can agree with, and plenty to chew on. I love what I see as your intentions and values on a human level, and how those affect your approach, which we seem to be fundamentally “aligned” on.
During my initial read, I noted some differences in our perspectives — not disagreements per se, likely just a product of the inherent incompleteness of our respective “training environments”. Your perspective certainly helped fill in many gaps in my perspective. And while I don’t want to be presumptuous, my intuition is that our perspectives are potentially complementary, so a discussion could be fruitful. I’ll start with the item that seems to be of greatest consequence, at least potentially.
I think it would be in the right ballpark to say that your overarching objective is to ensure that AGI (and especially super-human variants) is developed, deployed, and managed in a way that is safe, responsible, and ultimately increases altruism. And that the primary risk that concerns you is that a malign AGI would be released into the world that cannot be controlled, whether due to carelessness or malign purposes. Which I believe is a worthy objective and a valid concern.
In my mind, there is another, but far greater existential threat which I am concerned about, and believe it poses immediate threats that will only grow, perhaps exponentially, as we approach AGI and beyond. Please hear me out. I’m not interested in being defeatist or alarmist. Like you, I am only interested in pragmatic solutions, and do have some potential ideas to that end.
Keeping that in mind, I believe the far greater threat is humans — specifically, the degree to which we are not “aligned” as you use that word. Obviously, there are those who are consciously and intentionally malicious, and that’s always a concern. However, that’s not my primary concern. It has become abundantly clear to me in recent years the gravest danger is posed by those who are unwittingly misaligned, or can be manipulated into being so, especially in large numbers. And actually, being more even-handed and accurate, this misalignment is something in which we all share blame and responsibility to some extent, and repairing it will require collective action from all of us.
Even at current levels of capability, AI can be used by humans for malicious purposes. And I am concerned about the impacts that will have before even getting close to AGI, and that those could severely jeopardize your ability to achieve your objectives.
Additionally, in my mind, it doesn’t matter how pure and noble the AI agents are that you develop, even if you somehow beat everyone else to the finish line. Because as long as humans remain as misaligned as we currently are, we will be too divided to protect ourselves, and the worst of us will almost certainly infiltrate and corrupt the AGI agent population for their malign purposes. So the fundamental threat we need to address urgently and before AGI is reducing the degree of misalignment among ourselves as human beings.
For the record, I do have some ideas about how we might achieve that. And at least one relatively concrete way that could help harness the power of AI for that purpose, that may at least be worth harvesting for parts if nothing else.
I’m open to discussing any of this further, but for now, I’d really love to hear your thoughts, and would consider that an honor.