I am lumping them together since if you believe AGI isn’t that impactful, then much argumentation around AI and alignment doesn’t matter at all. Obviously, there is the bias argument that you responded to around doom, but there is another prong to that argument.
It seems—at least to to me—like the argumentation around AI and alignment would be a good source of new beliefs, since I can’t figure it all out on my own. People also seem to be figuring out new things fairly regularly.
Between those two things, I’m struggling to understand what it would be like to assert a static belief “field X doesn’t matter”, in way that is reasonably grounded in what is coming out of field X, particularly as the field X evolves.
Like, if I believe that AI Alignment won’t matter much and I use that to write off the field of AI Alignment, it feels like I’m either pre-emptively ignoring potentially relevant information, or I’m making a claim that I have some larger grounded insights into how the field is confused.
I get that we’re all bounded and don’t have the time or energy or inclination to engage with every field and every argument within those fields. If the claim was something like “I don’t see AI alignment as a personal priority to invest my time/energy in” that feels completely fine to me—I think I would have nodded and kept scrolling rather than writing something.
Worrying about where other people were spending their energy is also fine! If it were me, I’d want to be confident I was most informed about something they’d all missed, otherwise I’d be in a failure mode I sometimes get into where I’m on a not-so-well-grounded hamster wheel of worrying.
I guess I’m trying to tease apart the cases where you are saying “I have a belief that I’m not willing to spend time/energy to update” vs “I also believe that no updates are coming and so I’m locking in my current view based on that meta-belief”.
I’m also curious!
If you’ve seen something that would tip my evidential scales the whole way to “the field is built on sketchy foundations, with probability that balances out the expected value of doom if AI alignment is actually a problem”, then I’d really like to know! Although I haven’t seen anything like that yet.
And I’m also curious about what prongs I might be missing around the “people following their expected values to prevent P(doom) look like folks who were upset about nothing in the timelines where we all survived to be having after-the-fact discussions about them” ;)
Like, if I believe that AI Alignment won’t matter much and I use that to write off the field of AI Alignment, it feels like I’m either pre-emptively ignoring potentially relevant information, or I’m making a claim that I have some larger grounded insights into how the field is confused.
I think the key here is that if AGI only is something like say the internet, or perhaps the industrial revolution, then AGI alignment doesn’t matter much. A lot of the field of AGI alignment only really makes sense if the impact of AGI is very, very large.
I am lumping them together since if you believe AGI isn’t that impactful, then much argumentation around AI and alignment doesn’t matter at all. Obviously, there is the bias argument that you responded to around doom, but there is another prong to that argument.
It seems—at least to to me—like the argumentation around AI and alignment would be a good source of new beliefs, since I can’t figure it all out on my own. People also seem to be figuring out new things fairly regularly.
Between those two things, I’m struggling to understand what it would be like to assert a static belief “field X doesn’t matter”, in way that is reasonably grounded in what is coming out of field X, particularly as the field X evolves.
Like, if I believe that AI Alignment won’t matter much and I use that to write off the field of AI Alignment, it feels like I’m either pre-emptively ignoring potentially relevant information, or I’m making a claim that I have some larger grounded insights into how the field is confused.
I get that we’re all bounded and don’t have the time or energy or inclination to engage with every field and every argument within those fields. If the claim was something like “I don’t see AI alignment as a personal priority to invest my time/energy in” that feels completely fine to me—I think I would have nodded and kept scrolling rather than writing something.
Worrying about where other people were spending their energy is also fine! If it were me, I’d want to be confident I was most informed about something they’d all missed, otherwise I’d be in a failure mode I sometimes get into where I’m on a not-so-well-grounded hamster wheel of worrying.
I guess I’m trying to tease apart the cases where you are saying “I have a belief that I’m not willing to spend time/energy to update” vs “I also believe that no updates are coming and so I’m locking in my current view based on that meta-belief”.
I’m also curious!
If you’ve seen something that would tip my evidential scales the whole way to “the field is built on sketchy foundations, with probability that balances out the expected value of doom if AI alignment is actually a problem”, then I’d really like to know! Although I haven’t seen anything like that yet.
And I’m also curious about what prongs I might be missing around the “people following their expected values to prevent P(doom) look like folks who were upset about nothing in the timelines where we all survived to be having after-the-fact discussions about them” ;)
I think the key here is that if AGI only is something like say the internet, or perhaps the industrial revolution, then AGI alignment doesn’t matter much. A lot of the field of AGI alignment only really makes sense if the impact of AGI is very, very large.