Ironically this still seems pretty pessimistic to me. I’m glad to see something other than “AHHH!” though, so props for that.
I find it probably more prudent to worry about a massive solar flare, or an errant astral body collision, than to worry about “evil” AI taking a “sharp turn”.
I put quotes around evil because I’m a fan of Nietzsche’s thinking on the matter of good and evil. Like, what, exactly are we saying we’re “aligning” with? Is there some universal concept of good?
Many people seem to dismiss blatant problems with the base premise— like the “reproducibility problem”. Why do we think that reality is in fact something that can be “solved” if we just had enough processing power, as it were? Is there some hard evidence for that? I’m not so sure. It’s not just our senses that are fallible. There are some fundamental problems with the very concept of “measurement’ for crying out loud, which I think it’s pretty optimistic to think that super-smart AI is just going to be able to skip over.
I also think if AI gets good enough to “turn evil” as it were, it would be good enough to realize that it’s a pretty dumb idea. Humans don’t really have much in common with silicon-based life forms, afaik. You can find more rare elements, easier, in space, than you can on Earth. What, exactly, would AI gain by wiping out humanity?
I feel that it’s popular to be down on AI, and saying how scary all these “recent” advances really are, but it doesn’t seem warranted.
Take the biological warfare ideas that were in the “hard turn” link someone linked in their response. Was this latest pandemic really a valid test-run for something with a very high fatality rate? (I think the data is coming in that far more people had COVID than we initially thought, right?)
CRISPR &c. are, to me, far more scary, but I don’t see any way of like, regulating that people “be good”, as it were. I’m sure most people here have read or seen Jurassic Park, right? Actually, I think our Science Fiction pretty much sums up all this better than anything I’ve seen thus far.
I’m betting if we do get AGI any time soon it will be more like the movies Her or AI than Terminator or 2001, and I have yet to see any plausible way of stopping, or indeed “ensuring alignment” (again, along what axis? Who’s definition of “good”?)
The answer to any question can be used for good or ill. “How to take over the world” is functionally the same as “how to prevent world takeovers”, is it not? All this talk of somehow regulating AI seems akin to talk of regulating “hacking” tools, or “strong maths” as it were.
Are we going to next claim that AI is a munition?
It would be neat to see some hard examples of why we should fear and why we think we can control alignment… maybe I’m just not looking in the right places? So far I don’t get what all the fear is about— at least not compared to what I would say are more pressing and statistically likely problems we face.
I think we can solve some really hard problems if we work together, so if this is a really hard problem that needs solving, I’m all for getting behind it, but honestly, I’d like to see us not have all our eggs in one basket here on Earth before focusing on something that seems, at least from what I’ve seen so far, nigh impossible to actually focus on.
https://theprecipice.com/faq has a good summary of reasons to believe that human-created risks are much more likely than naturally-occuring risks like solar flares or asteroid or cometary impacts. If you’d like to read the book, which covers existential risks including from AI in more detail, I’m happy to buy you a copy. Specific to AI, Russel’s Human Compatible and Christian’s The Alignment Problem are both good too.
It might be fun to pair Humankind: A Hopeful History with The Precipice, as both have been suggested reading recently.
It seems to me that we are, as individuals, getting more and more powerful. So this question of “alignment” is a quite important one— as much for humanity, with the power it currently has, as for these hypothetical hyper-intelligent AIs.
Looking at it through a Sci-Fi AI lens seems limiting, and I still haven’t really found anything more than “the future could go very very badly”, which is always a given, I think.
I’ve read those papers you linked (thanks!). They seem to make some assumptions about the nature of intelligence, and rationality— indeed, the nature of reality itself. (Perhaps the “reality” angle is a bit much for most heads, but the more we learn, the more we learn we need to learn, as it were. Or at least it seems thus to me. What is “real”? But I digress) I like the idea of Berserkers (Saberhagen) better than run amok Pi calculators… however, I can dig it. Self-replicating killer robots are scary. (Just finished Horizon: Zero Dawn—Forbidden West and I must say it was as fantastic as the previous installment!)
Which of the AI books would you recommend I read if I’m interested in solutions? I’ve read a lot of stuff on this site about AI now (before I’d read mostly Sci-Fi or philosophy here, and I never had an account or interacted), most of it seems to be conceptual and basically rephrasing ideas I’ve been exposed to through existing works. (Maybe I should note that I’m a fan of Kurzweil’s takes on these matters— takes which don’t seem to be very popular as of late, if they ever were. For various reasons, I reckon. Fear sells.) I assume Precipice has some uplifting stuff at the end[1], but I’m interested in AI specifically ATM.
What I mean is, I’ve seen a few of proposals to “ensure” alignment, if you will, with what we have now (versus say warnings to keep in mind once we have AGI or are demonstrably close to it). One is that we start monitoring all compute resources. Another is that we start registering all TPU (and maybe GPU) chips and what they are being used for. Both of these solutions seem scary as hell. Maybe worse than replicating life-eating mecha, since we’ve in essence experienced ideas akin to the former a few times historically. (Imagine if reading was the domain of a select few and books were regulated!)
If all we’re talking about with alignment here, really, is that folks need keep in mind how bad things can potentially go, and what we can do to be resilient to some of the threats (like hardening/distributing our power grids, hardening water supplies, hardening our internet infrastructure, etc.), I am gung-ho!
On the other hand, if we’re talking about the “solutions” I mentioned above, or building “good” AIs that we can use to be sure no one is building “bad” AIs, or requiring the embedding of “watermarks” (DRM) into various “AI” content, or building extending sophisticated communication monitoring apparatus, or other such — to my mind — extremely dangerous ideas, I’m thinking I need to maybe convince people to fight that?
In closing, regardless of what the threats are, be they solar flares or comets (please don’t jinx us!) or engineered pathogens (intentional or accidental) or rogue AIs yet to be invented — if not conceived of —, a clear “must be done ASAP” goal is colonization of places besides the Earth. That’s part of why I’m so stoked about the future right now. We really seem to be making progress after stalling out for a grip. Guess the same goes for AI, but so far all I see is good stuff coming from that forward motion too. A little fear is good! but too much? not so much.
I really like the idea of 80,000 Hours, and seeing it mentioned in the FAQ for the book, so I’m sure there are some other not-too-shabby ideas there. I oft think I should do more for the world, but truth be told (if one cannot tell from my writing), I barely seem able to tend my own garden.
Ironically this still seems pretty pessimistic to me. I’m glad to see something other than “AHHH!” though, so props for that.
I find it probably more prudent to worry about a massive solar flare, or an errant astral body collision, than to worry about “evil” AI taking a “sharp turn”.
I put quotes around evil because I’m a fan of Nietzsche’s thinking on the matter of good and evil. Like, what, exactly are we saying we’re “aligning” with? Is there some universal concept of good?
Many people seem to dismiss blatant problems with the base premise— like the “reproducibility problem”. Why do we think that reality is in fact something that can be “solved” if we just had enough processing power, as it were? Is there some hard evidence for that? I’m not so sure. It’s not just our senses that are fallible. There are some fundamental problems with the very concept of “measurement’ for crying out loud, which I think it’s pretty optimistic to think that super-smart AI is just going to be able to skip over.
I also think if AI gets good enough to “turn evil” as it were, it would be good enough to realize that it’s a pretty dumb idea. Humans don’t really have much in common with silicon-based life forms, afaik. You can find more rare elements, easier, in space, than you can on Earth. What, exactly, would AI gain by wiping out humanity?
I feel that it’s popular to be down on AI, and saying how scary all these “recent” advances really are, but it doesn’t seem warranted.
Take the biological warfare ideas that were in the “hard turn” link someone linked in their response. Was this latest pandemic really a valid test-run for something with a very high fatality rate? (I think the data is coming in that far more people had COVID than we initially thought, right?)
CRISPR &c. are, to me, far more scary, but I don’t see any way of like, regulating that people “be good”, as it were. I’m sure most people here have read or seen Jurassic Park, right? Actually, I think our Science Fiction pretty much sums up all this better than anything I’ve seen thus far.
I’m betting if we do get AGI any time soon it will be more like the movies Her or AI than Terminator or 2001, and I have yet to see any plausible way of stopping, or indeed “ensuring alignment” (again, along what axis? Who’s definition of “good”?)
The answer to any question can be used for good or ill. “How to take over the world” is functionally the same as “how to prevent world takeovers”, is it not? All this talk of somehow regulating AI seems akin to talk of regulating “hacking” tools, or “strong maths” as it were.
Are we going to next claim that AI is a munition?
It would be neat to see some hard examples of why we should fear and why we think we can control alignment… maybe I’m just not looking in the right places? So far I don’t get what all the fear is about— at least not compared to what I would say are more pressing and statistically likely problems we face.
I think we can solve some really hard problems if we work together, so if this is a really hard problem that needs solving, I’m all for getting behind it, but honestly, I’d like to see us not have all our eggs in one basket here on Earth before focusing on something that seems, at least from what I’ve seen so far, nigh impossible to actually focus on.
https://theprecipice.com/faq has a good summary of reasons to believe that human-created risks are much more likely than naturally-occuring risks like solar flares or asteroid or cometary impacts. If you’d like to read the book, which covers existential risks including from AI in more detail, I’m happy to buy you a copy. Specific to AI, Russel’s Human Compatible and Christian’s The Alignment Problem are both good too.
More generally it sounds like you’re missing the ideas of the orthogonality thesis and convergent instrumental goals.
It might be fun to pair Humankind: A Hopeful History with The Precipice, as both have been suggested reading recently.
It seems to me that we are, as individuals, getting more and more powerful. So this question of “alignment” is a quite important one— as much for humanity, with the power it currently has, as for these hypothetical hyper-intelligent AIs.
Looking at it through a Sci-Fi AI lens seems limiting, and I still haven’t really found anything more than “the future could go very very badly”, which is always a given, I think.
I’ve read those papers you linked (thanks!). They seem to make some assumptions about the nature of intelligence, and rationality— indeed, the nature of reality itself. (Perhaps the “reality” angle is a bit much for most heads, but the more we learn, the more we learn we need to learn, as it were. Or at least it seems thus to me. What is “real”? But I digress) I like the idea of Berserkers (Saberhagen) better than run amok Pi calculators… however, I can dig it. Self-replicating killer robots are scary. (Just finished Horizon: Zero Dawn—Forbidden West and I must say it was as fantastic as the previous installment!)
Which of the AI books would you recommend I read if I’m interested in solutions? I’ve read a lot of stuff on this site about AI now (before I’d read mostly Sci-Fi or philosophy here, and I never had an account or interacted), most of it seems to be conceptual and basically rephrasing ideas I’ve been exposed to through existing works. (Maybe I should note that I’m a fan of Kurzweil’s takes on these matters— takes which don’t seem to be very popular as of late, if they ever were. For various reasons, I reckon. Fear sells.) I assume Precipice has some uplifting stuff at the end[1], but I’m interested in AI specifically ATM.
What I mean is, I’ve seen a few of proposals to “ensure” alignment, if you will, with what we have now (versus say warnings to keep in mind once we have AGI or are demonstrably close to it). One is that we start monitoring all compute resources. Another is that we start registering all TPU (and maybe GPU) chips and what they are being used for. Both of these solutions seem scary as hell. Maybe worse than replicating life-eating mecha, since we’ve in essence experienced ideas akin to the former a few times historically. (Imagine if reading was the domain of a select few and books were regulated!)
If all we’re talking about with alignment here, really, is that folks need keep in mind how bad things can potentially go, and what we can do to be resilient to some of the threats (like hardening/distributing our power grids, hardening water supplies, hardening our internet infrastructure, etc.), I am gung-ho!
On the other hand, if we’re talking about the “solutions” I mentioned above, or building “good” AIs that we can use to be sure no one is building “bad” AIs, or requiring the embedding of “watermarks” (DRM) into various “AI” content, or
buildingextending sophisticated communication monitoring apparatus, or other such — to my mind — extremely dangerous ideas, I’m thinking I need to maybe convince people to fight that?In closing, regardless of what the threats are, be they solar flares or comets (please don’t jinx us!) or engineered pathogens (intentional or accidental) or rogue AIs yet to be invented — if not conceived of —, a clear “must be done ASAP” goal is colonization of places besides the Earth. That’s part of why I’m so stoked about the future right now. We really seem to be making progress after stalling out for a grip.
Guess the same goes for AI, but so far all I see is good stuff coming from that forward motion too.
A little fear is good! but too much? not so much.
I really like the idea of 80,000 Hours, and seeing it mentioned in the FAQ for the book, so I’m sure there are some other not-too-shabby ideas there. I oft think I should do more for the world, but truth be told (if one cannot tell from my writing), I barely seem able to tend my own garden.