What won’t we be able to do by (say) the end of 2025? (See also this recent post.) Well, one easy way to generate such answers would be to consider tasks that require embodiment in the real world, or tasks that humans would find challenging to do. (For example, “solve the halting problem”, “produce a new policy proposal that has at least a 95% chance of being enacted into law”, “build a household robot that can replace any human household staff”.) This is cheating, though; the real challenge is in naming something where there’s an adjacent thing that _does_ seem likely (i.e. it’s near the boundary separating “likely” from “unlikely”).
One decent answer is that I don’t expect we’ll have AI systems that could write new posts _on rationality_ that I like more than the typical LessWrong post with > 30 karma. However, I do expect that we could build an AI system that could write _some_ new post (on any topic) that I like more than the typical LessWrong post with > 30 karma. This is because (1) 30 karma is not that high a filter and includes lots of posts I feel pretty meh about, (2) there are lots of topics I know nothing about, on which it would be relatively easy to write a post I like, and (3) AI systems easily have access to this knowledge by being trained on the Internet. (It is another matter whether we actually build an AI system that can do this.) Note that there is still a decently large difference between these two tasks—the content would have to be quite a bit more novel in the former case (which is why I don’t expect it to be solved by 2025).
Note that I still think it’s pretty hard to predict what will and won’t happen, so even for this example I’d probably assign, idk, a 10% chance that it actually does work out (if we assume some organization tries hard to make it work)?
Nice! I really appreciate that you are thinking about this and making predictions. I want to do the same myself.
I think I’d put something more like 50% on “Rohin will at some point before 2030 read an AI-written blog post on rationality that he likes more than the typical LW >30 karma post.” That’s just a wild guess, very unstable.
Another potential prediction generation methodology: Name something that you think won’t happen, but you think I think will.
Rohin will at some point before 2030 read an AI-written blog post on rationality that he likes more than the typical LW >30 karma post.
This seems more feasible, because you can cherrypick a single good example. I wouldn’t be shocked if someone on LW spent a lot of time reading AI-written blog posts on rationality and posted the best one, and I liked that more than a typical >30 karma post. My default guess is that no one tries to do this, so I’d still give it < 50% (maybe 30%?), but conditional on someone trying I think probably 80% seems right. (EDIT: Rereading this, I have no idea whether I was considering a timeline of 2025 (as in my original comment) or 2030 (as in the comment I’m replying to) when making this prediction.)
Name something that you think won’t happen, but you think I think will.
I spent a bit of time on this but I think I don’t have a detailed enough model of you to really generate good ideas here :/
Otoh, if I were expecting TAI / AGI in 15 years, then by 2030 I’d expect to see things like:
An AI system that can create a working website with the desired functionality “from scratch” (e.g. a simple Twitter-like website, an application that tracks D&D stats and dice rolls for you, etc, a simple Tetris game with an account system, …). The system allows even non-programmers to create these kinds of websites (so cannot depend on having a human programmer step in to e.g. fix compiler errors or issue shell commands to set up the web server).
At least one large, major research area in which human researcher productivity has been boosted 100x relative to today’s levels thanks to AI. (In calculating the productivity we ignore the cost of running the AI system.) Humans can still be in the loop here, but the large majority of the work must be done by AIs.
An AI system gets 20,000 LW karma in a year, when limited to writing one article per day and responses to any comments it gets from humans. (EDIT: I failed to think about karma inflation when making this prediction and feel a bit worse about it now.)
Productivity tools like todo lists, memory systems, time trackers, calendars, etc are made effectively obsolete (or at least the user interfaces are made obsolete); the vast majority of people who used to use these tools have replaced them with an Alexa / Siri style assistant.
Currently, I don’t expect to see any of these by 2030.
Ah right, good point, I forgot about cherry-picking. I guess we could make it be something like “And the blog post wasn’t cherry-picked; the same system could be asked to make 2 additional posts on rationality and you’d like both of them also.” I’m not sure what credence I’d give to this but it would probably be a lot higher than 10%.
Website prediction: Nice, I think that’s like 50% likely by 2030.
Major research area: What counts as a major research area? Suppose I go calculate that Alpha Fold 2 has already sped up the field of protein structure prediction by 100x (don’t need to do actual experiments anymore!), would that count? If you hadn’t heard of AlphaFold yet, would you say it counted? Perhaps you could give examples of the smallest and easiest-to-automate research areas that you think have only a 10% chance of being automated by 2030.
20,000 LW karma: Holy shit that’s a lot of karma for one year. I feel like it’s possible that would happen before it’s too late (narrow AI good at writing but not good at talking to people and/or not agenty) but unlikely. Insofar as I think it’ll happen before 2030 it doesn’t serve as a good forecast because it’ll be too late by that point IMO.
Productivity tool UI’s obsolete thanks to assistants: This is a good one too. I think that’s 50% likely by 2030.
I’m not super certain about any of these things of course, these are just my wild guesses for now.
20,000 LW karma: Holy shit that’s a lot of karma for one year.
I was thinking 365 posts * ~50 karma per post gets you most of the way there (18,250 karma), and you pick up some additional karma from comments along the way. 50 karma posts are good but don’t have to be hugely insightful; you can also get a lot of juice by playing to the topics that tend to get lots of upvotes. Unlike humans the bot wouldn’t be limited by writing speed (hence my restriction of one post per day). AI systems should be really, really good at writing, given how easy it is to train on text. And a post is a small, self-contained thing, that takes not very long to create (i.e. it has short horizons), and there are lots of examples to learn from. So overall this seems like a thing that should happen well before TAI / AGI.
I think I want to give up on the research area example, seems pretty hard to operationalize. (But fwiw according to the picture in my head, I don’t think I’d count AlphaFold.)
OK, fair enough. But what if it writes, like, 20 posts in the first 20 days which are that good, but then afterwards it hits diminishing returns because the rationality-related points it makes are no longer particularly novel and exciting? I think this would happen to many humans if they could work at super-speed.
That said, I don’t think this is that likely I guess… probably AI will be unable to do even three such posts, or it’ll be able to generate arbitrary numbers of them. The human range is small. Maybe. Idk.
But what if it writes, like, 20 posts in the first 20 days which are that good, but then afterwards it hits diminishing returns because the rationality-related points it makes are no longer particularly novel and exciting?
I’d be pretty surprised if that happened. GPT-3 already knows way more facts than I do, and can mimic far more writing styles than I can. It seems like by the time it can write any good posts (without cherrypicking), it should quickly be able to write good posts on a variety of topics in a variety of different styles, which should let it scale well past 20 posts.
(In contrast, a specific person tends to write on 1-2 topics, in a single style, and not optimizing that hard for karma, and many still write tens of high-scoring posts.)
What won’t we be able to do by (say) the end of 2025? (See also this recent post.) Well, one easy way to generate such answers would be to consider tasks that require embodiment in the real world, or tasks that humans would find challenging to do. (For example, “solve the halting problem”, “produce a new policy proposal that has at least a 95% chance of being enacted into law”, “build a household robot that can replace any human household staff”.) This is cheating, though; the real challenge is in naming something where there’s an adjacent thing that _does_ seem likely (i.e. it’s near the boundary separating “likely” from “unlikely”).
One decent answer is that I don’t expect we’ll have AI systems that could write new posts _on rationality_ that I like more than the typical LessWrong post with > 30 karma. However, I do expect that we could build an AI system that could write _some_ new post (on any topic) that I like more than the typical LessWrong post with > 30 karma. This is because (1) 30 karma is not that high a filter and includes lots of posts I feel pretty meh about, (2) there are lots of topics I know nothing about, on which it would be relatively easy to write a post I like, and (3) AI systems easily have access to this knowledge by being trained on the Internet. (It is another matter whether we actually build an AI system that can do this.) Note that there is still a decently large difference between these two tasks—the content would have to be quite a bit more novel in the former case (which is why I don’t expect it to be solved by 2025).
Note that I still think it’s pretty hard to predict what will and won’t happen, so even for this example I’d probably assign, idk, a 10% chance that it actually does work out (if we assume some organization tries hard to make it work)?
Nice! I really appreciate that you are thinking about this and making predictions. I want to do the same myself.
I think I’d put something more like 50% on “Rohin will at some point before 2030 read an AI-written blog post on rationality that he likes more than the typical LW >30 karma post.” That’s just a wild guess, very unstable.
Another potential prediction generation methodology: Name something that you think won’t happen, but you think I think will.
This seems more feasible, because you can cherrypick a single good example. I wouldn’t be shocked if someone on LW spent a lot of time reading AI-written blog posts on rationality and posted the best one, and I liked that more than a typical >30 karma post. My default guess is that no one tries to do this, so I’d still give it < 50% (maybe 30%?), but conditional on someone trying I think probably 80% seems right. (EDIT: Rereading this, I have no idea whether I was considering a timeline of 2025 (as in my original comment) or 2030 (as in the comment I’m replying to) when making this prediction.)
I spent a bit of time on this but I think I don’t have a detailed enough model of you to really generate good ideas here :/
Otoh, if I were expecting TAI / AGI in 15 years, then by 2030 I’d expect to see things like:
An AI system that can create a working website with the desired functionality “from scratch” (e.g. a simple Twitter-like website, an application that tracks D&D stats and dice rolls for you, etc, a simple Tetris game with an account system, …). The system allows even non-programmers to create these kinds of websites (so cannot depend on having a human programmer step in to e.g. fix compiler errors or issue shell commands to set up the web server).
At least one large, major research area in which human researcher productivity has been boosted 100x relative to today’s levels thanks to AI. (In calculating the productivity we ignore the cost of running the AI system.) Humans can still be in the loop here, but the large majority of the work must be done by AIs.
An AI system gets 20,000 LW karma in a year, when limited to writing one article per day and responses to any comments it gets from humans. (EDIT: I failed to think about karma inflation when making this prediction and feel a bit worse about it now.)
Productivity tools like todo lists, memory systems, time trackers, calendars, etc are made effectively obsolete (or at least the user interfaces are made obsolete); the vast majority of people who used to use these tools have replaced them with an Alexa / Siri style assistant.
Currently, I don’t expect to see any of these by 2030.
Ah right, good point, I forgot about cherry-picking. I guess we could make it be something like “And the blog post wasn’t cherry-picked; the same system could be asked to make 2 additional posts on rationality and you’d like both of them also.” I’m not sure what credence I’d give to this but it would probably be a lot higher than 10%.
Website prediction: Nice, I think that’s like 50% likely by 2030.
Major research area: What counts as a major research area? Suppose I go calculate that Alpha Fold 2 has already sped up the field of protein structure prediction by 100x (don’t need to do actual experiments anymore!), would that count? If you hadn’t heard of AlphaFold yet, would you say it counted? Perhaps you could give examples of the smallest and easiest-to-automate research areas that you think have only a 10% chance of being automated by 2030.
20,000 LW karma: Holy shit that’s a lot of karma for one year. I feel like it’s possible that would happen before it’s too late (narrow AI good at writing but not good at talking to people and/or not agenty) but unlikely. Insofar as I think it’ll happen before 2030 it doesn’t serve as a good forecast because it’ll be too late by that point IMO.
Productivity tool UI’s obsolete thanks to assistants: This is a good one too. I think that’s 50% likely by 2030.
I’m not super certain about any of these things of course, these are just my wild guesses for now.
I was thinking 365 posts * ~50 karma per post gets you most of the way there (18,250 karma), and you pick up some additional karma from comments along the way. 50 karma posts are good but don’t have to be hugely insightful; you can also get a lot of juice by playing to the topics that tend to get lots of upvotes. Unlike humans the bot wouldn’t be limited by writing speed (hence my restriction of one post per day). AI systems should be really, really good at writing, given how easy it is to train on text. And a post is a small, self-contained thing, that takes not very long to create (i.e. it has short horizons), and there are lots of examples to learn from. So overall this seems like a thing that should happen well before TAI / AGI.
I think I want to give up on the research area example, seems pretty hard to operationalize. (But fwiw according to the picture in my head, I don’t think I’d count AlphaFold.)
OK, fair enough. But what if it writes, like, 20 posts in the first 20 days which are that good, but then afterwards it hits diminishing returns because the rationality-related points it makes are no longer particularly novel and exciting? I think this would happen to many humans if they could work at super-speed.
That said, I don’t think this is that likely I guess… probably AI will be unable to do even three such posts, or it’ll be able to generate arbitrary numbers of them. The human range is small. Maybe. Idk.
I’d be pretty surprised if that happened. GPT-3 already knows way more facts than I do, and can mimic far more writing styles than I can. It seems like by the time it can write any good posts (without cherrypicking), it should quickly be able to write good posts on a variety of topics in a variety of different styles, which should let it scale well past 20 posts.
(In contrast, a specific person tends to write on 1-2 topics, in a single style, and not optimizing that hard for karma, and many still write tens of high-scoring posts.)