Based on the transcript this does not sound like a FOOM discussion (as in rapid self-improvement) other than mentioning “group learning” by autonomous cars, which is maybe somewhat related. Also the pregnancy ad story is much more about pattern recognition with lots of data than any serious AI.
Basically JP is, in this area, a complete layman (unlike Gates, Musk, or, from the other side, Pinker) whose opinion counts for little and not talking about FOOM anyway.
Yeah, it’s sort of awkward that there are two different things one might want to talk about with FOOM: the idea of recursive self improvement in the typical I.J. Good sense, and the “human threshold isn’t special and can be blown past quickly” idea. AlphaZero being able to hit the superhuman level at Go after 3 days of training, and doing so only a year or two after any professional Go player was defeated by a computer, feels relevant to the second thing but not the first (and is connected to the ‘fleets of cars will learn very differently’ thing Peterson is pointing at).
[And the two actually are distinct; RSI is an argument for ‘blowing past humans is possible’ but many ‘slow takeoff’ views look more like “RSI pulls humans along with it” than “things look slow to a Martian,” and there’s ways to quickly blow past humans that don’t involve RSI.]
“once those things get a little bit smart they’re not going to stop at a little bit smart for very long they’re gonna be unbelievably smart like overnight. ”
Celebrity opinions count for something in ways that expert opinions do not. They seem to reach more people, for one thing. That’s partially because people just accept what celebrities say because they admire them, but it seems to me that it’s also because celebrities tend to find ways of expressing the essence of ideas that are more accessible to laypeople.
Anyway, for whatever the reason, celebrities who openly express their opinions can make a difference, and I think we should celebrate when one of them gets something right and is willing to talk about it.
Based on the transcript this does not sound like a FOOM discussion (as in rapid self-improvement) other than mentioning “group learning” by autonomous cars, which is maybe somewhat related. Also the pregnancy ad story is much more about pattern recognition with lots of data than any serious AI.
Basically JP is, in this area, a complete layman (unlike Gates, Musk, or, from the other side, Pinker) whose opinion counts for little and not talking about FOOM anyway.
Yeah, it’s sort of awkward that there are two different things one might want to talk about with FOOM: the idea of recursive self improvement in the typical I.J. Good sense, and the “human threshold isn’t special and can be blown past quickly” idea. AlphaZero being able to hit the superhuman level at Go after 3 days of training, and doing so only a year or two after any professional Go player was defeated by a computer, feels relevant to the second thing but not the first (and is connected to the ‘fleets of cars will learn very differently’ thing Peterson is pointing at).
[And the two actually are distinct; RSI is an argument for ‘blowing past humans is possible’ but many ‘slow takeoff’ views look more like “RSI pulls humans along with it” than “things look slow to a Martian,” and there’s ways to quickly blow past humans that don’t involve RSI.]
I don’t think Gates, Musk, or Pinker should count as much more than laymen when it comes to AI risk, either.
“once those things get a little bit smart they’re not going to stop at a little bit smart for very long they’re gonna be unbelievably smart like overnight. ”
Celebrity opinions count for something in ways that expert opinions do not. They seem to reach more people, for one thing. That’s partially because people just accept what celebrities say because they admire them, but it seems to me that it’s also because celebrities tend to find ways of expressing the essence of ideas that are more accessible to laypeople.
Anyway, for whatever the reason, celebrities who openly express their opinions can make a difference, and I think we should celebrate when one of them gets something right and is willing to talk about it.
I somewhat overlooked this line and yes, it’s a nod in the right direction