Scott Garrabrant once said something that really stuck with me: (paraphrasing) “You can get a mind in two ways—by building it, or by searching for it. The way things are going with deep learning right now, it seems like we aren’t going to build AGI, we are going to search for it. And that means trouble.”
I feel like this is ultimately a pretty simple point but it is I think really important, it is a good foundation for thinking about (and taking seriously) things like inner alignment issues. Some comments:
--Obviously it’s not binary, there’s a spectrum between “building/designing” and “searching.”
--Given our current level of understanding, I claim that doing stochastic gradient descent to train big neural networks is by default pretty far towards the “searching” end of the spectrum. The thing we get out the other end is a giant messy glob of neurons and we don’t know in advance what it will be like (and often we don’t even understand it afterwards, not even a little bit.) Contrast with traditional software, which is pretty far along the “building/designing” end of the spectrum.
Scott Garrabrant once said something that really stuck with me: (paraphrasing) “You can get a mind in two ways—by building it, or by searching for it. The way things are going with deep learning right now, it seems like we aren’t going to build AGI, we are going to search for it. And that means trouble.”
I feel like this is ultimately a pretty simple point but it is I think really important, it is a good foundation for thinking about (and taking seriously) things like inner alignment issues. Some comments:
--Obviously it’s not binary, there’s a spectrum between “building/designing” and “searching.”
--Given our current level of understanding, I claim that doing stochastic gradient descent to train big neural networks is by default pretty far towards the “searching” end of the spectrum. The thing we get out the other end is a giant messy glob of neurons and we don’t know in advance what it will be like (and often we don’t even understand it afterwards, not even a little bit.) Contrast with traditional software, which is pretty far along the “building/designing” end of the spectrum.