We need to understand information encoding in the brain before we can achieve full AGI.
I disagree; it seems that we are going to brute-force AGI by searching for it, rather than building it, so to speak. Stochastic gradient descent on neural networks is basically a search in circuit-space for a circuit that does the job.
The machine learning stuff comes with preexisting artificial encoding. We label stuff ourselves.
I’m not sure what you mean by this but it seems false to me. “Pretraining” and “unsupervised learning” are a really big deal these days. I’m pretty sure lots of image classifiers, for example, generate their own labels for things basically, because they are just trained to predict stuff and then in order to do so they end up coming up with their own categories and concepts. GPT-3 et al did this too.
Without innate information encoding for language, infants won’t be learning much about the world through language, let alone picking up an entirely language from purely listening experiences.
GPT-3 begins as a totally blank slate. Yet it is able to learn language, and quite a lot about the world through language. It can e.g. translate between English and Chinese even though all it’s done is read loads of text. This strongly suggests that whatever innate stuff is present in the human brain is nice but nonessential, at least for the basics like learning language and learning about the world.
I’d like to be corrected if I’m wrong on AGI. I’ve only read a little about it back in my freshman years in college. I’m sure there has been a lot of development since then, and I’d like to learn from this community. From my experience of reading about AGI, it’s still dealing with more or less the confines of the computational statistics nature of intelligence.
I don’t know when you went to college, but a lot has changed in the last 10 years and in the last 2-5 years especially. If you are looking for more stuff to read on this, I recommend Gwern on the Scaling Hypothesis.
I disagree; it seems that we are going to brute-force AGI by searching for it, rather than building it, so to speak. Stochastic gradient descent on neural networks is basically a search in circuit-space for a circuit that does the job.
I’m not sure what you mean by this but it seems false to me. “Pretraining” and “unsupervised learning” are a really big deal these days. I’m pretty sure lots of image classifiers, for example, generate their own labels for things basically, because they are just trained to predict stuff and then in order to do so they end up coming up with their own categories and concepts. GPT-3 et al did this too.
GPT-3 begins as a totally blank slate. Yet it is able to learn language, and quite a lot about the world through language. It can e.g. translate between English and Chinese even though all it’s done is read loads of text. This strongly suggests that whatever innate stuff is present in the human brain is nice but nonessential, at least for the basics like learning language and learning about the world.
I don’t know when you went to college, but a lot has changed in the last 10 years and in the last 2-5 years especially. If you are looking for more stuff to read on this, I recommend Gwern on the Scaling Hypothesis.