I lean toward there being a meaningful distinction here: a system can learn a general-purpose learning algorithm, or it can ‘merely’ learn a very good conditional model.
Does human reasoning count as a general-purpose learning algorithm? I’ve heard it claimed that when we apply neural nets to tasks humans haven’t been trained on (like understanding DNA or materials science) the neural nets can rocket past human understanding, with way less computation and tools (and maybe even data) than humans have had access to (depending on how you measure). Tbc, I find this claim believable but haven’t checked it myself. Maybe SGD is the real general-purpose learning algorithm? Human reasoning could certainly be viewed formally as “a very good conditional model”.
So overall I lean towards thinking this is a continuous spectrum with no discontinuous changes (except ones like “better than humans or not”, which use a fixed reference point to get a discontinuity). So there could be a meaningful distinction, but it’s like the meaningful distinction between “warm water” and “hot water”, rather than the meaningful distinction between “water” and “ice”.
Does human reasoning count as a general-purpose learning algorithm? I’ve heard it claimed that when we apply neural nets to tasks humans haven’t been trained on (like understanding DNA or materials science) the neural nets can rocket past human understanding, with way less computation and tools (and maybe even data) than humans have had access to (depending on how you measure). Tbc, I find this claim believable but haven’t checked it myself. Maybe SGD is the real general-purpose learning algorithm? Human reasoning could certainly be viewed formally as “a very good conditional model”.
So overall I lean towards thinking this is a continuous spectrum with no discontinuous changes (except ones like “better than humans or not”, which use a fixed reference point to get a discontinuity). So there could be a meaningful distinction, but it’s like the meaningful distinction between “warm water” and “hot water”, rather than the meaningful distinction between “water” and “ice”.