Think like a cognitive scientist and AI programmer.
Is it possible to think “like an AI programmer” without being an AI programmer ? If the answer is “no”, as I suspect it is, then doesn’t this piece of advice basically say, “don’t be a philosopher, be an AI programmer instead” ? If so, then it directly contradicts your point that “philosophy is not useless”.
To put it in a slightly different way, is creating FAI primarily a philosophical challenge, or an engineering challenge ?
Creating AI is an engineering challenge. Making FAI requires an understanding of what we mean by Friendly. If you don’t think that is a philosophy question, I would point to the multiplicity of inconsistent moral theories throughout history to try to convince you otherwise.
Thanks, that does make sense. But, in this case, would “thinking like an AI programmer” really help you answer the question of “what we mean by Friendly” ? Of course, once we do get an answer, we’d need to implement it, which is where thinking like an AI programmer (or actually being one) would come in handy. But I think that’s also an engineering challenge at that point.
FWIW, I know there are people out there who would claim that friendliness/morality is a scientific question, not a philosophical one, but I myself am undecided on the issue.
But, in this case, would “thinking like an AI programmer” really help you answer the question of “what we mean by Friendly” ? Of course, once we do get an answer, we’d need to implement it, which is where thinking like an AI programmer (or actually being one) would come in handy. But I think that’s also an engineering challenge at that point.
If you don’t think like an AI programmer, you will be tempted to use concepts without understanding them well enough to program them. I don’t think that’s reduced to the level of ‘engineering challenge.’
Are you saying that it’s impossible to correctly answer the question “what does ‘friendly’ mean ?” without understanding how to implement the answer by writing a computer program ? If so, why do you think that ?
Edit: added “correctly” in the sentence above, because it’s trivially possible to just answer “bananas !” or something :-)
I don’t think the division is so sharp as all that. Rather, what Vanvier is getting at, I think, is that one is capable of correctly and usefully answering the question “What does ‘Friendy’ mean?” in proportion to one’s ability to reason algorithmically about subproblems of Friendliness.
I see, so you’re saying that a philosopher who is not familiar with AI might come up with all kinds of philosophically valid definitions of friendliness, which would still be impossible to implement (using a reasonable amount of space and time) and thus completely useless in practice. That makes sense. And (presumably) if we assume that humans are kind of similar to AIs, then the AI-savvy philosopher’s ideas would have immediate applications, as well.
So, that makes sense, but I’m not aware of any philosophers who have actually followed this recipe. It seems like at least a few such philosophers should exist, though… do they ?
[P]hilosophically valid definitions of friendliness, which would still be impossible to implement (using a reasonable amount of space and time) and thus completely useless in practice.
Yes, or more sneakily, impossible to implement due to a hidden reliance on human techniques for which there is as-yet no known algorithmic implementation.
Programmers like to say “You don’t truly understand how to perform a task until you can teach a computer to do it for you”. A computer, or any other sort of rigid mathematical mechanism, is unable to make the ‘common sense’ connections that a human mind can make. We humans are so good at that sort of thing that we often make many such leaps in quick succession without even noticing!
Implementing an idea on a computer forces us to slow down and understand every step, even the ones we make subconsciously. Otherwise the implementation simply won’t work. One doesn’t get as thorough a check when explaining things to another human.
Philosophy in general is enriched by an understanding of math and computation, because it provides a good external view of the situation. This effect is of course only magnified when the philosopher is specifically thinking about how to represent human mental processes (such as volition) in a computational way.
I agree with most of what you said, except for this:
Yes, or more sneakily, impossible to implement due to a hidden reliance on human techniques for which there is as-yet no known algorithmic implementation.
Firstly, this is an argument for studying “human techniques”, and devising algorithmic implementations, and not an argument for abandoning these techniques. Assuming the techniques are demonstrated to work reliably, of course.
Secondly, if we assume that uploading is possible, this problem can be hacked around by incorporating an uploaded human into the solution.
Firstly, this is an argument for studying “human techniques”, and devising algorithmic implementations, and not an argument for abandoning these techniques.
Indeed, I should have been more specific; not all processes used in AI need to be analogous to humans, of course. All I meant was that it is very easy, when trying to provide a complete spec of a human process, to accidentally lean on other human mental processes that seem on zeroth-glance to be “obvious”. It’s hard to spot those mistakes without an outside view.
Secondly, if we assume that uploading is possible, this problem can be hacked around by incorporating an uploaded human into the solution.
To a degree, though I suspect that even in an uploaded mind it would be tricky to isolate and copy-out individual techniques, since they’re all likely to be non-locally-cohesive and heavily interdependent.
It’s hard to spot those mistakes without an outside view.
Right, that makes sense.
To a degree, though I suspect that even in an uploaded mind it would be tricky to isolate and copy-out individual techniques...
True, but I wasn’t thinking of using an uploaded mind to extract and study those ideas, but simply to plug the mind into your overall architecture and treat it like a black box that gives you the right answers, somehow. It’s a poor solution, but it’s better than nothing—assuming that the Singularity is imminent and we’re all about to be nano-recycled into quantum computronium, unless we manage to turn the AI into an FAI in the next 72 hours.
Computational complexity theory is a huge, sprawling field; naturally this essay will only touch
on small parts of it...One might think that, once we know something is computable, whether it takes 10 seconds or
20 seconds to compute is obviously the concern of engineers rather than philosophers. But that
conclusion would not be so obvious, if the question were one of 10 seconds versus 101010
seconds! And indeed, in complexity theory, the quantitative gaps we care about are usually so vast that one
has to consider them qualitative gaps as well. Think, for example, of the difference between reading
a 400-page book and reading every possible such book, or between writing down a thousand-digit
number and counting to that number. More precisely, complexity theory asks the question: how do the resources needed to solve
a problem scale with some measure n of the problem size...
Need it be primarily one or the other? But if I must pick one, I pick philosophy.
I’m afraid I don’t see how this article is analogous. The article points out that computational complexity puts a very real limit on what can be computed in practice. Thus, even if you’d proved that something is computable in principle, it may not be computable in our current Universe, with its limited lifespan. You can apply computational complexity to practical problems (f.ex., devising an optimal route for inspecting naval buoys) as well as to theoretical ones (f.ex., discarding the hypothesis that the human brain is a giant lookup table). But these are still engineering and scientific concerns, not philosophical ones.
Need it be primarily one or the other? But if I must pick one, I pick philosophy.
I still don’t understand why. If you want to know the probability of FAI being feasible at all, you’re asking a scientific question; in order to answer it, you’ll need to formulate a hypothesis or two, gather evidence, employ Bayesian reasoning to compute the probability of your hypothesis being true, etc. If, on the other hand, you are trying to actually build an FAI, then you are solving a specific engineering problem; of course, determining whether FAI is feasible or not would be a great first step.
So, I can see how you’d apply science or engineering to the problem, but I don’t see how you’d apply philosophy.
Sorry, I couldn’t parse your comment at all; I’m not sure what you mean by “content”. My hunch is that you meant the same thing as TimS, above; if so, my reply to him should be relevant. If not, my apologies, but could you please explain what you meant ?
Is it possible to think “like an AI programmer” without being an AI programmer ? If the answer is “no”, as I suspect it is, then doesn’t this piece of advice basically say, “don’t be a philosopher, be an AI programmer instead” ? If so, then it directly contradicts your point that “philosophy is not useless”.
To put it in a slightly different way, is creating FAI primarily a philosophical challenge, or an engineering challenge ?
Creating AI is an engineering challenge. Making FAI requires an understanding of what we mean by Friendly. If you don’t think that is a philosophy question, I would point to the multiplicity of inconsistent moral theories throughout history to try to convince you otherwise.
Thanks, that does make sense. But, in this case, would “thinking like an AI programmer” really help you answer the question of “what we mean by Friendly” ? Of course, once we do get an answer, we’d need to implement it, which is where thinking like an AI programmer (or actually being one) would come in handy. But I think that’s also an engineering challenge at that point.
FWIW, I know there are people out there who would claim that friendliness/morality is a scientific question, not a philosophical one, but I myself am undecided on the issue.
If you don’t think like an AI programmer, you will be tempted to use concepts without understanding them well enough to program them. I don’t think that’s reduced to the level of ‘engineering challenge.’
Are you saying that it’s impossible to correctly answer the question “what does ‘friendly’ mean ?” without understanding how to implement the answer by writing a computer program ? If so, why do you think that ?
Edit: added “correctly” in the sentence above, because it’s trivially possible to just answer “bananas !” or something :-)
I don’t think the division is so sharp as all that. Rather, what Vanvier is getting at, I think, is that one is capable of correctly and usefully answering the question “What does ‘Friendy’ mean?” in proportion to one’s ability to reason algorithmically about subproblems of Friendliness.
I see, so you’re saying that a philosopher who is not familiar with AI might come up with all kinds of philosophically valid definitions of friendliness, which would still be impossible to implement (using a reasonable amount of space and time) and thus completely useless in practice. That makes sense. And (presumably) if we assume that humans are kind of similar to AIs, then the AI-savvy philosopher’s ideas would have immediate applications, as well.
So, that makes sense, but I’m not aware of any philosophers who have actually followed this recipe. It seems like at least a few such philosophers should exist, though… do they ?
Yes, or more sneakily, impossible to implement due to a hidden reliance on human techniques for which there is as-yet no known algorithmic implementation.
Programmers like to say “You don’t truly understand how to perform a task until you can teach a computer to do it for you”. A computer, or any other sort of rigid mathematical mechanism, is unable to make the ‘common sense’ connections that a human mind can make. We humans are so good at that sort of thing that we often make many such leaps in quick succession without even noticing!
Implementing an idea on a computer forces us to slow down and understand every step, even the ones we make subconsciously. Otherwise the implementation simply won’t work. One doesn’t get as thorough a check when explaining things to another human.
Philosophy in general is enriched by an understanding of math and computation, because it provides a good external view of the situation. This effect is of course only magnified when the philosopher is specifically thinking about how to represent human mental processes (such as volition) in a computational way.
I agree with most of what you said, except for this:
Firstly, this is an argument for studying “human techniques”, and devising algorithmic implementations, and not an argument for abandoning these techniques. Assuming the techniques are demonstrated to work reliably, of course.
Secondly, if we assume that uploading is possible, this problem can be hacked around by incorporating an uploaded human into the solution.
Indeed, I should have been more specific; not all processes used in AI need to be analogous to humans, of course. All I meant was that it is very easy, when trying to provide a complete spec of a human process, to accidentally lean on other human mental processes that seem on zeroth-glance to be “obvious”. It’s hard to spot those mistakes without an outside view.
To a degree, though I suspect that even in an uploaded mind it would be tricky to isolate and copy-out individual techniques, since they’re all likely to be non-locally-cohesive and heavily interdependent.
Right, that makes sense.
True, but I wasn’t thinking of using an uploaded mind to extract and study those ideas, but simply to plug the mind into your overall architecture and treat it like a black box that gives you the right answers, somehow. It’s a poor solution, but it’s better than nothing—assuming that the Singularity is imminent and we’re all about to be nano-recycled into quantum computronium, unless we manage to turn the AI into an FAI in the next 72 hours.
Endorsed.
An analogy:
http://eccc.hpi-web.de/report/2011/108/
Need it be primarily one or the other? But if I must pick one, I pick philosophy.
I’m afraid I don’t see how this article is analogous. The article points out that computational complexity puts a very real limit on what can be computed in practice. Thus, even if you’d proved that something is computable in principle, it may not be computable in our current Universe, with its limited lifespan. You can apply computational complexity to practical problems (f.ex., devising an optimal route for inspecting naval buoys) as well as to theoretical ones (f.ex., discarding the hypothesis that the human brain is a giant lookup table). But these are still engineering and scientific concerns, not philosophical ones.
I still don’t understand why. If you want to know the probability of FAI being feasible at all, you’re asking a scientific question; in order to answer it, you’ll need to formulate a hypothesis or two, gather evidence, employ Bayesian reasoning to compute the probability of your hypothesis being true, etc. If, on the other hand, you are trying to actually build an FAI, then you are solving a specific engineering problem; of course, determining whether FAI is feasible or not would be a great first step.
So, I can see how you’d apply science or engineering to the problem, but I don’t see how you’d apply philosophy.
To fill in the content the term “FAI” stands for, science isn’t enough. Engineering is by guess and check, I suppose, but not really.
Sorry, I couldn’t parse your comment at all; I’m not sure what you mean by “content”. My hunch is that you meant the same thing as TimS, above; if so, my reply to him should be relevant. If not, my apologies, but could you please explain what you meant ?
I meant what I think he did, so you got it.