The linked bet doesn’t reference “a week,” and the “week” reference in the main linked post is about going from infrahuman to superhuman, not using that intelligence to destroy humanity.
That bet seems underspecified. Does attention to “Friendliness” mean any attention to safety whatsoever, or designing an AI with a utility function such that it’s trustworthy regardless of power levels? Is “superhuman” defined relative to the then-current level of human (or upload, or trustworthy less intelligent AI) capacity with any enhancements (or upload speedups, etc)? What level of ability counts as superhuman? You two should publicly clarify the terms.
A few comments later on the same comment thread someone asked me how much time was necessary, and I said I thought a week was enough, based on Eliezer’s previous statements, and he never contradicted this, so it seems to me that he accepted it by default, since some time limit will be necessary in order for someone to win the bet.
I defined superhuman to mean that everyone will agree that it is more intelligent than any human being existing at that time.
I agree that the question of whether there is attention to Friendliness might be more problematic to determine. But “any attention to safety whatsoever” seems to me to be clearly stretching the idea of Friendliness—for example, someone could pay attention to safety by trying to make sure that the AI was mostly boxed, or whatever, and this wouldn’t satisfy Eliezer’s idea of Friendliness.
Right. And if this scenario happened, there would be a good chance that it would not be able to foom, or at least not within a week. Eliezer’s opinion seems to be that this scenario is extremely unlikely, in other words that the first AI will already be far more intelligent than the human race, and that even if it is running on an immense amount of hardware, it will have no need to acquire more hardware, because it will be able to construct nanotechnology capable of controlling the planet through actions originating on the internet as you suggest. And as you can see, he is very confident that all this will happen within a very short period of time.
What guarantee?.
With a guarantee backed by $1000.
The linked bet doesn’t reference “a week,” and the “week” reference in the main linked post is about going from infrahuman to superhuman, not using that intelligence to destroy humanity.
That bet seems underspecified. Does attention to “Friendliness” mean any attention to safety whatsoever, or designing an AI with a utility function such that it’s trustworthy regardless of power levels? Is “superhuman” defined relative to the then-current level of human (or upload, or trustworthy less intelligent AI) capacity with any enhancements (or upload speedups, etc)? What level of ability counts as superhuman? You two should publicly clarify the terms.
A few comments later on the same comment thread someone asked me how much time was necessary, and I said I thought a week was enough, based on Eliezer’s previous statements, and he never contradicted this, so it seems to me that he accepted it by default, since some time limit will be necessary in order for someone to win the bet.
I defined superhuman to mean that everyone will agree that it is more intelligent than any human being existing at that time.
I agree that the question of whether there is attention to Friendliness might be more problematic to determine. But “any attention to safety whatsoever” seems to me to be clearly stretching the idea of Friendliness—for example, someone could pay attention to safety by trying to make sure that the AI was mostly boxed, or whatever, and this wouldn’t satisfy Eliezer’s idea of Friendliness.
Ah. So an AI could, e.g. be only slightly superhuman and require immense quantities of hardware to generate that performance in realtime.
Right. And if this scenario happened, there would be a good chance that it would not be able to foom, or at least not within a week. Eliezer’s opinion seems to be that this scenario is extremely unlikely, in other words that the first AI will already be far more intelligent than the human race, and that even if it is running on an immense amount of hardware, it will have no need to acquire more hardware, because it will be able to construct nanotechnology capable of controlling the planet through actions originating on the internet as you suggest. And as you can see, he is very confident that all this will happen within a very short period of time.