Shane [Legg], FAI problems are AGI problems, they are simply a particular kind and style of AGI problem in which large sections of the solution space have been crossed out as unstable. FAI research = Friendly-style AGI research. “Do the right thing” is not a module, it is the AI.
I’ve already worked out a handful of basic problems; noticed that AGIfolk want to go ahead without understanding even those; and they look like automatic killers to me. Meanwhile the AGIfolk say, “If you delay, someone else will take the prize!” I know reversed stupidity is not intelligence, but still, I think I can stand to learn from this.
You have to surpass that sheer blank wall, whose difficulty is not matched to your skills. An unalterable demand of Nature, which you cannot negotiate down. Though to be sure, if you try to shave off just a little (because everyone has to compromise now and then), Nature will not try to negotiate back up.
Until you can turn your back on your rivals and the ticking clock, blank them completely out of your mind, you will not be able to see what the problem itself is asking of you. In theory, you should be able to see both at the same time. In practice, you won’t.
The sheer blank wall doesn’t care how much time you have. It’s just there. Pass-fail. You’re not being graded on a curve. Don’t have enough time? Too bad.
I think that solving Friendliness will take longer than building the first powerful AGI. Thus, if you do 1 before getting into 2, I think it’s unlikely that you’ll be first.
Who are you trying to negotiate with?
Cormac: In Bayesian terms, how much have your gross past failures affected your confidence in your current thinking?
Confidence is cheap. Pretending to unconfidence is equally cheap. Anyone can say they are certain and anyone can say they are uncertain.
My past failures have drastically affected the standards to which I hold an AI idea before I am willing to put my weight down on it. They’ve prevented me from writing code as yet. They’ve caused me to invest large amounts of time in better FAI theories, and more recently, in preparing for the possibility that someone else may have to take over from me. That’s “affect”. Confidence is cheap, and so is doubt.
On a side note—it’s also interesting that someone who is as open to admitting failures as you are still writes in the style of someone who’s never once before admitted a failure. I understand your desire to write with strength—but I’m not sure if it’s always the most effective way to influence others.
...I think that’s just the way I write.
Just as confidence is only a writing style, so too, it is cheap to write in a style of anguished doubt. It is just writing. If you can’t see past my writing style that happens to sound confident, to ask “What is he doing?”, then you will also not be able to see through writing that sounds self-doubtful, to ask “What are they doing?”
The sheer blank wall doesn’t care how much time you have. It’s just there. Pass-fail. You’re not being graded on a curve. Don’t have enough time? Too bad.
Yes, the “sheer blank wall” model could lead to gambling on getting a pass.
However, is the “sheer blank wall” model right? I think common sense dictates that there are a range of possible outcomes, of varying desirability. However, I suppose it is not totally impossible that there are a bunch of outcomes, widely regarded as being of very low value, which collectively make up a “fail wall”.
The 2008 GLOBAL CATASTROPHIC RISKS SURVEY apparently pegged the risk of hitting such a wall before 2100 as being 5%. Perhaps it can’t be completely ruled out.
The “pass-or-fail” mentality could cause serious problems, though, if the exam isn’t being graded that way.
Shane [Legg], FAI problems are AGI problems, they are simply a particular kind and style of AGI problem in which large sections of the solution space have been crossed out as unstable. FAI research = Friendly-style AGI research. “Do the right thing” is not a module, it is the AI.
I’ve already worked out a handful of basic problems; noticed that AGIfolk want to go ahead without understanding even those; and they look like automatic killers to me. Meanwhile the AGIfolk say, “If you delay, someone else will take the prize!” I know reversed stupidity is not intelligence, but still, I think I can stand to learn from this.
You have to surpass that sheer blank wall, whose difficulty is not matched to your skills. An unalterable demand of Nature, which you cannot negotiate down. Though to be sure, if you try to shave off just a little (because everyone has to compromise now and then), Nature will not try to negotiate back up.
Until you can turn your back on your rivals and the ticking clock, blank them completely out of your mind, you will not be able to see what the problem itself is asking of you. In theory, you should be able to see both at the same time. In practice, you won’t.
The sheer blank wall doesn’t care how much time you have. It’s just there. Pass-fail. You’re not being graded on a curve. Don’t have enough time? Too bad.
Who are you trying to negotiate with?
Confidence is cheap. Pretending to unconfidence is equally cheap. Anyone can say they are certain and anyone can say they are uncertain.
My past failures have drastically affected the standards to which I hold an AI idea before I am willing to put my weight down on it. They’ve prevented me from writing code as yet. They’ve caused me to invest large amounts of time in better FAI theories, and more recently, in preparing for the possibility that someone else may have to take over from me. That’s “affect”. Confidence is cheap, and so is doubt.
...I think that’s just the way I write.
Just as confidence is only a writing style, so too, it is cheap to write in a style of anguished doubt. It is just writing. If you can’t see past my writing style that happens to sound confident, to ask “What is he doing?”, then you will also not be able to see through writing that sounds self-doubtful, to ask “What are they doing?”
Yes, the “sheer blank wall” model could lead to gambling on getting a pass.
However, is the “sheer blank wall” model right? I think common sense dictates that there are a range of possible outcomes, of varying desirability. However, I suppose it is not totally impossible that there are a bunch of outcomes, widely regarded as being of very low value, which collectively make up a “fail wall”.
The 2008 GLOBAL CATASTROPHIC RISKS SURVEY apparently pegged the risk of hitting such a wall before 2100 as being 5%. Perhaps it can’t be completely ruled out.
The “pass-or-fail” mentality could cause serious problems, though, if the exam isn’t being graded that way.