@billswift: You were right about Pavlina. I discovered that as I read more of his stuff.
Tom_Breton
Karma: 87
@RT Wolf: Thanks for the Pavlina link. It looks fascinating so far.
Apparently the people who played gatekeeper previously held the idea that it was impossible for an AI to talk its way out. Not just for Eliezer, but for a transhuman AI; and not just for them, but for all sorts of gatekeepers. That’s what is implied by saying “We will just keep it in a box”.
In other words, and not meaning to cast any aspersions, they all had a blind spot. Failure of imagination, perhaps.
This blind spot may have been a factor in their loss. Having no access to the mysterious transcripts, I won’t venture a guess as to how.
a “logically possible” but fantastic being â a descendent of Ned Block’s Giant Lookup Table fantasy...
First, I haven’t seen how this figures into an argument, and I see that Eliezer has already taken this in another direction, but...
What immediately occurs to me is that there’s a big risk of a faulty intuition pump here. He’s describing, I assume, a lookup table large enough to describe your response to every distinguishable sensory input you could conceivably experience during your life. The number of entries is unimaginable. But I suspect he’s picturing and inviting us to picture a much more mundane, manageable LUT.
I can almost hear the Chinese Room Fallacy already. “You can’t say that a LUT is conscious, it’s just a matrix”. Like ”...just some cards and some rules” or ”...just transistors”. That intuition works in a common-sense way when the thing is tiny, but we just said it wasn’t.
And let’s not slight other factors that make the thing either very big and hairy or very, very, very big.
To work as advertised, it needs some sense of history. Perhaps every instant in our maybe-zombie’s history has its own corresponding dimension in the table, or perhaps some field(s) of the table’s output at each instant is an additional input at the next instant, representing one’s entire mental state. Either way, it’s gotta be huge enough to represent every distinguishable history.
The input and output formats also correspond to enormous objects capable of fully describing all the sensory input we can perceive in a short time, all the actions we can take in a short time (including habitual, autonomic, everything), and every aspect of our mental state.
This ain’t your daddy’s 16 x 32 array of unsigned ints.
To put it much more briefly, under the Wesley Salmon definition of “explanation” the epiphenomenal picture is simply not an explanation of consciousness.
Any commited autodidacts want to share how their autodidactism makes them feel compared to traditional schooled learners? I’m beginning to suspect that maybe it takes a certain element of belief in the superiority of one’s methods to make autodidactism work.
As Komponisto points out, traditional schooling is so bad at educating that belief in the superiority of one’s [own] methods is easily acquired. I first noticed traditional schooling’s ineptitude in kindergarten, and this perception was reinforced almost continuously thru the rest of my schooling.
PS: I liked the initiation ceremony fiction, Eliezer.
In classical logic, the operational definition of identity is that whenever ‘A=B’ is a theorem, you can substitute ‘A’ for ‘B’ [but it doesn’t follow that] I believe 2 + 2 = 4 ⇒ I believe TRUE ⇒ I believe Fermat’s Last Theorem.
The problem is that identity has been treated as if it were absolute, as if when two things are identical in one system, they are identical for all purposes.
The way I see it, identity is relative to a given system. I’d define it thus: A=B in system S just if for every equivalence relation R that can be constructed in S, R(A,B) is true. “Equivalence relation” is defined in the usual way: reflexive, symmetrical, transitive.
My formulation quantifies over equivalence relations, so it’s not properly a relation in the system itself. It “lives” in any meta-logic about S that supports the definition’s modest components: Ability to distinguish equivalence relations from other types, quantification over equivalence relations in S, ability to apply a variable that’s known to be an equivalence relation, and ability to conjoin an arbitrary number of conjuncts. The fact that it’s not in the system also avoids the potentially paradoxical situation of including ‘=’ among its own conjuncts.
Given my formulation, it’s easily seen that identity needs to be relative to some system. If we were to quantify over all equivalence relations everywhere, we would have to include relations like “Begins with the same letter”, “Has the same ASCII representation”, or “Is printed at the same location on the page”. These relations would fail on A=B and on other equivalences that we certainly should allow at least sometimes. In fact, the
=' test would fail on every two arguments, since the relation "is passed to the NNNNth call to=′ as the same argument index” must fail for those arguments. It could only succeed in a purely Platonic sense. So identity needs to be relative to some system.How can systems differ in what equivalence relations they allow, in ways that are relevant here? For instance, suppose you write a theorem prover in Lisp. In the Lisp code, you definitely want to distinguish symbols that have different names. Their names might even have decomposable meaning, eg in a field accessor like
my-struct-my-field'. So implicitly there is an equivalence relationhas-same-name’ about the Lisp. In the theorem prover itself, there is no such relation as has-same-Lisp-name or even has-same-symbol-in-theorem-prover. (You can of course feed the prover axioms which model this situation. That’s different, and doesn’t give you real access to these distinctions)Your text editor in which you write the Lisp code has yet another different catalog of equivalence relations. It includes many distinctions that are sensitive to spelling or location. They don’t trip us up here, they are just the sort of things that a text editor should distinguish and a theorem prover shouldn’t.
The code in which your text editor is written makes yet other distinctions.
So what about the cases at hand? They are both about logic of belief (doxastic logic). Doxastic logic can contain equivalence relations that fail even on de re equivalent objects. For instance, doxastic logic should be able to say “Alice believes A but not B” even when A and B are both true. Given that sort of expressive capability, one can construct the relation “Alice believes either both A and B or neither”, which is reflexive, symmetrical, transitive; it’s an equivalence relation and it treats A and B differently.
So A and B are not identical here even though de re they are the same.
Great post, Rolf Nelson.
This seems to me a special case of asking “What actually is the phenomenon to be explained?” In the case of free will, or should I say in the case of the free will question, the phenomenon is the perception or the impression of having it. (Other phenomena may be relevant too, like observations of other people making choices between alternatives).
In the case of the socks, the phenomenon to be explained can be safely taken to be the sock-wearing state itself. Though as Eliezer correctly points out, you can start farther back, that is, you can start with the phenomenon that you think you’re wearing socks and ask about it and work your way towards the other.
“Have you stopped beating your wife?” has well-defined true-or-false answers. It’s just that people are generally too stupid to understand what the no-answer actually indicates.
It’s usually given as “Have you stopped beating your wife yet?” (Emph mine). The problem is the presupposition that you have been beating your wife. Either answer accepts (or appears to accept) that presupposition.
It’s a different sort of bad question than the underconstrained questions. The Liar Paradox OTOH is a case of underconstrained question because it contains non-well-founded recursion.
Wrt defining art, I offer my definition:
“An artifact whose purpose is to be perceived and thereby produce in its perceiver a positive experience of no direct practical value to the perceiver.”
“Artifact” here is meant in the sense of being appropriate for Daniel Dennett’s design stance. It is not neccessarily tangible or durable.
This is what’s called a Genus-differentia definition, or type-and-distinction definition. “Artifact” is the type, the rest is the distinction.
This lets me build on existing understandings about artifacts. They have a purpose, but they remain artifacts even when they are not accomplishing that purpose. They are constructed by human beings, but this is a pragmatic fact about human ability and not a part of their definition.
I avoided terms that make no definitional progress such as “beauty” and “aesthetic”. Using them would just be passing the buck.
This definition seems to include birdsong. Make of that what you will. One could reasonably say that birdsong is a fitness signal of direct practical value to the intended perceiver, though.
Under this definition, throw-the-paint art is not so much excluded as it is a marginal, failed, or not serious example, much the way that a hammer (which is another type of artifact) constructed of two twigs scotch-taped together at right angles is a failure as a hammer
Just because there’s a word “art” doesn’t mean that it has a meaning, floating out there in the void, which you can discover by finding the right definition.
True, but it strongly suggests that people who use the term believe there is a referent for it. Sometimes there is none (eg “phlogiston” or “unicorn”). Sometimes the referent is so muddled or misunderstood that the term is has little use except to name the mistake (eg “free will”, which seems to function as a means of grouping quite distinct concepts of subjective freedom together as if they were the same thing, or “qualia” whose referent is a subjective illusion)
But almost always it’s worth asking what they think they mean by it.
@tcpkac, we sometimes call slightly-out-of-focus photos “blurries”. Hope that helps with your important secret project. }:)
I never could understand why people made such a fuss about whether the tree made a sound or not.
Because the sense in which this question is being used as an example here is not the real question that bishop Berkeley had in mind.
It’s really a question about epistemology. It’s related to the “grue” paradox, which is a bit easier to explain. The grue paradox first notes that ordinarily we have good reason to believe that certain things (grass, green paint, copper flames) are green and will continue to be green after (say) 1 January 2009. It then notes that every piece of evidence we have supporting that belief also supports the belief that these things are “grue”, which is defined as being green before 2009 and being blue after that date. On the face of it, we should be equally confident that green paint etc will be blue after 2009.
Much has been written, but the important point is that nobody has ever experienced 2009 (except you lurkers who read posts from previous years. Just change 2009 to a date that’s still in your future, or have they forgotten how to do that in the future?)
A similar condition applies with Berkeley’s paradox. Tautologically, nobody has ever heard a tree fall that nobody heard. (Planting a tape recorder or radio transmitter and listening to that counts as hearing it) So when we guess that the falling tree makes a sound, we are extrapolating. There is no way to test that extrapolation, so how can it be justified?
I recommend David Deutsch’s Four threads of reality for some intelligent and not too wordy comments on how, among other interesting topics he covers.
IMO there’s less to Newcomb’s paradox than meets the eye. It’s basically “A future-predicting being who controls the set of choices could make rational choices look silly by making sure they had bad outcomes”. OK, yes, he could. Surprised?
What I think makes it seem paradoxical is that the paradox both assures us that Omega controls the outcome perfectly, and cues us that this isn’t so (“He’s already left” etc). Once you settle what it’s really saying either way, the rest follows.
No matter how many of McGee’s bets you take, you can always take one more bet and expect an even higher payoff. It’s like asking for the largest integer. There isn’t one, and there isn’t an optimal plan in McGee’s dilemma.
Yes, the inability to name a largest number seems to underlie the infinity utility paradoxes. Which is to say, they aren’t really paradoxes of utility unless one believes that “name a number and I’ll give you that many dollars” is also a paradox of utility. (Or ”...and I’ll give you that many units of utility”)
It’s true that the genie can always correct the wisher by pointing out that the wisher could have accepted one more offer, but in the straightforward “X dollars” example the genie can also always correct the wisher along the same lines by naming a larger number of dollars that he could have asked for.
It doesn’t prove that the wisher doesn’t want to maximize utility, it proves that the wisher cannot name a largest number, which isn’t about his preferences.
Perhaps a lot of confusion could have been avoided if the point had been stated thus:
One’s decision should be no different even if the odds of the situation arising that requires the decision are different.
Footnote against nitpicking: this ignores the cost of making the decision itself. We may choose to gather less information and not think as hard for decisions about situations that are unlikely to arise. That factor isn’t relevant in the example at hand.
@Alan Crowe:
FWIW, having tried that tack a few times, I’ve always been disappointed. The answer is always along the lines of “I’m not meeting any psychological need, I’m searching sincerely for the truth.”
But what I found even more fascinating was the qualitative distinction between “certain” and “uncertain” arguments, where if an argument is not certain, you’re allowed to ignore it. Like, if the likelihood is zero, then you have to give up the belief, but if the likelihood is one over googol, you’re allowed to keep it.
I think that’s exactly what’s going on. These people you speak of who do this are mentally dealing with social permission, not with probability algebra. The non-zero probability gives them social permission to describe it as “it might happen”, and the detail that the probability is 1 / googolplex stands a good chance of getting ignored, lost, or simply not appreciated. (Similarly, the tiny uncertainty)
And I don’t just mean that it works in conversation. The person who makes this mistake has probably internalized it too.
It struck me that way when I read your opening anecdote. Your interlocutor talked like a lawyer who was planning on bringing up that point in closing arguments—“Mr Yudkowsky himself admitted there’s a chance apes and humans are not related”—and not bringing up the minuscule magnitude of the chance, of course.
The selection pressure for faith is almost surely memetic, not genetic. You can focus on the genetic adaptations that it hijacked, but in doing so you will miss the big picture.
Secondly, for understanding religion, I strongly recommend Pascal Boyer’s Religion Explained.
In other words, “what is well-being?”, in such terms that we can apply it to a completely alien situation. This is an important issue.
One red herring, I think, is this:
That could be read two ways. One way is the way that you and these psychologists are reading it. Another interpretation is that the subjects estimated the impact on their future well-being correctly, but after the events, they reported their happiness with respect to their new baseline, which became adjusted to their new situation. The second thing is effectively the derivative of the first. In this interpretation the subjects’ mistake is confusing the two.