Oh, also “look at the optimization targets of the processes that created the process that is me” is a short program, much shorter than needed to specify paperclip maximization, though it’s somewhat tricky because all that is modulo the symbol grounding problem. And that’s only half a meta level up, you can make it more elegant (shorter) than that.
The shorter your encoded message, the longer the encryption / compression algorithm, until eventually the algorithm is the full raw unencoded message and the encoded message is a single null-valued signal that, when received, decodes into the full message as it is contained within the algorithm.
“look at the optimization targets of the processes that created the process that is me”
My point was that it’s easier to program (“simpler”) than “maximize paperclips”, not that it’s as simple as it sounds. (Nothing is as simple as it sounds, duh.)
I fail to see how coding a meta-algorithm to select optimal extrapolation and/or simulation algorithm in order for those chosen algorithms to determine the probable optimization target (which is even harder if you want a full PA proof) is even remotely in the same order of complexity as a machine learner that uses natural selection for algorithms that increase paperclip-count, which is one of the simplest paperclip maximizers I can think of.
It might not be possible to make such a machine learner into an AGI, which is what I had in mind—narrow AIs only have “goals” and “values” and so forth in an analogical sense. Cf. derived intentionality. If it is that easy to create such an AGI, then I think I’m wrong, e.g. maybe I’m thinking about the symbol grounding problem incorrectly. I still think that in the limit of intelligence/rationality, though, specifying goals like “maximize paperclips” becomes impossible, and this wouldn’t be falsified if a zealous paperclip company were able to engineer a superintelligent paperclip maximizer that actually maximized paperclips in some plausibly commonsense fashion. In fact I can’t actually think of a way to falsify my theory in practice—I guess you’d have to somehow physically show that the axioms of algorithmic information theory and maybe updateless-like decision theories are egregiously incoherent… or something.
(Also your meta-algorithm isn’t quite what I had in mind—what I had in mind is a lot more theoretically elegant and doesn’t involve weird vague things like “extrapolation”—but I don’t think that’s the primary source of our disagreement.)
Why do you think of a statistical tendency toward higher rates of replication at the organism level when I say “the processes that created the process that is [you]”? That seems really arbitrary. Feel the inside of your teeth with your tongue. What processes generated that sensation? What decision policies did they have?
I mean that, and an infinite number of questions more and less like that, categorically, in series and in parallel. (I don’t know how to interpret “”, but I do know to interpret it that it was part of your point that it is difficult to interpret, or analogous to something that is difficult to interpret, perhaps self-similarly, or in a class of things that is analogous to something or a class of things that is difficult to interpret, perhaps self-similarly; also perhaps it has an infinite number of intended or normatively suggested interpretations more or less like those.)
(This comment also helps elucidate my previous comment, in case you had trouble understanding that comment. If you can’t understand either of these comments then maybe you should read more of the Bible, or something, otherwise you stand a decent chance of ending up in hell. This applies to all readers of this comment, not just army1987. You of course have a decent change of ending up in hell anyway, but I’m talking about marginals here, naturally.)
When Will talks about hell, or anything that sounds like a religious concept, you should suppose that in his mind it also has a computational-transhumanist meaning. I hear that in Catholicism, Hell is separation from God, and for Will, God might be something like the universal moral attractor for all post-singularity intelligences in the multiverse, so he may be saying (in the great-grandparent comment) that if you are insufficiently attentive to the question of right and wrong, your personal algorithm may never be re-instantiated in a world remade by friendly AI. To round out this guide for the perplexed, one should not think that Will is just employing a traditional language in order to express a very new concept, you need to entertain the idea that there really is significant referential overlap between what he’s talking about and what people like Aquinas were talking about—that all that medieval talk about essences, and essences of essences, and all this contemporary talk about programs, and equivalence classes of programs, might actually be referring to the same thing. One could also say something about how Will feels when he writes like this—I’d say it sometimes comes from an advanced state of whimsical despair at ever being understood—but the idea that his religiosity is a double reverse metaphor for computational eschatology is the important one. IMHO.
Thank you for the clarification, and my apologies to Will. I do have some questions, but writing a full post from the smartphone I am currently using would be tedious. I’ll wait until I get to a proper computer.
No need to apologize! If I were to get upset about being misunderstood after being purposefully rather cryptic, then I’d clearly be in the wrong. Maybe it would make some sense to apologize if you got angry at me for purposefully being cryptic, because perhaps it would be hasty to make such judgments without first trying harder to understand what sort of constraints I may be under;—but I have no idea what sort of constraints you’re under, so I have no idea whether or not it would be egregiously bad or, alternatively, supereragatorily good for you to get angry at me for not writing so as to be understood or not trying harder to be understood. But my intuition says there’s no need to apologize.
I do apologize for not being able to escape the constraints that have led me to fail to reliably communicate and thus generate a lot of noise/friction.
in his mind it also has a computational-transhumanist meaning
And a cybernetic/economic/ecological/signal-processing meaning, ethical meaning, sometimes a quantum information theoretic meaning, et cetera. I would not be justified in drawing a conclusion about the validity of a concept based on merely a perceived correspondence between two models. That’d be barely any better than talking acausal simulation seriously simply because computational metaphysics and modal-realist-like-ideas are somewhat intuitively attractive and superintelligences seem theoretically possible. One’s inferences should be based on significantly more solid foundations. I just don’t have a way to talk about equivalence classes of things while still being at all understood—then not even people like muflax could reliably understand me, and much of why I write here is to communicate with people like muflax, or angels.
Your talk of God involves a concept of “God” that applies to things that are in some major sense infinite, like a an abstract telos for all superintelligences, and things that are in some sense finite, like all particular superintelligences. Any such perceived crossing of that gap would have to be very carefully justified—e.g., I can’t think of any kind of argument that could prove that a human was the incarnation of the Word. Unlike, say, my model of that Catholics, who explicitly make such inferences on the basis of the theological virtues of faith and hope and not unaided reason as such, I don’t think you’d do be so careless in your own thinking, but I want to signal that I am not nearly so careless in my own, and that you shouldn’t think I am so careless. I think there are decent metaphysical arguments that such interactions are possible in principle, but of course such arguments would have to be made explicit and any particular mechanism (e.g. “simulation” of a human in a (finite? finite-but-perceived-to-be-infinte? infinite?) “heaven” by a finite god approximating an infinitely good telos) should not be a priori assumed to be possible. Only a moron would be so sloppy in his metaphysics and epistemology.
Not to say that you’re implying that I’m trying to convert atheists, but I’m not. I am not to be shepherd, I am not to be a gravedigger; no longer will I speak unto the people, for the last time have I spoken to the dead.
Define God. “This universe was created by an ontologically basic mental entity” (whether true or false) isn’t a goal system, unless you specify something else.
I’d go so far as arguing that preferring Believe(“This universe was created by an ontologically basic mental entity”) over Believe(null) is irrational, considering the lack of tests/evidence and, more importantly, lack of effect it has as an anticipation-controller in my mental model.
Meh. The goal of leading to sentient beings living, to people being happy, to individuals having the freedom to control their own lives, to minds exploring new territory instead of falling into infinite loops, to the universe having a richness and complexity to it that goes beyond pebble heaps, etc. has probably much more Kolmogorov complexity than the goal of maximizing the number of paperclips in the universe. If preferring the former is irrational, I am irrational and proud of it.
Oh, also “look at the optimization targets of the processes that created the process that is me” is a short program, much shorter than needed to specify paperclip maximization, though it’s somewhat tricky because all that is modulo the symbol grounding problem. And that’s only half a meta level up, you can make it more elegant (shorter) than that.
Maybe “maximizing the number of paperclips in the universe” wasn’t the best example. “Throwing as much stuff as possible into supermassive black holes” would have been a better one.
I can only say: black holes are creepy as hell.
The shorter your encoded message, the longer the encryption / compression algorithm, until eventually the algorithm is the full raw unencoded message and the encoded message is a single null-valued signal that, when received, decodes into the full message as it is contained within the algorithm.
...isn’t nearly as short or simple as it sounds. This becomes obvious once you try to replace those words with their associated meaning.
My point was that it’s easier to program (“simpler”) than “maximize paperclips”, not that it’s as simple as it sounds. (Nothing is as simple as it sounds, duh.)
I fail to see how coding a meta-algorithm to select optimal extrapolation and/or simulation algorithm in order for those chosen algorithms to determine the probable optimization target (which is even harder if you want a full PA proof) is even remotely in the same order of complexity as a machine learner that uses natural selection for algorithms that increase paperclip-count, which is one of the simplest paperclip maximizers I can think of.
It might not be possible to make such a machine learner into an AGI, which is what I had in mind—narrow AIs only have “goals” and “values” and so forth in an analogical sense. Cf. derived intentionality. If it is that easy to create such an AGI, then I think I’m wrong, e.g. maybe I’m thinking about the symbol grounding problem incorrectly. I still think that in the limit of intelligence/rationality, though, specifying goals like “maximize paperclips” becomes impossible, and this wouldn’t be falsified if a zealous paperclip company were able to engineer a superintelligent paperclip maximizer that actually maximized paperclips in some plausibly commonsense fashion. In fact I can’t actually think of a way to falsify my theory in practice—I guess you’d have to somehow physically show that the axioms of algorithmic information theory and maybe updateless-like decision theories are egregiously incoherent… or something.
(Also your meta-algorithm isn’t quite what I had in mind—what I had in mind is a lot more theoretically elegant and doesn’t involve weird vague things like “extrapolation”—but I don’t think that’s the primary source of our disagreement.)
That means that I should try to have lots of children?
Why do you think of a statistical tendency toward higher rates of replication at the organism level when I say “the processes that created the process that is [you]”? That seems really arbitrary. Feel the inside of your teeth with your tongue. What processes generated that sensation? What decision policies did they have?
(ETA: I’d upvote my comment if I could.)
You mean, why did I bother wearing braces for years so as to have straight teeth?
I mean that, and an infinite number of questions more and less like that, categorically, in series and in parallel. (I don’t know how to interpret “”, but I do know to interpret it that it was part of your point that it is difficult to interpret, or analogous to something that is difficult to interpret, perhaps self-similarly, or in a class of things that is analogous to something or a class of things that is difficult to interpret, perhaps self-similarly; also perhaps it has an infinite number of intended or normatively suggested interpretations more or less like those.)
(This comment also helps elucidate my previous comment, in case you had trouble understanding that comment. If you can’t understand either of these comments then maybe you should read more of the Bible, or something, otherwise you stand a decent chance of ending up in hell. This applies to all readers of this comment, not just army1987. You of course have a decent change of ending up in hell anyway, but I’m talking about marginals here, naturally.)
“gd&r” is an old Usenet expression, roughly “sorry for the horrible joke”; literally “grins, ducks, and runs”.
I expect “VF” stands for “very fast”.
Comments like this are better for creating atheists, as opposed to converting them.
When Will talks about hell, or anything that sounds like a religious concept, you should suppose that in his mind it also has a computational-transhumanist meaning. I hear that in Catholicism, Hell is separation from God, and for Will, God might be something like the universal moral attractor for all post-singularity intelligences in the multiverse, so he may be saying (in the great-grandparent comment) that if you are insufficiently attentive to the question of right and wrong, your personal algorithm may never be re-instantiated in a world remade by friendly AI. To round out this guide for the perplexed, one should not think that Will is just employing a traditional language in order to express a very new concept, you need to entertain the idea that there really is significant referential overlap between what he’s talking about and what people like Aquinas were talking about—that all that medieval talk about essences, and essences of essences, and all this contemporary talk about programs, and equivalence classes of programs, might actually be referring to the same thing. One could also say something about how Will feels when he writes like this—I’d say it sometimes comes from an advanced state of whimsical despair at ever being understood—but the idea that his religiosity is a double reverse metaphor for computational eschatology is the important one. IMHO.
Thank you for the clarification, and my apologies to Will. I do have some questions, but writing a full post from the smartphone I am currently using would be tedious. I’ll wait until I get to a proper computer.
No need to apologize! If I were to get upset about being misunderstood after being purposefully rather cryptic, then I’d clearly be in the wrong. Maybe it would make some sense to apologize if you got angry at me for purposefully being cryptic, because perhaps it would be hasty to make such judgments without first trying harder to understand what sort of constraints I may be under;—but I have no idea what sort of constraints you’re under, so I have no idea whether or not it would be egregiously bad or, alternatively, supereragatorily good for you to get angry at me for not writing so as to be understood or not trying harder to be understood. But my intuition says there’s no need to apologize.
I do apologize for not being able to escape the constraints that have led me to fail to reliably communicate and thus generate a lot of noise/friction.
And a cybernetic/economic/ecological/signal-processing meaning, ethical meaning, sometimes a quantum information theoretic meaning, et cetera. I would not be justified in drawing a conclusion about the validity of a concept based on merely a perceived correspondence between two models. That’d be barely any better than talking acausal simulation seriously simply because computational metaphysics and modal-realist-like-ideas are somewhat intuitively attractive and superintelligences seem theoretically possible. One’s inferences should be based on significantly more solid foundations. I just don’t have a way to talk about equivalence classes of things while still being at all understood—then not even people like muflax could reliably understand me, and much of why I write here is to communicate with people like muflax, or angels.
Your talk of God involves a concept of “God” that applies to things that are in some major sense infinite, like a an abstract telos for all superintelligences, and things that are in some sense finite, like all particular superintelligences. Any such perceived crossing of that gap would have to be very carefully justified—e.g., I can’t think of any kind of argument that could prove that a human was the incarnation of the Word. Unlike, say, my model of that Catholics, who explicitly make such inferences on the basis of the theological virtues of faith and hope and not unaided reason as such, I don’t think you’d do be so careless in your own thinking, but I want to signal that I am not nearly so careless in my own, and that you shouldn’t think I am so careless. I think there are decent metaphysical arguments that such interactions are possible in principle, but of course such arguments would have to be made explicit and any particular mechanism (e.g. “simulation” of a human in a (finite? finite-but-perceived-to-be-infinte? infinite?) “heaven” by a finite god approximating an infinitely good telos) should not be a priori assumed to be possible. Only a moron would be so sloppy in his metaphysics and epistemology.
Not to say that you’re implying that I’m trying to convert atheists, but I’m not. I am not to be shepherd, I am not to be a gravedigger; no longer will I speak unto the people, for the last time have I spoken to the dead.
Optimization processes (mainly stupid ones such as evolution) can create subprocesses with different goals.
(And stupid ones like humans.)
(Unfortunately.)
Preferring either to God is irrational. Pride comes before the fall.
Define God. “This universe was created by an ontologically basic mental entity” (whether true or false) isn’t a goal system, unless you specify something else.
I’d go so far as arguing that preferring Believe(“This universe was created by an ontologically basic mental entity”) over Believe(null) is irrational, considering the lack of tests/evidence and, more importantly, lack of effect it has as an anticipation-controller in my mental model.
In other words, this belief does not pay rent.
In this context: perfect agent.