Scientist by training, coder by previous session,philosopher by inclination, musician against public demand.
Team Piepgrass: “Worried that typical commenters at LW care way less than I expected about good epistemic practice. Hoping I’m wrong.”
TAG
Karma: 1,199
You can only get from the premise “we can only know our own maps” to the conclusion “we can only care about our own maps” via the minor premise “you can only care about what you fully understand ”. That premise is clearly wrong: one can care about unknown reality, just as one can care about the result of a football match that hasn’t happened yet. A lot of people do care about reality directionally.
Embedded agents are in the territory. How helpful that is depends on the territory
you can model the territory under consideration well enough to make the map-territory distinction illusory.
Well,no. A perfect map is still a map. The map territory distinction dies not lie in imperfect representation alone.
To specify the Universe, you only have to specify enough information to pick it out from the landscape of all possible Universes
Of course not. You have to specify the landscape itself, otherwise it’s like saying “page 273 of [unspecified book]” .
According to string theory (which is a Universal theory in the sense that it is Turing-complete)
As far as I can see, that is only true in that ST allows Turing machines to exist physically. That’s not the kind s of Turing completeness you want. You want to know that String Theory is itself Turing computable, not requiring hypercomputation. Or whatever is actually the ultimate physical theory. Because K complexity doesn’t work other wise. And the computability of physics is far from a given:-
https://en.m.wikipedia.org/wiki/Computability_in_Analysis_and_Physics
Note that the fact that a theory might consist of a small number differential equations is quite irrelevant, because any one equation could be uncomputable.
They are not the same things though. Quantum mechanical measure isn’t actually a head count, like classical measure. The theory doesn’t say that—it’s an extraneous assumption. It might be convenient if it worked that way, but that would be assuming your conclusion.
QM measure isn’t probability—the probability of something occurring or not—because all possible branches occur in MWI.
Another part of the problem stems from the fact that what other people experience is relevant to them, whereas for a probability calculation, I only need to be able to statistically predict my own observations. Using QM to predict my own observations, I can ignore the question of whether something has a ten percent chance of happening in the one and only world, or a certainty of happening in one tenth of possible worlds. However, these are not necessarily equivalent ethically.
@Dagon
This comes down to a HUGE unknown—what features of reality need to be replicated in another medium in order to result in sufficiently-close results
That’s at least two unknowns: what needs to be replicated in order to get the objective functioning; and what needs to be replicated to get the subjective awarness as well.
Which is all just to say—isn’t it much more likely that the problem has been solved, and there are people who are highly confident in the solution because they have verified all the steps that led them there, and they know with high confidence which features need to be replicated to preserve consciousness...
And how do they that, in terms of the second problem? The final stage would need to be confirmation of subjective awareness. We don’t have instruments for that, and it’s no good just asking the sim, since a functional duplicate is likely to answer yes, even if it’s a zombie.
And that’s why it it can be argued that consciousness is a uniqueness difficult problem, beyond the “non-existent proof”
because “find the correct solution” and “convince people of a solution” are mostly independent problems,
That’s not just a theoretical possibility People , eg. Dennett,keep claiming to have explained consciousness, and other people keep being unconvinced because they notice they have skipped the hard part.
“That’s just saying he hasn’t explained some invisible essence of consciousness , equivalent to élan vital”.
“Qualia aren’t invisible, they are the most obvious thing there is to the person that has them”.
Physicalist epiphenomenalism is the only philosophy that is compatible with the autonomy of matter and my experience of consciousness, so it has not competitors as a cosmovision
No, identity theory and illusionism are competitors. And epiphenenomenalism is dualism, not physicalism. As I have pointed out before.
And one of Wallace’s axioms, which he calls ‘branching indifference’, essentially says that it doesn’t matter how many branches there are, since macroscopic differences are all that we care about for decisions..
The macroscopically different branches and their weights?
Focussing on the weight isn’t obviously correct , ethically. You cant assume that the answer to “what do I expect to see” will work the same as the answer to “what should I do”. Is-ought gap and all that.
Its tempting to think that you can apply a standard decision theory in terms of expected value to Many Worlds, since it is a matter of multiplying subjective value by probability. It seems reasonable to assess the moral weight of someone else’s experiences and existence from their point of view. (Edit: also, our experiences seem fully real to us, although we are unlikely to be in a high measure world) That is the intuition behind the common rationalist/utilitarian/EA view that human lives don’t decline in moral worth with distance. So why should they decline with lower quantum mechanical measure?
There is quandary here: sticking to the usual “adds up to normality” principle,as an apriori axiom means discounting the ethical importance of low-measure worlds in order to keep your favourite decision theory operating in the usual single-universe way...even if you are in a multiverse. But sticking to the equally usual universalist axiom, that you dont get to discount someone’s moral worth on the basis of factors that aren’t intrinsic to them, means you should not discount..and that the usual decision theory does not apply.
Basically, there is a tension between four things Rationalists are inclined to believe in:-
-
Some kind of MWI is true.
-
Some kind of utilitarian and universalist ethics is true.
-
Subjective things like suffering are ethically relevant. It’s not all about number of kittens
-
It’s all business as normal...it all adds up to normality.. fundamental ontological differences should not affect your decision theory.
-
According the many-worlds interpretation (MWI) of quantum mechanics, the universe is constantly splitting into a staggeringly large number of decoherent branches containing galaxies, civilizations, and people exactly like you and me
There is more than one many worlds interpretation. The version stated above is not known to be true.
There is an approach to MWI based on coherent superpositions, and a version based on decoherence. These are (for all practical purposes) incompatible. Coherent splitting gives you the very large numbers of “worlds”..except that they are not worlds, conceptually.
Many worlders are pointing at something in the physics and saying “that’s a world”....but whether it qualifies as a world is a separate question , and a separate kind of question, from whether it is really there in the physics. One would expect a world, or universe, to be large, stable, non-interacting, objective and so on . A successful MWI needs to jump three hurdles: mathematical correctness, conceptual correctness, and empirical correctness.
Decoherent branches are expected to be large, stable, non interacting, objective and irreversible...everything that would be intuitively expected of a “world”. But there is no empirical evidence for them , nor are they obviously supported by the core mathematics of quantum mechanics, the Schrödinger equation. Coherent superpositions are small scale , down to single particles, observer dependent, reversible, and continue to interact (strictly speaking , interfere) after “splitting”.
(Note that Wallace has given up on the objectivity of decoherent branches. That’s another indication that MWI is not a single theory).
There isn’t the slightest evidence that irrevocable splitting, splitting into decoherent branches occurs at every microscopic event—that would be combining the frequency of coherent style splitting with the finality of decoherent splitting. We dont know much about decoherence , but we know it is a multi-particle process that takes time, so decoherent splitting, if there is such a thing, must be rarer than the frequency of single particle interactions. ( And so decoherence isn’t simple ). As well as the conceptual incoherence, there is In fact plenty of evidence—eg. the existence of quantum computing—that it doesnt work that way
Also see
I’m not going to argue for this view as that was done very well by Eliezer in his Quantum Physics.
Which view? Everetts view? DeWitts view? Deutsch’s Zeh’s view? Wallace’s view? Saunders view?
I feel like branches being in fact an uncountable continuum is essentially a given
Decoherent branches being being countable, uncountable, or anything else is not given, since there is no established theory of of decoherence.
It’s a given that some observables have continuous spectra..but what’s that got to do with splitting? A observed state that isn’t sharp (in some basis) can get entangled with an apparatus, which then goes into a non-sharp state, and so on. And the whole shebang never splits , or becomes classically sharp.
I mean that the amount of universes that is created will be created anyway, just as a consequence of time passing. So it doesn’t matter anyway. If your actions e.g. cause misery in 20% of those worlds, then the fraction is all that matters; the worlds will exist anyway, and the total amount is not something you’re affecting or controlling.
That’s a special case of “no moral responsibility under determinism”. which might be true , but it’s very different from “utilitarianism works fine under MWI”.
**Enough of the physics confusions—onto the ethics confusions!”″
As well as confusion over the correct version of many worlds, there is of course confusion about which theory of ethics is correct.
There’s broadly three areas where MWI has ethical implications. One is concerned with determinism, freedom of choice, and moral responsibility. One is over the fact that MW means low probability events have to happen every time—as opposed to single universe physics, where they usually don’t. The other is over whether they are discounted in moral significance for being low in quantum mechanical measure or probability
MWI and Free Will
MWI allows probabilities of world states to change over time, but doesn’t allow them to be changed, in a sense amounting to libertarian free will. Agents are just part of the universal wave function, not anything outside the system, or operating by different rules.MWI is, as it’s proponents claim, a deterministic theory, and it only differs from single world determinism in that possible actions can’t be refrained from, and possible futures can’t be avoided. Alternative possibilities are realities, in other words.
MWI, Moral Responsibility, and Refraining.
A standard argument holds that causal determinism excludes libertarian free will by removing alternative possibilities. Without alternative possibilities, you could but have done other than you did, and , the argument goes, you cannot be held responsible for what you had no choice but to do.
Many world strongly implies that you make all possible decisions: according to David Deutsch’s argument that means it allows alternative possibilities, and so removes the objection from moral responsibility despite being a basically deterministic theory.
However, deontology assumed that performing a required act involves restraining from alternatives.. and that it is possible to retain from forbidden acts. Neither is possible under many worlds. Many worlds creates the possibility, indeed the necessity, of doing otherwise, but removes the possibility of refraining from an act. Even though many worlds allows Alternative Possibilities, unfortunately for Deutschs argument, that other aspects create a similar objection on the basis of moral responsibility: why would you hold someone morally responsible for an act if they could not refrain from it?
MWI, Probability, and Utilitarian Ethics
Its tempting to think that you can apply a standard decision theory in terms of expected value to Many Worlds, since it is a matter of multiplying subjective value by probability. One wrinkle is that QM measure isn’t probability—the probability of something occurring or not—because all possible branches occur in MWI. Another is that it is reasonable to assess the moral weight of someone else’s experiences and existence from their point of view. That is the intuition behind the common rationalist/utilitarian/EA view that human lives don’t decline in moral worth with distance. So why should they decline with lower quantum mechanical measure? There is quandary here: sticking to the usual “adds up to normality” principle,as an apriori axiom means discounting the ethical importance of low-measure worlds in order to keep your favourite decision theory operating in the usual single-universe way, even if you are in a multiverse. But sticking to the equally usual universalist axiom, that that you dont get to discount someone’s moral worth on the basis of factors that aren’t intrinsic to them, means you should not
Measure is not probability.
Mathematically, Quantum mechanical measure—amplitude—isn’t ordinary probability, which is why you need the Born rule.The point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Ontologcally, it also not probability, because it does not represent the likelihood of one happening instead of another. And it has its own role, unlike that if ordinary probability, which is explaining how much contribution to a coherent superposition each component state makes (although what that means in the case of irrevocably decohered branches is unclear)
Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice.
The Ethical Weight or Low Measure Worlds
MWI creates the puzzle that low probability outcomes still happen, and have to be taken into account ethically. Many rationalists assume that they simply matter less, because that is the only way to restore anything like a normal view of ethical action—but one should not assume something merely because it is convenient.
It can be argued that most decision theoretic calculations come out the same under different interpretations of QM...but altruistic ethics is different. In standard decision theory, you can tell directly how much utility you are getting; but in altruistic ethics , you are not measuring your suffering/happiness, you are assessing someone else’s...and in the many worlds setting, that means solving the problem of how they are affected by their measure. It is not clear how low measure worlds should be considered in utilitarian ethics. It’s tempting to ethically discount low measure worlds in some way, because that most closely approximates conventional single world utilitarianism. The alternative might force one to the conclusion that overall good outcomes are impossible to attain , so long as one cannot reduce the measures of worlds full of suffering zero. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn’t comes with built in ethics
One part of the problem is that QM measure isn’t probability, because all possible branches occur in MWI. Another stems from the fact that what other people experience is relevant to them, wheareas for a probability calculation, I only need to be able to statistically predict my own observations.. Using QM to predict my own observations, I can ignore the question of whether something has a ten percent chance of happening in the one and only world, or a certainty of happening in one tenth of possible worlds. However, these are not necessarily.equivalent ethically.
Suppose they low measure worlds are discounted ethically. If people in low measure worlds experience their suffering fully, then a 1%, of creating a hell-world would be equivalent in suffering to a 100% chance, and discount is unjustified. But if people in low measure worlds are like philosophical zombies, with little or no phenomenal consciousness, so that their sensations are faint or nonexistent, the moral hazard is much lower, and the discount is justified. A point against discounting is that our experiences seem fully real to us, although we are unlikely to be in a high measure world
A similar, but slightly less obvious argument applies to causing death. Causing the “death” of a complete zombie is presumably as morally culpable as causing the death of a character in a video game...which, by common consent, is not problem at all. So… causing the death of a 50% zombie would be only half as bad as killing a real person...maybe.
Classical Measure isn’t Quantum Mechanical Measure
A large classical universe is analogous to Many Worlds in that the same structures—the same people and planets—repeat over long distances. It’s even possible to define a measure, by counting repetitions up to a certain level of similarity. And one has the option if thinking about Quantum Mechanical measure that way,as a “head count”....but one is not forced to do so. On one hand, it features normality, on the other hand It is not “following the maths” ,because there’s nothing in the formalism to suggest summing a number of identical low measure states is the only way to get a high measure one. So, again, it’s an extraneous assumption, and circular reasoning .
Ethical Calculus is not Decision Theory
Of course, MWI doesn’t directly answer the question about consciousness and zombiehood .You can have objective information about observations, and if your probability calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows physics to be less wrong. And you can have subjective information about your own mental states, and if your personal calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows personal decision theory to be less wrong.
Altruistic ethics is different. You don’t have either kind of direct evidence, because you are concerned with other people’s subjective sensations , not objective evidence , or your own subjectivity. Questions about ethics are downstream of questions about qualia, and qualia are subjective, and because they are subjective, there is no reason to expect them to behave like third person observations.
“But it all adds up to normality!”
If “it all” means every conjecture you can come up with, no It doesn’t. Most conjectures are wrong. The point of empirical testing is to pick out the right ones—the ones that make correct predictions, save appearances, add up to normality That’s a difficult process, not something you get for free.
So “it all adds up to normality” is not some universal truth And ethical theories relating to someone else’s feelings are difficult to test, especially if someone else is in the far future, or an unobservable branch of the multiverse. Testability isn’t an automatic given either.
There are no major ethical implications at all...Wallace makes a similar claim in his book: “But do [the many worlds in MWI] matter to ordinary, banal thought, action and language? Friendship is still friendship. Boredom is still boredom. Sex is still sex
That’s very narrow circle ethics, if it’s ethics at all—he just likes a bunch of things that impact him directly And it’s rather obvious that small circle ethical theories have the least interaction with large universe physical theories. So it likely he hasn’t even considered the question of altruistic ethics in many worlds, and is therefore coming to the conclusion that it all adds up to normality rather cheaply. It’s his ethical outlook that is the structural element , not his take on MWI.
Every quantum event splits the multiverse, so my measure should decline by 20 orders of magnitude every second.
There isn’t the slightest evidence that irrevocable splitting, splitting into decoherent branches occurs at every microscopic event—that would be combining the frequency of coherentism style splitting with the finality of decoherent splitting. As well as the conceptual incoherence, there is In fact plenty of evidence—eg. the existence of quantum computing—that it doesnt work that way
“David Deutsch, one of the founders of quantum computing in the 1980s, certainly thinks that it would. Though to be fair, Deutsch thinks the impact would “merely” be psychological – since for him, quantum mechanics has already proved the existence of parallel uni- verses! Deutsch is fond of asking questions like the following: if Shor’s algorithm succeeds in factoring a 3000-digit integer, then where was the number factored? Where did the computational resources needed to factor the number come from, if not from some sort of “multiverse” exponentially bigger than the universe we see? To my mind, Deutsch seems to be tacitly assuming here that factoring is not in BPP – but no matter; for purposes of argument, we can certainly grant him that assumption. It should surprise no one that Deutsch’s views about this are far from universally accepted. Many who agree about the possibil- ity of building quantum computers, and the formalism needed to describe them, nevertheless disagree that the formalism is best inter- preted in terms of “parallel universes.” To Deutsch, these people are simply intellectual wusses – like the churchmen who agreed that the Copernican system was practically useful, so long as one remembers that obviously the Earth doesn’t really go around the sun. So, how do the intellectual wusses respond to the charges? For one thing, they point out that viewing a quantum computer in terms of “parallel universes” raises serious difficulties of its own. In particular, there’s what those condemned to worry about such things call the “preferred basis problem.” The problem is basically this: how do we define a “split” between one parallel universe and another? There are infinitely many ways you could imagine slic- ing up a quantum state, and it’s not clear why one is better than another! One can push the argument further. The key thing that quan- tum computers rely on for speedups – indeed, the thing that makes quantum mechanics different from classical probability theory in the first place – is interference between positive and negative amplitudes. But to whatever extent different “branches” of the multiverse can usefully interfere for quantum computing, to that extent they don’t seem like separate branches at all! I mean, the whole point of inter- ference is to mix branches together so that they lose their individual identities. If they retain their identities, then for exactly that reason we don’t see interference. Of course, a many-worlder could respond that, in order to lose their separate identities by interfering with each other, the branches had to be there in the first place! And the argument could go on (indeed, has gone on) for quite a while. Rather than take sides in this fraught, fascinating, but perhaps ultimately meaningless debate...”..Scott Aaronson , QCSD, p148
Also see
It seems common for people trying to talk about AI extinction to get hung up on whether statements derived from abstract theories containing mentalistic atoms can have objective truth or falsity values. They can. And if we can first agree on such basic elements of our ontology/epistemology as that one agent can be objectively smarter than another, that we can know whether something that lives in a physical substrate that is unlike ours is conscious, and that there can be some degree of objective truth as to what is valuable [not that all beings that are merely intelligent will necessarily pursue these things], it in fact becomes much more natural to make clear statements and judgments in the abstract or general case, about what very smart non-aligned agents will in fact do to the physical world.
Why does any of that matter for AI safety? AI safety is a matter of public policy. In public policy making, you have a set of preferences, which you get from votes or surveys, and you formulate policy based on your best objective understanding of cause and effect. The preferences don’t have to be objective, because they are taken as given. It’s quite different to philosophy, because you are trying to achieve or avoid something, not figure out what something ultimately is. You do t have to answer Wolfram’s questions in their own terms, because you can challenge the framing.
And if we can first agree on such basic elements of our ontology/epistemology as that one agent can be objectively smarter than another,
It’s not all that relevant to AI safety, because an AI only needs some potentially dangerous capabilities. Admittedly, a lot of the literature gives the opposite impression.
that we can know whether something that lives in a physical substrate that is unlike ours is conscious,
You haven’t defined consciousness and you haven’t explained how . It doesn’t follow automatically from considerations about intelligence. And it doesn’t follow from having some mentalistic terms in our theories.
and that there can be some degree of objective truth as to what is valuable
there doesn’t need to be. You don’t have to solve ethics to set policy.
Arguably, “basic logical principles” are those that are true in natural language.
That’s where the problem starts, not where it stops. Natural language supports a bunch of assumptions that are hard to formally reconcile: if you want your strict PNC, you have to give up on something else. The whole 2500 yeah history of logic has been a history of trying to come up with formal systems that fulfil various desiderata. It is now formally proven that you can’t have all of them at once, and it’s not obvious what to keep and what to ditch. (Godelian problems can be avoided with lower power systems, but that’s another tradeoff, since high power is desirable).
Formalists are happy to pick a system that’s appropriate for a practical domain, and to explore the theoretical properties of different systems in parallel.
Platonists believe that only one axiom system has truth in addition to usefulness, but can’t agree which one it is, so it makes no difference in practice
I’m not seeing a specific problem with sets—you can avoid some of the problems of naive self theory by adding limitations, but that’s tradeoffs again.
Otherwise nothing stops us from considering absurd logical systems where “true and true” is false, or the like.
“You can’t have all the intuitive principles in full strength in one system”
doesn’t imply
“adopt unintuitive axioms”.
Even formalists don’t believe all axiomisations are equally useful.
Likewise, “one plus one is two” seems to be a “basic mathematical principle” in natural language.
What’s 12+1?
Any axiomatization which produces “one plus one is three” can be dismissed on grounds of contradicting the meanings of terms like “one” or “plus” in natural language.
They’re ambiguous in natural language, hence the need for formalisation.
The trouble with set theory is that, unlike logic or arithmetic, it often doesn’t involve strong intuitions from natural language.
It involves some intuitions . It works like clubs. Being a senator is being a member of a set, not exemplifying a universal.
Sets are a fairly artificial concept compared to natural language collections (empty sets, for example, can produce arbitrary nestings), especially when it comes to infinite sets.
If you want finitism, you need a principled way to select a largest finite number.
However, I find myself appealing to basic logical principles like the law of non-contradiction.
The law of non contradiction isn’t true in all “universes” , either. It’s not true in paraconsistent logic, specifically.
Yes, and Logan is claiming that arguments which cannot be communicated to him in no more than two sentences suffer from a conjunctive complexity burden that renders them “weak”.
@Logan Zoellner being wrong doesn’t make anyone else right. If the actual argument is conjunctive and complex, then all the component claims need to be high probability. That is not the case. So Logan is right for not quite the right reasons—it’s not length alone.
That’s not trivial. There’s no proof that there is such a coherent entity as “human values”, there is no proof that AIs will be value-driven agents, etc, etc. You skipped over 99% of the Platonic argument there.
Many possible objections here, but of course spelling everything out would violate Logan’s request for a short argument.
And it wouldn’t help anyway. I have read the Sequences , and there is nothing resembling a proof , or even strong argument, for the claim about coherent human values. Ditto the standard claims about utility functions, agency , etc. Reading the sequence would allow him to understand the LessWrong collective, but should not persuade him.
Whereas the same amount of time could, more reasonably, be spent learning how AI actually works.
Needless to say, that request does not have anything to do with effectively tracking reality,
Tracking reality is a thing you have to put effort into, not something you get for free, by labelling yourself a rationalist.
The original Sequences have did not track reality , because they are not evidence based—they are not derived from academic study or industry experience. Yudkowsky is proud that they are “derived from the empty string”—his way of saying that they are armchair guesswork.
His armchair guesses are based on Bayes,von Neumann rationality, utility maximisation, brute force search etc, which isnt the only way to think about AI, or particularly relevant to real world AI. But it does explain many doom arguments, since they are based on the same model—the kinds of argument that immediately start talking about values and agency. But of course that’s a problem in itself. The short doomer arguments use concepts from the Bayes/VonNeumann era in a “sleepwalking” way, out of sheer habit, given that the basis is doubtful. Current examples of AIs aren’t agents, and it’s doubtful whether they have values. It’s not irrational to base your thinking on real world examples, rather than speculation.
In addition , they haven’t been updated in the light of new developments , something else you have to do to track reality. tracking reality has a cost—you have to change your mind and admit you are wrong. If you don’t experience the discomfort of doing that, you are not tracking reality.
People other than Yudkowsky have written about AI safety from the perspective of how real world AIs work, but adding that injust makes the overall mass of information larger and more confusing.
where there is no “platonic” argument for any non-trivial claim describable in only two sentence, and yet things continue to be true
You are confusing truth and justification.
You need to say something about motivation.
There are dozens of independent ways in which AI can cause a mass extinction event at different stages of its existence.
While each may have around a 10 percent chance a priori, cumulatively there is more than a 99 percent chance that at least one bad thing will happen.
Same problem. Yes, there’s lots of means. That’s not the weak spot. The weak spot is motivation.
Same problem. You’ve done nothing to fill the gap between “ASI will happen” and “ASI will kill us all”.
As other people have said, this is a known argument; specifically, it’s in The Generalized Anti-Zombie Principle in the Physicalism 201 series. From the very early days of LessWrong
Albert: “Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules.”
I think this proof relies on three assumptions. The first (which you address in the post) is that consciousness must happen within physics. (The opposing view would be substance dualism where consciousness causally acts on physics from the outside.) The second (which you also address in the post) is that consciousness and reports about consciousness aren’t aligned by chance. (The opposing view would be epiphenomenalism, which is also what Eliezer trashes extensively in this sequence.) physical duplicate might do the same, although. that would imply the original’s consciousness is epiphenomenal. Which is itself a reason to disbelieve in p-zombies , although not an impossibility proof.
This of course contradicts the Generalised Anti Zombie Principle announced by Eliezer Yudowsky. The original idea was that in a zombie world, it would be incredibly unlikely for an entity’s claims of consciousness to be caused by something other than consciousness. ”
Excluding coincidence doesn’t proved that an entity’s reports of consciousness are directly caused by its own consciousness. Robo-Chalmers will claim to be conscious because Chalmers does. It might actually be conscious, as an additional reason, or it might not. The fact that the claim is made does not distinguish the two cases. Yudkowsky makes much of the fact that Robo-Chalmers claim.would be caused indirectly by consciousness—Chalmers has to be conscious in order to make a computational duplicate of his consciousness—but at best that refutes the possibility of a zombie world, where entities claim to be conscious, although consciousness has never existed. Robo-Chalmers would still be possible in this world for reasons Yudkowsky accepts. So there is one possible kind of zombie, even given physicalism so the Generalised Anti Zombie Principle is false
(Note that I am talking about computational zombies, or c-zombies, not p-zombies
Computationalism isn’t a direct consequence of physicalism. Physicalism has it that an exact atom-by-atom duplicate of a person will be a person and not a zombie, because there is no nonphysical element to go missing. That’s the argument against p-zombies. But if actually takes an atom-by-atom duplication to achieve human functioning, then the computational theory of mind will be false, because there CTM implies that the same algorithm running on different hardware will be sufficient. Physicalism doesn’t imply computationalism, and arguments against p-zombies don’t imply the non existence of c-zombies-duplicates that are identical computationally, but not physically).
That sounds like a Chalmers paper. https://consc.net/papers/qualia.html
Argument length is substantially a function of shared premises
A stated argument could have a short length if it’s communicated between two individuals who have common knowledge of each others premises..as opposed to the “Platonic” form, where every load bearing component is made explicit, and there is noting extraneous.
But that’s a communication issue....not a truth issue. A conjunctive argument doesn’t become likelier because you don’t state some of the premises. The length of the stated argument has little to do with its likelihood.
How true an argument is, how easily it persuades another person, how easy it is to understand have little to do with each other.
The likelihood of an ideal argument depends in the likelihood of it’s load bearing premises...both how many there are, and their individual likelihoods.
Public communication, where you have no foreknowledge of shared premises, needs to keep the actual form closer to the Platonic form.
Public communication is obviously the most important kind when it comes to avoiding AI doom.
This is important, because the longer your argument, the more details that have to be true, and the more likely that you have made a mistake
Correct. The fact that you don’t have to explicitly communicate every step of an argument to a known recipient, doesnt stop the overall probability of a conjunctive argument from depending on the number, and individual likelihood, of the steps of the Platonic version, where everything necessary is stated and nothing unnecessary is stated
Argument strength is not an inverse function with respect to argument length, because not every additional “piece” of an argument is a logical conjunction which, if false, renders the entire argument false.
Correct. Stated arguments can contain elements that are explanatory, or otherwise redundant for an ideal recipient.
Nonetheless, there is a Platonic form, that does not contain redundant elements or unstated, load bearing steps.
Anyways, the trivial argument that AI doom is likely [...]s that it’s not going to have values that are friendly to humans
That’s not trivial. There’s no proof that there is such a coherent entity as “human values”, there is no proof that AIs will be value-driven agents, etc, etc. You skipped over 99% of the Platonic argument there.
This is a classic example of failing to communicate with people outside the bubble. Your assumptions about values and agency just aren’t shared by the general public or political leaders.
PS .
A fact cannot be self evidently true if many people disagree with it.
That’s self evidently true. So why does it have five disagreement downvotes ?
I mean that if turing machine is computing universe according to the laws of quantum mechanics,
I assume you mean the laws of QM except the collapse postulate.
observers in such universe would be distributed uniformly,
Not at all. The problem is that their observations would mostly not be in a classical basis.
not by Born probability.
Born probability relates to observations, not observers.
So you either need some modification to current physics, such as mangled worlds,
Or collapse. Mangled worlds is kind of a nothing burger—its a variation on the idea than interference between superposed states leads to both a classical basi and the Born probabilities, which is an old idea, but wihtout making it any more quantiative.
or you can postulate that Born probabilities are truly random.
??
One might be determined to throw in the towel on cognitive effort if they were to take a particular interpretation of determinism, and they, and the rest of us, would be worse off for it.
Determinists are always telling each other to act like libertarians. That’s a clue that libertarianism is worth wanting. @James Stephen Brown
Compatibilist free will has all the properties worth wanting: your values and beliefs determine the future, to the extent you exert the effort to make good decisions.
No it doesn’t, because it doesn’t have the property of being able to shape the future, or steer towards a future that wasn’t inevitable. Which is pretty important if you are trying to avoid the AI kills everyone future
Libertarian free will is able to do that.
Naturalistic libertarianism appeals to some form of indeterminism, or randomness, inherent in physics rather than a soul or ghost-in-the-machine unique to humans, , that overrides the physical behaviour of the brain. The problem is to explain how indeterminism does not undermine other features of a kind free will “worth wanting”—purposiveness, rationality and so on.
Randomness is not what we want
Explaining NLFW in terms of “randomness” is difficult, because the word has connotations of purposelessness , meaninglessness, and so on. But these are only connotations, not strict implications. “Not deteminism” doesn’t imply lack of reason , purpose , or control. It doesn’t have to separate your from from your beliefs and values. Therefore,I prefer the term “indeterminism” over the term “randomness”.
So, how to explain indeterminism does not undermine other features of a kind free will “worth wanting”.
Part of the answer is to note that mixtures of indeterminism and determinism are possible, so that libertarian free will is not just pure randomness, where any action is equally likely.
Another part is proposing a mechanism , with indeterminism occurring at different places and times, rather than being slathered evenly over neural activity.
Another part is noting that control doesn’t have to mean predetermination.
Another part is that notice that a choice between things you wish to do cannot leave you doing something you do not wish to do, something unconnected to your desires and beliefs.
The basic mechanism is that the unconscious mind proposes various ideas and actions , which a the conscious mind decides between. Thus is similar to the mechanism provided by the determinist Sam Harris. He makes much of the fact that the conscious mind, the executive function, does not predetermined the suggestions: I argue that the choice between them, the decision to act in one rather than another, *is* conscious control. -- and conscious control clearly exists in health adults.
I noticed the same thing—even Scott Alexander dropped a reference to it without explaining it. Anyway, here what I came up with:-
https://www.reddit.com/r/askphilosophy/s/lVNnjhTurI
(That’s me done for another two days)
You are a subject, and you determine your own future
Not so much , given deteminism.
Determinism allows you to cause the future in a limited sense. Under determinism, events still need to be caused,and your (determined) actions can be part of the cause of a future state that is itself determined, that has probability 1.0. Determinism allows you to cause the future ,but it doesn’t allow you to control the future in any sense other than causing it. (and the sense in which you are causing the future is just the sense in which any future state depends on causes in he past—it is nothing special and nothing different from physical causation). It allows, in a purely theoretical sense “if I had made choice b instead of choice a, then future B would have happened instead of future A” … but without the ability to have actually chosen b. You are a link in a deterministic chain that leads to a future state, so without you, the state will not happen … not that you have any choose use in the matter. You can’t stop or change the future because you can’t fail to make your choices, or make them differently. You can’t anything of your own, since everything about you and your choices was determined by at the time of the Big Bang. Under determinism , you are nothing special...only the BB is special.
Tthis is still true under many worlds. even though MWI implies that there is not a single inevitable future, it doesn’t allow you to influence the future in a way that makes future A more likely than future B , as a result of some choice you make now. Under MW determinism, the probabilities of A and B are what they are, and always were—before you make a decision, after you make a decision , and before you were born. You can’t choosee between them, even in the sense of adjusting the probabilities.
Libertarian free will, by contrast, does allow the future to depend on decisions which are not themselves determined. That means there are valid statements of the form “if I had made choice b instead of choice a, then future B would have happened instead of future A”. And you actually could have made choice a or choice b....these are real possibilities, not merely conceptual or logical ones.
Your model of muon decay doesn’t conserve charge—you start with −1e , then have −2e and finally have zero. Also, the second electron is never observed.
If you define “care about” as “put resources into trying to achieve” , there’s plenty of evidence that people care about things that can’t fully define, and don’t fully understand, not least the truth-seeking that happens here.