johnsonmx

• johnsonmxApr 5, 2017, 5:21 PM

  2 points

  0

  on: A Prob­lem for the Si­mu­la­tion Hypothesis

  I think the elephant in the room is the purpose of the simulation.

  Bostrom takes it as a given that future intelligences will be interested in running ancestor simulations. Why is that? If some future posthuman civilization truly masters physics, consciousness, and technology, I don’t see them using it to play SimUniverse. That’s what we would do with limitless power; it’s taking our unextrapolated, 2017 volition and asking what we’d do if we were gods. But that’s like asking a 5-year-old what he wants to do when he grows up, then taking the answer seriously.

  Ancestor simulations sound cool to us- heck, they sound amazingly interesting to me, but I strongly suspect posthumans would find better uses for their resources.

  Instead, I think we should try to reason about the purpose of a simulation from first principles.

  Here’s an excerpt from Principia Qualia, Appendix F:

  Why simulate anything?

  At any rate, let’s assume the simulation argument is viable- i.e., it’s possible we’re a simulation, and due to the anthropic math, that it’s plausible that we’re in one now.

  Although it’s possible that we are being simulated but for no reason, let’s assume entities smart enough to simulate universes would have a good reason to do so. So- what possible good reason could there be to simulate a universe? Two options come to mind: (a) using the evolution of the physical world to compute something, or (b) something to do with qualia.

  In theory, (a) could be tested by assuming that efficient computations will exhibit high degrees of Kolmogorov complexity (incompressibility) from certain viewpoints, and low Kolmogorov complexity from others. We could then formulate an anthropic-aware measure for this applicable from ‘within’ a computational system, and apply it to our observable universe. This is outside the scope of this work.

  However, we can offer a suggestion about (b): if our universe is being simulated for some reason associated with qualia, it seems plausible that it has to do with producing a large amount of some kind of particularly interesting or morally relevant qualia.

• johnsonmxMar 27, 2017, 12:27 AM

  3 points

  on: Uneth­i­cal Hu­man Be­hav­ior In­cen­tivised by Ex­is­tence of AGI and Mind-Uploading

  We don’t live in a universe that’s nice or just all the time, so perhaps there are nightmare scenarios in our future. Not all traps have an escape. However, I think this one does, for two reasons.

  (1) all the reasons that RobinHanson mentioned;

  (2) we seem really confused about how consciousness works, which suggests there are large ‘unknown unknowns’ in play. It seems very likely that if we extrapolate our confused models of consciousness into extreme scenarios such as this, we’ll get even more confused results.

• johnsonmxSep 28, 2015, 10:14 PM

  0 points

  in reply to: cameroncowan’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  A rigorous theory of valence wouldn’t involve cultural context, much as a rigorous theory of electromagnetism doesn’t involve cultural context.

  Cultural context may matter a great deal in terms of how to build a friendly AGI that preserves what’s valuable about human civilization—or this may mostly boil down to the axioms that ‘pleasure is good’ and ‘suffering is bad’. I’m officially agnostic on whether value is simple or complex in this way.

  One framework for dealing with the stuff you mention is Coherent Extrapolated Volition (CEV)- it’s not the last word on anything but it seems like a good intuition pump.

• johnsonmxSep 19, 2015, 9:32 PM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  We’re not on the same page. Let’s try this again.

  • The assertion I originally put forth is AI safety; it is not about reverse-engineering qualia. I’m willing to briefly discuss some intuitions on how one may make meaningful progress on reverse-engineering qualia as a courtesy to you, my anonymous conversation partner here, but since this isn’t what I originally posted about I don’t have a lot of time to address radical skepticism, especially when it seems like you want to argue against some strawman version of IIT.

  • You ask for references (in a somewhat rude monosyllabic manner) on “some of the empirical work on coma patients IIT has made possible” and I give you exactly that. You then ignore it as “not really qualia research”- which is fine. But I’m really not sure how you can think that this is completely irrelevant to supporting or refuting IIT: IIT made a prediction, Casali et al. tested the prediction, the prediction seemed to hold up. No qualiometer needed. (Granted, this would be a lot easier if we did have them.)

  This apparently leads to you say,

  You are taking a problem we don;t know how to make a start on, and turning it into a smaller problem we also don’t know how to make a start on.

  More precisely, I’m taking a problem you don’t know how to make a start on, and am turning it into a smaller problem that you also don’t seem to know how to make a start on. Which is fine, and I don’t wish to be a jerk about it, and not merely because Tononi/​Tegmark/​Griffith could be wrong in how they’re approaching consciousness, and I could be wrong in how I’m adapting their stuff to try to explain some specific things about qualia. But you seem to just want to give up, to put this topic beyond the reach of science, and criticize anyone trying to find clever indirect approaches. Needless to say I vehemently disagree with the productiveness of that attitude.

  I think we are in agreement that valence could be a fairly simple property. I also agree that the brain is Vastly Complex, and that qualia research has some excruciatingly difficult methodological hurdles to overcome, and I agree that IIT is still a very speculative hypothesis which shouldn’t be taken on faith. I think we differ radically on our understandings of IIT and related research. I guess it’ll be an empirical question whether IIT morphs into something that can substantially address questions of qualia- based on my understandings and intuitions, I’m pretty optimistic about this.

• johnsonmxSep 18, 2015, 8:38 PM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  If you’re looking for a Full, Complete Data-Driven And Validated Solution to the Qualia Problem, I fear we’ll have to wait a long, long time. This seems squarely in the ‘AI complete’ realm of difficulty.

  But if you’re looking for clever ways of chipping away at the problem, then yes, Casali’s Perturbational Complexity Index should be interesting. It doesn’t directly say anything about qualia, but it does indirectly support Tononi’s approach, which says much about qualia. (Of course, we don’t yet know how to interpret most of what it says, nor can we validate IIT directly yet, but I’d just note that this is such a hard, multi-part problem that any interesting/​predictive results are valuable, and will make the other parts of the problem easier down the line.)

• johnsonmxSep 18, 2015, 12:40 AM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  The stuff by Casali is pretty topical, e.g. his 2013 paper with Tononi.

• johnsonmxSep 17, 2015, 5:41 PM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  Testing hypotheses derived from or inspired by IIT will probably be on a case-by-case basis. But given some of the empirical work on coma patients IIT has made possible. I think it may be stretching things to critique IIT as wholly reliant on circular reasoning.

  That said, yes there are deep methodological challenges with qualia that any approach will need to overcome. I do see your objection quite clearly- I’m confident that I address this in my research (as any meaningful research on this must do) but I don’t expect you to take my word for it. The position that I’m defending here is simply that progress in valence research will have relevance to FAI research.

  Out of curiosity, do you think valence has a large or small kolgoromov complexity?

• johnsonmxSep 17, 2015, 3:18 AM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  I do have some detailed thoughts on your two questions—in short, given certain substantial tweaks, I think IIT (or variants by Tegmark/​Griffiths) can probably be salvaged from its (many) problems in order to provide a crisp dataset on which to base testable hypotheses about qualia.

  (If you’re around the Bay Area I’d be happy to chat about this over a cup of coffee or something.)

  I would emphasize, though, that this post only talks about the value results in this space would have for FAI, and tries to be as agnostic as possible on how any reverse-engineering may happen.

• johnsonmxSep 15, 2015, 6:44 PM

  0 points

  in reply to: TheAncientGeek’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  Are you referring to any specific “current research into qualia”, or just the idea of qualia research in general? I definitely agree that valence research is a subset of qualia research- but there’s not a whole lot of either going on at this point, or at least not much that has produced anything quantitative/​predictive.

  I suspect valence is actually a really great path to approach more ‘general’ qualia research, since valence could be a fairly simple property of conscious systems. If we can reverse-engineer one type of qualia (valence), it’ll help us reverse other types.

• johnsonmxSep 9, 2015, 12:39 AM

  0 points

  in reply to: johnswentworth’s comment on: FAI and the In­for­ma­tion The­ory of Pleasure

  It would probably be highly dependent on the AI’s architecture. The basic idea comes from Shulman and Bostrom—Superintelligence, chapter 9, in the “Incentive methods” section (loc 3131 of 8770 on kindle).

  My understanding is that such a strategy could help as part of a comprehensive strategy of limitations and inventivization but wouldn’t be viable on its own.

FAI and the In­for­ma­tion The­ory of Pleasure

• johnsonmxJun 28, 2015, 1:20 AM

  1 point

  in reply to: cosmicvoyager’s comment on: The mys­tery of pain and pleasure

  Right, absolutely. These are all things that we don’t know, but should.

  Are you familiar with David Pearce’s Hedonistic Imperative movement? He makes a lot of the same points and arguments, basically outlining that it doesn’t seem impossible that we could (and should) radically reduce, and eventually eliminate, suffering via technology.

  But the problem is, we don’t know what suffering is. So we have to figure that out before we can make much radical progress on this sort of work. I.e., I think a rigorous definition of suffering will be an information-theoretic one—that it’s a certain sort of pattern within conscious systems—but we know basically nothing about what sort of pattern it is.

  (I like the word “valence” instead of pain/​pleasure, joy/​suffering, eudaimonia, hedonic tone, etc. It’s a term from psychology that just means ‘the pleasantness or unpleasantness attached to any experience’ and seems to involve less baggage than these other terms.)

  I hope to have a formal paper on this out by this winter. In the meantime, if you’re in the Bay Area, feel free to ping me and I can share some thoughts. You may also enjoy a recent blog post: Effective Altruism, and building a better QALY.

• johnsonmxMar 5, 2015, 6:44 PM

  0 points

  in reply to: minusdash’s comment on: The mys­tery of pain and pleasure

  Although life, sin, disease, redness, maleness, and dogness are (I believe) inherently ‘leaky’ /​ ‘fuzzy’ abstractions that don’t belong with electromagnetism, this is a good comment. If a hypothesis is scientific, it will make falsifiable predictions. I hope to have something more to share on this soon.

• johnsonmxMar 3, 2015, 1:01 AM

  2 points

  in reply to: minusdash’s comment on: The mys­tery of pain and pleasure

  I think we’re still not seeing eye-to-eye on the possibility that valence, i.e., whatever pattern within conscious systems innately feels good, can be described crisply.

  If it’s clear a priori that it can’t, then yes, this whole question is necessarily confused. But I see no argument to that effect, just an assertion. From your perspective, my question takes the form: “what’s the thing that all dogs have in common?”- and you’re trying to tell me it’s misguided to look for some platonic ‘essence of dogness’. Concepts don’t work like that. I do get that, and I agree that most concepts are like that. But from my perspective, your assertion sounds like, “all concepts pertaining to this topic are necessarily vague, so it’s no use trying to even hypothesize that a crisp mathematical relationship could exist.” I.e., you’re assuming your conclusion. Now, we can point to other contexts where rather crisp mathematical models do exist: electromagnetism, for instance. How do you know the concept of valence is more like ‘dogness’ than electromagnetism?

  Ultimately, the details, or mathematics, behind any ‘universal’ or ‘rigorous’ theory of valence would depend on having a well-supported, formal theory of consciousness to start from. It’s no use talking about patterns within conscious systems when we don’t have a clear idea of what constitutes a conscious system. A quantitative approach to valence needs a clear ontology, which we don’t have yet (Tononi’s IIT is a good start, but hardly a final answer). But let’s not mistake the difficulty in answering these questions with them being inherently unanswerable.

  We can imagine someone making similar critiques a few centuries ago regarding whether electromagnetism was a sharply-defined concept, or whether understanding it matters. It turned out electromagnetism was a relatively sharply-defined concept: there was something to get, and getting it did matter. I suspect a similar relationship holds with valence in conscious systems. I’m not sure it does, but I think it’s more reasonable to accept the possibility than not at this point.

• johnsonmxMar 2, 2015, 9:09 PM

  0 points

  in reply to: Richard_Kennaway’s comment on: The mys­tery of pain and pleasure

  Right- good questions.

  First, I think getting a rigorous answer to this ‘mystery of pain and pleasure’ is contingent upon having a good theory of consciousness. It’s really hard to say anything about which patterns in conscious systems lead to pleasure without a clear definition of what our basic ontology is.

  Second, I’ve been calling this “The Important Problem of Consciousness”, a riff off Chalmers’ distinction between the Easy and Hard problems. I.e., if someone switched my red and green qualia in some fundamental sense it wouldn’t matter; if someone switched pain and pleasure, it would.

  Third, it seems to me that patternist accounts of consciousness can answer some of your questions, to some degree, just by ruling out consciousness (things can only experience suffering insofar as they’re conscious). How to rank each of your examples in severity, however, is… very difficult.

• johnsonmxMar 2, 2015, 9:01 PM

  0 points

  in reply to: Lumifer’s comment on: The mys­tery of pain and pleasure

  Right. It might be a little bit more correct to speak of ‘temporal arrangements of arrangements of particles’, for which ‘processes’ is a much less awkward shorthand.

  But saying “pleasure is a neurological process” seems consistent with saying “it all boils down to physical stuff- e.g., particles, eventually”, and doesn’t seem to necessarily imply that “you can’t find a ‘pleasure pattern’ that’s fully generalized. The information is always contextual.”

• johnsonmxMar 2, 2015, 8:56 PM

  1 point

  in reply to: minusdash’s comment on: The mys­tery of pain and pleasure

  Good is a complex concept, not an irreducible basic constituent of the universe. It’s deeply rooted in our human stuff like metabolism (food is good), reproduction (sex is good), social environment (having allies is good) etc

  It seems like you’re making two very distinct assertions here: first, that valence is not a ‘natural kind’, that it doesn’t ‘carve reality at the joints’, and is impossible to form a crisp, physical definition of; and second, that valence is highly connected to drives that have been evolutionarily advantageous to have. The second is clearly correct; the first just seems to be an assertion (one that I understand, and I think reasonable people can hold at this point, but that I disagree with).

• johnsonmxMar 2, 2015, 8:31 AM

  0 points

  in reply to: minusdash’s comment on: The mys­tery of pain and pleasure

  I see the argument, but I’ll note that your comments seem to run contrary to the literature on this: see, e.g., Berridge on “Dissecting components of reward: ‘liking’, ‘wanting’, and learning”, as summed up by Luke in The Neuroscience of Pleasure. In short, behavior, memory, and enjoyment (‘seeking’, ‘learning’, and ‘liking’ in the literature) all seem to be fairly distinct systems in the brain. If we consider a being with a substantially different cognitive architecture, whether through divergent evolution or design, it seems problematic to view behavior as the gold standard of whether it’s experiencing pleasure or suffering. At this point it may be the most practical approach, but it’s inherently imperfect.

  My strong belief is that although there is substantial plasticity in how we interpret experiences as positive or negative, this plasticity isn’t limitless. Some things will always feel painful; others will always feel pleasurable, given a not-too-highly-modified human brain. But really, I think this line of thinking is a red herring: it’s not about the stimulus, it’s about what’s happening inside the brain, and any crisp/​rigorous/​universal principles will be found there.

  Is valence a ‘natural kind’? Does it ‘carve reality at the joints’? Intuitions on this differ (here’s a neat article about the lack of consensus about emotions). I don’t think anger, or excitement, or grief carve reality at the joints- I think they’re pretty idiosyncratic to the human emotional-cognitive architecture. But if anything about our emotions is fundamental/​universal, I think it’d have to be their valence.

• johnsonmxMar 2, 2015, 8:12 AM

  1 point

  in reply to: 27chaos’s comment on: The mys­tery of pain and pleasure

  Surely neurological processes are “arrangements of particles” too, though.

  I think your question gets to the heart of the matter- is there a general principle to be found with regard to which patterns within conscious systems innately feel good, or isn’t there? It would seem very surprising to me if there wasn’t.

• johnsonmxMar 2, 2015, 8:09 AM

  0 points

  in reply to: dxu’s comment on: The mys­tery of pain and pleasure

  I had posted the original in 2013, and did a major revision today, before promoting it (leaving the structure of the questions intact, to preserve previous discussion referents).

  I hope I haven’t committed any faux pas in doing this.