Dach

Karma: 92

Dach Oct 15, 2020, 4:56 AM
2 points
in reply to: Said Achmiz’s comment on: Industrial literacy
Why, exactly, is this our only job (or, indeed, our job at all)? Surely it’s possible to value present-day things, people, etc.?
The space that you can affect is your light cone, and your goals can be “simplified” to “applying your values over the space that you can affect”, therefore your goal is to apply your values over your light cone. It’s you’re “only job”.
There is, of course, a specific notion that I intended to evoke by using this rephrasing: the idea that your values apply strongly over humanity’s vast future. It’s possible to value present-day things, people, and so on- and I do. However… whenever I hear that fact in response to my suggestions that the future is large and it matters more than today, I interpret it as playing defense for their preexisting strategies. Everyone was aware of this before the person said it, and it doesn’t address the central point- it’s...
“There are 4 * 10^20 stars out there. You’re in a prime position to make sure they’re used for something valuable to you- as in, you’re currently experiencing the top 10^-30% most influential hours of human experience because of your early position in human history, etc. Are you going to change your plans and leverage your unique position?”
“No, I think I’ll spend most of my effort doing the things I was already going to do.”
Really- Is that your final answer? What position would you need to be in to decide that planning for the long term future is worth most of your effort?
Seeing as how future humanity (with capital letters or otherwise) does not, in fact, currently exist, it makes very little sense to say that ensuring their existence is something that we would be doing “for” them.
“Seeing as how a couple’s baby does not yet exist, it makes very little sense to say that saving money for their clothes and crib is something that they would be doing ‘for’ them.” No, wait, that’s ridiculous- It does make sense to say that you’re doing things “for” people who don’t exist.
We could rephrase these things in terms of doing them for yourself- “you’re only saving for their clothes and crib because you want them to get what they want”. But, what are we gaining from this rephrasing? The thing you want is for them to get what they want/need. It seems fair to say that you’re doing it for them.
There’s some more complicated discussion to be had on the specific merits of making sure that people exist, but I’m not (currently) interested in having that discussion. My point isn’t really related to that- it’s that we should be spending most of our effort on planning for the long term future.
Also, in the context of artificial intelligence research, it’s an open question as to what the border of “Future Humanity” is. “Existing humans” and “Future Humanity” probably have significant overlap, or so the people at MIRI, DeepMind, OpenAI, FHI, etc. tend to argue- and I agree.

Dach Oct 11, 2020, 1:27 AM
1 point
in reply to: Walker Vargas’s comment on: Not Even Evidence
This doesn’t require faster than light signaling. If you and the copy are sent way with identical letters, that you open after crossing each other’s event horizons. You learn want was packed with your clone when you open your letter. Which lets you predict what your clone will find.
Nothing here would require the event of your clone seeing the letter to affect you. You are affected by the initial set up.
Another example would be if you learn a star that has crossed your cosmic event horizon was 100 solar masses, it’s fair to infer that it will become a black hole and not a white dwarf.
If you can send a probe to a location, radiation, gravitational waves, etc. from that location will also (in normal conditions) be intercepting you, allowing you to theoretically make pretty solid inferences about certain future phenomena at that location. However, we let the probe fall out of our cosmological horizon- information is reaching it that couldn’t/can’t have reached the other probes, or even the starting position of that probe.
In this setup, you’re gaining information about arbitrary phenomena. If you send a probe out beyond your cosmological horizon, there’s no way to infer the results of, for example, non-entangled quantum experiments.
I think we may eventually determine the complete list of rules and starting conditions for the universe/multiverse/etc. Using our theory of everything and (likely) unobtainable amounts of computing power, we could (perhaps) uniquely locate our branch of the universal wave function (or similar) and draw conclusions about the outcomes of distant quantum experiments (and similar). That’s a serious maybe- I expect that a complete theory of everything would predict infinitely many different instances of us in a way that doesn’t allow for uniquely locating ourselves.
However… this type of reasoning doesn’t look anything like that. If SSA/SSSA require us to have a complete working theory of everything in order to be usable, that’s still invalidating for my current purposes.
For the record, I ran into a more complicated problem which turns out to be incoherent for similar reasons- namely, information can only propagate in specific ways, and it turns out that SSA/SSA allows you to draw conclusions about what your reference class looks like in ways that defy the ways in which information can propagate.
You are affected by the initial set up. If the clone counterfactually saw something else, this wouldn’t affect you according to SIA.
This specific hypothetical doesn’t directly apply to the SIA- it relies on adjusting the relative frequencies of different types of observers in your reference class, which isn’t possible using SIA. SIA still suffers from the similar problem of allowing you to draw conclusions about what the space of all possible observers looks like.

Dach Oct 6, 2020, 10:01 PM
1 point
in reply to: ike’s comment on: Not Even Evidence
I don’t understand why you’re calling a prior “inference”. Priors come prior to inferences, that’s the point.
SIA is not isomorphic to “Assign priors based on Kolmogorov Complexity”. If what you mean by SIA is something more along the lines of “Constantly update on all computable hypotheses ranked by Kolmogorov Complexity”, then our definitions have desynced.
Also, remember: you need to select your priors based on inferences in real life. You’re a neural network that developed from scatted particles- your priors need to have actually entered into your brain at some point.
Regardless of whether your probabilities entered through your brain under the name of a “prior” or an “update”, the presence of that information still needs to work within our physical models and their conclusions about the ways in which information can propagate.
SIA has you reason as if you were randomly selected from the set of all possible observers. This is what I mean by SIA, and is a distinct idea. If you’re using SIA to gesture to the types of conclusions that you’d draw using Solomonoff Induction, I claim definition mismatch.
It clearly is unnecessary—nothing in your examples requires there to be tiling, you should give an example with a single clone being produced, complete with the priors SIA gives as well as your theory, along with posteriors after Bayesian updating.
I specifically listed the point of the tiling in the paragraph that mentions tiling:
for you to agree that the fact you don’t see a pink pop-up appear provides strong justified evidence that none of the probes saw <event x>
The point of that the tiling is, as I have said (including in the post), to manipulate the relative frequencies of actually existent observers strongly enough to invalidate SSA/SSSA in detail.
I don’t see any such implications. You need to simplify and more fully specify your model and example.
There’s phenomena which your brain could not yet have been impacted by, based on the physical ways in which information propagates. If you think you’re randomly drawn from the set of all possible observers, you can draw conclusions about what the set of all possible observers looks like, which is problematic.
I don’t see any such implications. You need to simplify and more fully specify your model and example.
Just to reiterate, my post isn’t particularly about SIA. I showed the problem with SSA/SSSA- the example was specified for doing something else.

Dach Oct 6, 2020, 8:35 PM
3 points
in reply to: CronoDAS’s comment on: Industrial literacy
That’s surprisingly close, but I don’t think that counts. That page explains that the current dynamics behind phosphate recycling are bad as a result of phosphate being cheap- if phosphate was scarce, recycling (and potentially the location of new phosphate reserves, etc.) would become more economical.

Dach Oct 6, 2020, 8:14 PM
1 point
in reply to: ike’s comment on: Not Even Evidence
My formulation of those assumptions, as I’ve said, is entirely a prior claim.
You can’t gain non-local information using any method, regardless of the words or models you want to use to contain that information.
If you agree with those priors and Bayes, you get those assumptions.
You cannot reason as if you were selected randomly from the set of all possible observers. This allows you to infer information about what the set of all possible observers looks like, despite provably not having access to that information. There are practical implications of this, the consequences of which were shown in the above post with SSA.
You can’t say that you accept the prior, accept Bayes, but reject the assumption without explaining what part of the process you reject. I think you’re just rejecting Bayes, but the unnecessary complexity of your example is complicating the analysis. Just do Sleeping Beauty with the copies in different light cones.
It’s not a specific case of sleeping beauty. Sleeping beauty has meaningfully distinct characteristics.
This is a real world example that demonstrates the flaws with these methods of reasoning. The complexity is not unnecessary.
I’m asking for your prior in the specific scenario I gave.
My estimate is 2/3rds for the 2-Observer scenario. Your claims that “priors come before time” makes me want to use different terminology for what we’re talking about here. Your brain is a physical system and is subject to the laws governing other physical systems- whatever you mean by “priors coming before time” isn’t clearly relevant to the physical configuration of the particles in your brain.
The fact that I execute the same Bayesian update with the same prior in this situation does not mean that I “get” SIA- SIA has additional physically incoherent implications.

Dach Oct 6, 2020, 7:40 PM
1 point
in reply to: ike’s comment on: Not Even Evidence
The version of the post I responded to said that all probes eventually turn on simulations.
The probes which run the simulations of you without the pop-up run exactly one. The simulation is run “on the probe.”
Let me know when you have an SIA version, please.
I’m not going to write a new post for SIA specifically- I already demonstrated a generalized problem with these assumptions.
The up until now part of this is nonsense—priors come before time. Other than that, I see no reason to place such a limitation on priors, and if you formalize this I can probably find a simple counterexample. What does it even mean for a prior to correspond to a phenomena?
Your entire brain is a physical system, it must abide by the laws of physics. You are limited on what your priors can be by this very fact- there is some stuff that the position of the particles in your brain could not have yet been affected by (by the very laws of physics).
The fact that you use some set of priors is a physical phenomenon. If human brains acquire information in ways that do not respect locality, you can break all of the rules, acquire infinite power, etc.
Up until now refers to the fact that the phenomena have, up until now, been unable to affect your brain.
I wrote a whole post trying to get people to look at the ideas behind this problem, see above. If you don’t see the implication, I’m not going to further elaborate on it, sorry.
All SIA is doing is asserting events A, B, and C are equal prior probability. (A is living in universe 1 which has 1 observer, B and C are living in universe 2 with 2 observers and being the first and second observer respectively. B and C can be non-local.)
SIA is asserting more than events A, B, and C are equal prior probability.
Sleeping Beauty and these hypotheticals here are different- these hypotheticals make you observe something that is unreasonably unlikely in one hypothesis but very likely in another, and then show that you can’t update your confidences in these hypothesis in the dramatic way demonstrated in the first hypothetical.
You can’t change the number of possible observers, so you can’t turn SIA into an FTL telephone. SIA still makes the same mistake that allows you to turn SSA/SSSA into FTL telephones, though.
If you knew for a fact that something couldn’t have had an impact, this might be valid. But in your scenarios, these could have had an impact, yet didn’t. It’s a perfectly valid update.
There really couldn’t have been an impact. The versions of you that wake up and don’t see pop-ups (and their brains) could not have been affected by what’s going on with the other probes- they are outside of one another’s cosmological horizon. You could design similar situations where your brain eventually could be affected by them, but you’re still updating prematurely.
I told you the specific types of updates that you’d be allowed to make. Those are the only ones you can justifiably say are corresponding to anything- as in, are as the result of any observations you’ve made. If you don’t see a pop-up, not all of the probes saw <event x>, your probe didn’t see <event x>, you’re a person who didn’t see a pop-up, etc. If you see a pop-up, your assigned probe saw <event x>, and thus at least one probe saw <event x>, and you are a pop-up person, etc.
However, you can’t do anything remotely looking like the update mentioned in the first hypothetical. You’re only learning information about your specific probe’s fate, and what type of copy you ended up being.
You should simplify to having exactly one clone created. In fact, I suspect you can state your “paradox” in terms of Sleeping Beauty—this seems similar to some arguments people give against SIA there, claiming one does not acquire new evidence upon waking. I think this is incorrect—one learns that one has woken in the SB scenario, which on SIA’s priors leads one to update to the thirder position.
You can’t simplify to having exactly one clone created.
There is a different problem going on here than in the SB scenario. I mostly agree with the 1/3rds position- you’re least inaccurate when your estimate for the 2-Observer scenario is 2/3rds. I don’t agree with the generalized principle behind that position, though. It requires adjustments, in order to be more clear about what it is you’re doing, and why you’re doing it.

Dach Oct 6, 2020, 6:06 PM
1 point
in reply to: ike’s comment on: Not Even Evidence
If you reject both the SIA and SSA priors (in my example, SIA giving ¹⁄₃ to each of A, B, and C, and SSA giving ¹⁄₂ to A and ¹⁄₄ to B and C), then what prior do you give?
I reject these assumptions, not their priors. The actual assumptions and the methodology behind them have physically incoherent implications- the priors they assign may still be valid, especially in scenarios where it seems like there are exactly two reasonable priors, and they both choose one of them.
Whatever prior you give you will still end up updating as you learn information. There’s no way around that unless you reject Bayes or you assert a prior that places 0 probability on the clones, which seems sillier than any consequences you’re drawing out here.
The point is not that you’re not allowed to have prior probabilities for what you’re going to experience. I specifically placed a mark on the prior probability of what I expected to experience in the “What if...” section.
If you actually did the sleeping beauty experiment in the real world, it’s very clear that “you would be right most often when you woke up” if you said you were in the world with two observers.

Dach Oct 6, 2020, 7:00 AM
1 point
in reply to: ike’s comment on: Not Even Evidence
Can you formulate this as a challenge to SIA in particular? You claim that it affects SIA, but your issue is with reference classes, and SIA doesn’t care about your reference class.
The point is that SIA similarly overextends its reach- it claims to make predictions about phenomena that could not yet have had any effect on your brain’s operation, for reasons demonstrated with SSA in the example in the post.
Your probability estimates can only be affected by a pretty narrow range of stuff, in practice, and because SIA does not deliberately draw the line of all possible observers around “All possible observers which could have so far had impact on my probability estimates, as evidenced by the speed of light and other physical restrictions on the propagation of information”, it unfortunately implies that your probability estimates are corresponding with things which, via physics, they can’t be.
Briefly, “You cannot reason about things which could not yet have had an impact on your brain.”
SSSA/SSA are more common, which is why I focused on them. For the record, I used an example in which SSSA and SSA predict exactly the same things. SIA doesn’t predict the same thing here, but the problem that I gestured to is also present in SIA, but with a less laborious argument.
Your probe example is confusingly worded. You include time as a factor but say time doesn’t matter. Can you reduce it to the simplest possible that still yields the paradoxical result you want?
Yea, sorry- I’m still editing this post. I’ll reword it tomorrow. I’m not sure if I’ll remove that specific disclaimer, though.
We could activate the simulated versions of you at any time- whether or not the other members of your reference class are activated at different times doesn’t matter under standard usage of SIA/SSA/SSSA. I’m just including the extra information that the simulations are all spun up at the same time in case you have some weird disagreement with that, and in order to more closely match intuitive notions of identity.
I included that disclaimer because there’s questions to be had about time- the probes are presumably in differently warped regions of spacetime, thus it’s not so clear what it means to say these events are happening at the same time.
I don’t think SIA says you should update in this manner, except very slightly. If I’m understanding your example correctly, all the probes end up tiling their light cones, so the number of sims is equal regardless of what happened. The worlds with fewer probes having seen x become slightly more likely than the prior, but no anthropic reasoning is needed to get that result.
Only the probes which see <event x> end up tiling their light cones. The point is to change the relative frequencies of the members of your reference class. Because SSA/SSSA assume that you are randomly selected from your reference class, by shifting the relative frequencies of different future observations within your reference class SSA/SSSA imply you can gain information about arbitrary non-local phenomena. This problem is present even outside of this admittedly contrived hypothetical- this contrived hypothetical takes an extra step and turns the problem into an FTL telephone.
It doesn’t seem that there’s any way to put your hand on the scale of the number of possible observers, therefore (as previously remarked) this example doesn’t apply to SIA. The notion that SIA is overextending its reach by claiming to make justified claims about things we can show (using physics) you cannot make justified claims about to still applies.
In general, I think of SIA as dictating our prior, while all updates are independent of anthropics. Our posterior is simply the SIA prior conditioned on all facts we know about our own existence. Roughly speaking, SSA represents a prior that we’re equally likely to exist in worlds that are equally likely to exist, while SIA represents a prior that we’re equally likely to be any two observers that are equally likely to exist.
The problem only gets pushed back- we can also assert that your priors cannot be corresponding to phenomena which (up until now) have been non-local to you. I’m hesitant to say that you’re not allowed to use this form of reasoning- in practice using SIA may be quite useful. However, it’s just important to be clear that SIA does have this invalid implication.

Not Even Evidence

DachOct 6, 2020, 1:20 AM

6 points

15 comments8 min readLW link

Dach Oct 5, 2020, 10:48 PM
1 point
in reply to: TAG’s comment on: Industrial literacy
It’s possible, but very improbable. We have vastly more probable concerns (misaligned AGI, etc.) than resource depletion sufficient to cripple the entire human project.
What critical resources is Humanity at serious risk of depleting? Remember that most resources have substitutes- food is food.

Dach Oct 5, 2020, 9:48 PM
1 point
in reply to: Emiya’s comment on: Industrial literacy
Why do you seem to imply that burning fossil fuels would help at all the odds of the long term human project?
I don’t imply that. For clarification:
I would waste any number of resources if that was what was best for the long-term prospects of Humanity. In practice, that means that I’m willing to sacrifice really really large amounts of resources that we won’t be able to use until after we develop AGI or similar, in exchange for very very small increases to our probability of developing aligned AGI or similar.
Because I think we won’t be able to use significant portions of most of the types of resources available on Earth before we develop AGI or similar, I’m willing to completely ignore conservation of those resources. I still care about the side effects of the process of gathering and using those resources, but...
The oil example isn’t meant to be any reflection of my affinity for fossil fuels.
My point that “Super long term conservation of resources” isn’t a concern. If there are near term non “conservation of resources” reasons why doing something is bad, I’m open to those concerns- we don’t need to worry about ensuring that humans 100 years from now have access to fuel sources.
For the record, I think nuclear and solar seem to clearly be better energy sources than fossil fuels for most applications. Especially nuclear.
I’m also not fighting defense for climate change activists- I don’t care about how many species die out, unless those species are useful (short term- next 50 years, 100 years max?) to us. If you want to make sure future humanity has access to Tropical Tree Frog #952, and you’re concerned about them going extinct, go grab some genetic samples and preserve them. If the species makes many humans very happy, provides us valuable resources, etc., fine.
At the current rate of fishing, all fish species could be practically extinct by 2050
I’m open to the notion that regulating our fish intake is the responsible move- it seems like a pretty easy sell. It keeps our fishing equipment, boats, and fishermen useful. I’m taking this action because it’s better for humanity, not because it’s better for the fish or better for the Earth.
The Strategy is not to excessively use resources and destroy the environment just because we can, it’s to actively and directly use our resources to accomplish our goals, which I have doubts strongly aligns with preserving the environment.
Let’s list a few ways in which our conservation efforts are bad:
- Long term (100+ years) storage of nuclear waste.
- Protecting species which aren’t really useful to Humanity.
- Planning with the idea that we will be indefinitely (Or, for more than 100 years) living in the current technological paradigm, i.e. without artificial general intelligence.
And in which they’re valid:
- Being careful with our harvesting of easily depletable species which we’ll be better off having alive for the next 100 years.
- Being careful with our effect on global temperatures and water levels, in order to avoid the costs of relocating large numbers of humans.
- Being careful with our management of important freshwater reserves, at least until we develop sufficiently economical desalinization plants.
I personally don’t want to see my personal odds of survival diminishing because I’ll have to deal with riots, food shortages, totalitarian fascist governments or… who know?
The greatest risks to your survival are, by far, (unless you’re a very exceptional person) natural causes and misaligned artificial general intelligence. You shouldn’t significantly concern yourself with dealing with weird risk factors such as riots or food shortages unless you’ve already found that you can’t do anything about natural causes and misaligned artificial general intelligence. Spoiler: It seems you can do something about these risk factors.
Every economical estimate I saw said that the costs would be a lot less than the economic damage from climate change alone, many estimates agree that it would actually improve the economy, and nobody is saying “let’s toss industry and technology out of the window, back to the caves everyone!”.
Many people are saying things I consider dangerously close to “Let’s toss industry and technology out of the window!”. Dagon suggested that our current resource expenditure was reckless, and that we should substantially downgrade our resource expenditures. I consider this to be a seriously questionable perspective on the problem.
I’m not arguing against preserving the environment if it would boost the economy for at least the next 100 years, keeping in mind opportunity cost. I want to improve humanity’s generalized power to pursue its goals- I’m not attached to any particular short guiding principle for doing this, such as “Protect the Earth!” or “More oil!”. I don’t have Mad Oil Baron Syndrome.

Dach Oct 4, 2020, 11:30 PM
8 points
in reply to: Dagon’s comment on: Industrial literacy
I suspect that if people really understood the cost to future people of the contortions we go through to support this many simultaneous humans in this level of luxury, we’d have to admit that we don’t actually care about them very much. I sympathize with those who are saying “go back to the good old days” in terms of cutting the population back to a sustainable level (1850 was about 1.2B, and it’s not clear even that was sparse/spartan enough to last more than a few millennia).
There’s enough matter in our light cone to support each individual existing human for roughly 10^44 years.
The problem is not “running out of resources”- there are so many resources it will require cosmic engineering for us to use more of them than entropy, even if we multiply our current population by ten billion.
Earth is only one planet- it does not matter how much of earth we use here and now. Our job is to make sure that our light cone ends up being used for what we find valuable. That’s our only job. The finite resources available on earth are almost irrelevant to the long term human project, beyond the extent to which those resources help us accomplish our job- I would burn a trillion pacific oceans worth of oil for a .000000000000000001% absolute increase to our probability of succeeding at our job.
I sympathize with people who are thinking like this, because it shows that they’re at least trying to think about the future. But… Future Humanity doesn’t need the petty resources available on earth any more than we need good flint to make hunting spears with. The only important thing and the best thing we can do for them is to ensure that they will ever exist at all!

Dach Oct 2, 2020, 8:48 AM
1 point
in reply to: TAG’s comment on: The Short Case for Verificationism
The existence of places like LessWrong, philosophy departments, etc, indicate that people do have some sort of goal to understand things in general, aside from any nitpicking about what is a true terminal value.
I agree- lots of people (including me, of course) are learning because they want to- not as part of some instrumental plan to achieve their other goals. I think this is significant evidence that we do terminally value learning. However, the way that I personally have the most fun learning is not the way that is best for cultivating a perfect understanding of reality (nor developing the model which is most instrumentally efficient, for that matter). This indicates that I don’t necessarily want to learn so that I can have the mental model that most accurately describes reality- I have fun learning for complicated reasons which I don’t expect align with any short guiding principle.
Also, at least for now, I get basically all of my expected value from learning from my expectations for being able to leverage that knowledge. I have a lot more fun learning about e.g. history than the things I actually spend my time on, but historical knowledge isn’t nearly as useful, so I’m not spending my time on it.
In retrospect, I should’ve said something more along the lines of “We value understanding in and of itself, but (at least for me, and at least for now) most of the value in our understanding is from its practical role in the advancement of our other goals.”
I’ve already stated than I am not talking about confirming specific models.
There’s been a mix-up here- my meaning for “specific” also includes “whichever model corresponds to reality the best”

Dach Oct 2, 2020, 2:18 AM
1 point
in reply to: ike’s comment on: The Short Case for Verificationism
E.g. “maybe you’re in an asylum” assumes that it’s possible for an asylum to “exist” and for someone to be in it, both of which are meaningless under my worldview.
What do you mean by “reality”? You keep using words that are meaningless under my worldview without bothering to define them.
You’re implementing a feature into your model which doesn’t change what it predicts but makes it less computationally efficient.
The fact you’re saying “both of which are meaningless under my worldview” is damning evidence that your model (or at least your current implementation of your model) sucks, because that message transmits useful information to someone using my model but apparently has no meaning in your model. Ipso facto, my model is better. There’s no coherent excuse for this.
This isn’t relevant to the truth of verificationism, though. My argument against realism is that it’s not even coherent. If it makes your model prettier, go ahead and use it.
What does it mean for your model to be “true”? There are infinitely many unique models which will predict all evidence you will ever receive- I established this earlier and you never responded.
It’s not about making my model “prettier”- my model is literally better at evoking the outcomes that I want to evoke. This is the correct dimension on which to evaluate your model.
You’ll just run into trouble if you try doing e.g. quantum physics and insist on realism—you’ll do things like assert there must be loopholes in Bell’s theorem, and search for them and never find them.
My preferred interpretation of quantum physics (many worlds) was made before bell’s theorem, and it turns out that bell’s theorem is actually strong evidence in favor of many worlds. Bell’s theorem does not “disprove realism”, it just disproves hidden variable theories. My interpretation already predicted that.
I suspect this isn’t going anywhere, so I’m abdicating.

Dach Oct 1, 2020, 10:22 PM
1 point
in reply to: ike’s comment on: The Short Case for Verificationism
This is false. I actually have no idea what it would mean for an experience to be a delusion—I don’t think that’s even a meaningful statement.
I’m comfortable with the Cartesian argument that allows me to know that I am experiencing things.
Everything you’re thinking is compatible with a situation in which you’re actually in a simulation hosted in some entirely alien reality (2 + 2 = 3, experience is meaningless, causes follow after effects, (True ^ True) = False, etc, which is being manipulated in extremely contrived ways which produce your exact current thought processes.
There are an exhausting number of different riffs on this idea- maybe you’re in an asylum and all of your thinking including “I actually have no idea what it would mean for an experience to be a delusion” is due to some major mental disorder. Oh, how obvious- my idea of experience was a crazy delusion all along. I can’t believe I said that it was my daughter’s arm. “I think therefore I am”? Absurd!
If you have an argument against this problem, I am especially interested in hearing it- it seems like the fact you can’t tell between this situation and reality (and you can’t know whether this situation is impossible as a result, etc.) is part of the construction of the scenario. You’d need to show that the whole idea that “We can construct situations in which you’re having exactly the same thoughts as you are right now, but with some arbitrary change (Which you don’t even need to believe is theoretically possible or coherent) in the background” is invalid.
Do I think this is a practical concern? Of course not. The Cartesian argument isn’t sufficient to convince me, though- I’m just assuming that I really exist and things are broadly as they seem. I don’t think it’s that plausible to expect that I would be able to derive these assumptions without using them- there is no epistemological rock bottom.
On the contrary, it’s the naive realist model that doesn’t pay rent by not making any predictions at all different from my simpler model.
Your model is (I allege) not actually simpler. It just seems simpler because you “removed something” from it. A mind could be much “simpler” than ours, but also less useful- which is the actual point of having a simpler model. The “simplest” model which accurately predicts everything we see is going to be a fundamental physical theory, but making accurate predictions about complicated macroscopic behavior entirely from first principles is not tractable with eight billion human brains worth of hardware.
The real question of importance is, does operating on a framework which takes specific regular notice of the idea that naïve realism is technically a floating belief increase your productivity in the real world? I can’t see why that would be the case- it requires occasionally spending my scare brainpower on reformatting my basic experience of the world in more complicated terms, I have to think about whether or not I should argue with someone whenever they bring up the idea of naïve realism, etc. You claim adopting the “simpler” model doesn’t change your predictions, so I don’t see what justifies these costs. Are there some major hidden costs of naïve realism that I’m not aware of? Am I actually wasting more unconscious brainpower working with the idea of “reality” and things “really existing”?
If I have to choose between two models which make the exact same predictions (i.e. my current model and your model), I’m going to choose between the model which is better at achieving my goals. In practice, this is the more computationally efficient model, which (I allege) is my current model.

Dach Oct 1, 2020, 3:12 AM
1 point
in reply to: TAG’s comment on: The Short Case for Verificationism
Refer to my disclaimer for the validity of the idea of humans having terminal values. In the context of human values, I think of “terminal values” as the ones directly formed by evolution and hardwired into our brains, and thus broadly shared. The apparent exceptions are rarish and highly associated with childhood neglect and brain damage.
“Broadly shared” is not a significant additional constraint on what I mean by “terminal value”, it’s a passing acknowledgement of the rare counterexamples.
If that’s your argument then we somewhat agree. I’m saying that the model you should use is the model that most efficiently pursues your goals, and (in response to your comment) that utility schemes which terminally value having specific models (and thus whose goals are most efficiently pursued through using said arbitrary terminally valued model and not a more computationally efficient model) are not evidently present among humans in great enough supply for us to expect that that caveat applies to anyone who will read any of these comments.
Real world examples of people who appear at first glance to value having specific models (e.g. religious people) are pretty sketchy- if this is to be believed, you can change someone’s terminal values with the argumentative equivalent of a single rusty musket ball and a rubber band. That defies the sort of behaviors we’d want to see from whatever we’re defining as a “terminal value”, keeping in mind the inconsistencies between the way human value systems are structured and the way the value systems of hypothetical artificial intelligences are structured.
The argumentative strategy required to convince someone to ignore instrumentally unimportant details about the truth of reality looks more like “have a normal conversation with them” than “display a series of colorful flashes as a precursor to the biological equivalent of arbitrary code execution” or otherwise psychologically breaking them in a way sufficient to get them to do basically anything, which is what would be required to cause serious damage to what I’m talking about when I say “terminal values” in the context of humans.

Dach Sep 30, 2020, 7:48 PM
3 points
in reply to: TAG’s comment on: The Short Case for Verificationism
It’s also true for “I terminally value understanding the world, whatever the correct model is”.
I said e.g, not i.e, and “I terminally value understanding the world, whatever the correct model is” is also a case of trivial values.
First, a disclaimer: It’s unclear how well the idea of terminal/instrumental values maps to human values. Humans seem pretty prone to value drift- whenever we decide we like some idea and implement it, we’re not exactly “discovering” some new strategy and then instrumentally implementing it. We’re more incorporating the new strategy directly into our value network. It’s possible (Or even probable) that our instrumental values “sneak in” to our value network and are basically terminal values with (usually) lower weights.
Now, what would we expect to see if “Understanding the world, whatever the correct model is” was a broadly shared terminal value in humans, in the same way as the other prime suspects for terminal value (survival instinct, caring for friends and family, etc)? I would expect:
1. It’s exhibited in the vast majority of humans, with some medium correlation between intelligence and the level to which this value is exhibited. (Strongly exhibiting this value tends to cause greater effectiveness i.e. intelligence, but most people already strongly exhibit this value)
2. Companies to have jumped on this opportunity like a pack of wolves and have designed thousands of cheap wooden signs with phrases like “Family, love, ‘Understanding the world, whatever the correct model is’”.
3. Movements which oppose this value are somewhat fringe and widely condemned.
4. Most people who espouse this value are not exactly sure where it’s from, in the same way they’re not exactly sure where their survival instinct or their love for their family came from.
But, what do we see in the real world?
1. Exhibiting this value is highly correlated with intelligence. Almost everyone lightly exhibits this value, because its practical applications are pretty obvious (Pretending your mate isn’t cheating on you is just plainly a stupid strategy), but it’s only strongly and knowingly exhibited among really smart people interested in improving their instrumental capabilities.
2. Movements which oppose this value are common.
3. Most people who espouse this value got it from an intellectual tradition, some wise counseling, etc.

Dach Sep 30, 2020, 12:42 AM
1 point
in reply to: TAG’s comment on: The Short Case for Verificationism
This is only true for trivial values, e.g. “I terminally value having this specific world model”.
For most utility schemes (Including, critically, that of humans), the supermajority of the purpose of models and beliefs is instrumental. For example, making better predictions, using less computing power, etc.
In fact, humans who do not recognize this fact and stick to beliefs or models because they like them are profoundly irrational. If the sky is blue, I wish to believe the sky is blue, and so on. So, assuming that only prediction is valuable is not question begging- I suspect you already agreed with this and just didn’t realize it.
In the sense that beliefs (and the models they’re part of) are instrumental goals, any specific belief is “unnecessary”. Note the quotations around “unnecessary” in this comment and the comment you’re replying to. By “unnecessary” I mean the choice of which beliefs and which model to use is subject to the whims of which is more instrumentally valuable- in practice, a complex tradeoff between predictive accuracy and computational demands.

Dach Sep 29, 2020, 2:42 AM
1 point
on: The Short Case for Verificationism
It’s a well known tragedy that (unless Humanity gains a perspective on reality far surpassing my wildest expectations) there are arbitrarily many nontrivially unique theories which correspond to any finite set of observations.
The practical consequence of this (A small leap, but valid) is that we can remove any idea you have and make exactly the same predictions about sensory experiences by reformulating our model. Yes, any idea. Models are not even slightly unique- the idea of anything “really existing” is “unnecessary”, but literally every belief is “unnecessary”. I’d expect some beliefs would, for the practical purposes of present-day-earth human brains, be impossible to replace, but I digress.
(Joke: what’s the first step of more accurately predicting your experiences? Simplifying your experiences! Ahaha!)
You cannot “know” anything, because you’re experiencing exactly the same thing as you could possibly be experiencing if you were wrong. You can’t “know” that you’re either wrong or right, or neither, you can’t “know” that you can’t “know” anything, etc. etc. etc.
There are infinitely many different ontologies which support every single piece of information you have ever or will ever experience.
In fact, no experience indicates anything- we can build a theory of everything which explains any experience but undermines any inferences made using it, and we can do this with a one-to-one correspondence to theories that support that inference.
In fact, there’s no way to draw the inference that you’re experiencing anything. We can build infinitely many models (Or, given the limits on how much matter you can store in a Hubble volume, an arbitrarily large but finite number of models) in which the whole concept of “experience” is explained away as delusion...
And so on!
The main point of making beliefs pay rent is having a more computationally efficient model- doing things more effectively. Is your reformulation more effective than the naïve model? No.
Your model, and this whole line of thought, is not paying rent.

Dach Sep 24, 2020, 6:29 PM
1 point
in reply to: TAG’s comment on: Dach’s Shortform
Right, that isn’t an exhaustive list. I included the candidates which seemed most likely.
So, I think superintelligence is unlikely in general- but so is current civilization. I think superintelligences have a high occurrence rate given current civilization (for lots of reasons), which also means that current civilization isn’t that much more likely than superintelligence. It’s more justified to say “Superintelligences which make human minds” have a super low occurrence rate relative to natural examples of me and my environment, but that still seems to be an unlikely explanation.
Based on the “standard” discussion on this topic, I get the distinct impression that the probability our civilization will construct an aligned superintelligence is significantly greater than, for example, 10^-20%, and the large amounts of leverage that a superintelligence would have (There’s lots of matter out there) would produce this same effect.