Nick_Beckstead

• Nick_BecksteadSep 2, 2013, 10:13 AM

  1 point

  in reply to: Wei Dai’s comment on: Nick Beck­stead: On the Over­whelming Im­por­tance of Shap­ing the Far Future

  OK, I”ll ask Paul or Stewart next time I see them.

  Does your proposal also violate #1 because the simplicity of an observer-situated-in-a-world is a holistic property of the the observer-situated-in-a-world rather than a local one?

• Nick_BecksteadAug 30, 2013, 12:27 PM

  4 points

  in reply to: Wei Dai’s comment on: Nick Beck­stead: On the Over­whelming Im­por­tance of Shap­ing the Far Future

  That aside, I do have an object-level comment. Nick states (in section 6.3.1) that Period Independence is incompatible with bounded utility function, but I think that’s wrong. Consider a total utilitarian who exponentially discounts each person-stage according to their distance from some chosen space-time event. Then the utility function is both bounded (assuming the undiscounted utility for each person-stage is bounded) and satisfies Period Independence.

  I agree with this. I think I was implicitly assuming some additional premises, particularly Temporal Impartiality. I believe that bounded utility + Temporal Impartiality is inconsistent with bounded utility. (Even saying this implicitly assumes other stuff, like transitive rankings, etc., though I agree that Temporal Impartiality is much more substantive.)

  Another idea for a bounded utility function satisfying Period Independence, which I previously suggested on LW and was originally motivated by multiverse-related considerations, is to discount or bound the utility assigned to each person-stage by their algorithmic probability.

  I am having a hard time parsing this. Could you explain where the following argument breaks down?

  Let A(n,X) be a world in which there are n periods of quality X.

  1. The value of what happens during a period is a function of what happens during that period, and not a function of what happens in other periods.

  2. If the above premise is true, then there exists a positive period quality X such that, for any n, A(n,X) is a possible world.

  3. Assuming Period Independence and Temporal Impartiality, as n approaches infinity, the value of A(n,X) approaches infinity.

  4. Therefore, Period Independence and Temporal Impartiality imply an unbounded utility function.

  The first premise here is something I articulate in Section 3.2, but may not be totally clear given the informal statement of Period Independence that I run with.

  Let me note that one thing about your proposal confuses me, and could potentially be related to why I don’t see which step of the above argument you deny. I primarily think of probability as a property of possible worlds, rather than individuals. Perhaps you are thinking of probability as a property of centered possible worlds? Is your proposal that the goodness of a world A with is of the form:

  g(A) = well-being of person 1 prior centered world probability of person 1 in world A + well-being of person 2 prior centered world probability of person 2 in A + …

  ? If it is, this is a proposal I have not thought about and would be interested in hearing more about its merits and why it is bounded.

• Nick_BecksteadAug 26, 2013, 9:25 AM

  2 points

  in reply to: MichaelVassar’s comment on: Com­mon sense as a prior

  Would be interested to know more about why you think this is “fantastically wrong” and what you think we should do instead. The question the post is trying to answer is, “In practical terms, how should we take account of the distribution of opinion and epistemic standards in the world?” I would like to hear your answer to this question. E.g., should we all just follow the standards that come naturally to us? Should certain people do this? Should we follow the standards of some more narrowly defined group of people? Or some more narrow set of standards still?

  I see the specific sentence you objected to as very much a detail rather than a core feature of my proposal, so it would be surprising to me if this was the reason you thought the proposal was fantastically wrong. For what it’s worth, I do think that particular sentence can be motivated by epistemology rather than conformity. It is naturally motivated by the aggregation methods I mentioned as possibilities, which I have used in other contexts for totally independent reasons. I also think it is analogous to a situation in which I have 100 algorithms returning estimates of the value of a stock and one of them says the stock is worth 100x market price and all the others say it is worth market price. I would not take straight averages here and assume the stock is worth about 2x market price, even if the algorithm giving a weird answer was generally about as good as the others.

• Nick_BecksteadAug 19, 2013, 8:25 AM

  2 points

  in reply to: Brillyant’s comment on: My daily re­flec­tion routine

  The answers over the last 6 weeks have not been very repetitive at all. I’m not sure why this is exactly, since when I was much younger and would pray daily the answers were highly repetitive. It may have something to do with greater maturity and a greater appreciation of the purpose of the activity.

• Nick_BecksteadAug 18, 2013, 6:32 PM

  2 points

  in reply to: MumpsimusLane’s comment on: My daily re­flec­tion routine

  I think of the gratitude list as things that stood out as either among the best parts of the day or as unusually good (for you personally). And mistakes go the opposite way.

My daily re­flec­tion routine

• Nick_BecksteadAug 16, 2013, 12:01 AM

  0 points

  in reply to: Eliezer Yudkowsky’s comment on: Com­mon sense as a prior

  That sounds reeeeaaally suspicious in terms of potentially post-facto assignments. (Though defeasibly so—I can totally imagine a case being made for, “Yes, this really was generally visible to the person on the street at the time without benefit of hindsight.”)

  This isn’t something I’ve looked into closely, though from looking at it for a few minutes I think it is something I would like to look into more. Anyway, on the Wikipedia page on diffusion of innovation:

  This is the second fastest category of individuals who adopt an innovation. These individuals have the highest degree of opinion leadership among the other adopter categories. Early adopters are typically younger in age, have a higher social status, have more financial lucidity, advanced education, and are more socially forward than late adopters. More discrete in adoption choices than innovators. Realize judicious choice of adoption will help them maintain central communication position (Rogers 1962 5th ed, p. 283).”

  I think this supports my claim that elite common sense is quicker to join and support new good social movements, though as I said I haven’t looked at it closely at all.

  Can you use elite common sense to generate an near-term testable prediction that would sound bold relative to my probability assignments or LW generally?

  I can’t think of anything very good, but I’ll keep it in the back of my mind. Can you think of something that would sound bold relative to my perspective?

• Nick_BecksteadAug 15, 2013, 10:22 PM

  0 points

  in reply to: Ustice’s comment on: Com­mon sense as a prior

  How would this apply to social issues do you think? It seems that this is a poor way to be on the front of social change? If this strategy was widely applied, would we ever have seen the 15th and 19th amendments to the Constitution here in the US?

  My impression is that the most trustworthy people are more likely to be at the front of good social movements than the general public, so that if people generally adopted the framework, many of the promising social movements would progress more quickly than they actually did. I am not sufficiently aware of the specific history of the 15th and 19th amendments to say more than that at this point.

  There is a general question about how the framework is related to innovation. Aren’t innovators generally going against elite common sense? I think that innovators are often overconfident about the quality of their ideas, and have significantly more confidence in their ideas than they need for their projects to be worthwhile by the standards of elite common sense. E.g., I don’t think you need to have high confidence that Facebook is going to pan out for it to be worthwhile to try to make Facebook. Elite common sense may see most attempts at innovation as unlikely to succeed, but I think it would judge many as worthwhile in cases where we’ll get to find out whether the innovation was any good or not. This might point somewhat in the direction of less innovation.

  However, I think that the most trustworthy people tend to innovate more, are more in favor of innovation than the general population, and are less risk-averse than the general population. These factors might point in favor of more innovation. It is unclear to me whether we would have more or less innovation if the framework were widely adopted, but I suspect we would have more.

  On a more personal basis, I’m polyamorous, but if I followed your framework, I would have to reject polyamory as a viable relationship model. Yes, the elite don’t have a lot of data on polyamory, but although I have researched the good and the bad, and how it can work compared to monogamy, but I don’t think that I would be able to convince the elite of my opinions.

  My impression is that elite common sense is not highly discriminating against polyamory as a relationship model. It would probably be skeptical of polyamory for the general person, but say that it might work for some people, and that it could make sense for certain interested people to try it out.

  If your opinion is that polyamory should be the norm, I agree that you wouldn’t be able to convince elite common sense of this. My personal take is that it is far from clear that polyamory should be the norm. In any event, this doesn’t seem like a great test case for taking down the framework because the idea that polyamory should be the norm does not seem like a robustly supported claim.

• Nick_BecksteadAug 13, 2013, 7:32 PM

  3 points

  in reply to: Lumifer’s comment on: Com­mon sense as a prior

  I think other people are significantly more responsive to values disagreements than Brian is, and that this suggests they are significantly more open to the possibility that their idiosyncratic personal values judgments are mistaken. You can get a sense of how unusual Brian’s perspectives are by examining his website, where his discussions of negative utilitarianism and insect suffering stand out.

• Nick_BecksteadAug 13, 2013, 7:09 PM

  3 points

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  I’m not always as unreasonable as suggested there, but I was mainly trying to point out that if I refuse to go along with certain ideas, it’s not dependent on a controversial theory of meta-ethics. It’s just that I intuitively don’t like the ideas and so reject them out of hand. Most people do this with ideas they find too unintuitive to countenance.

  Whether you want to call it a theory of meta-ethics or not, and whether it is a factual error or not, you have an unusual approach to dealing with moral questions that places an unusual amount of emphasis on Brian Tomasik’s present concerns. Maybe this is because there is something very different about you that justifies it, or maybe it is some idiosyncratic blind spot or bias of yours. I think you should put weight on both possibilities, and that this pushes in favor of more moderation in the face of values disagreements. Hope that helps articulate where I’m coming from in your language. This is hard to write and think about.

• Nick_BecksteadAug 13, 2013, 12:00 PM

  2 points

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  Here I would say, “Screw ethics and meta-ethics. All I’m saying is I want to do what I feel like doing, even if you and other elites don’t agree with it.”

  I think that there is a genuine concern that many people have when they try to ask ethical questions and discuss them with others, and that this process can lead to doing better in terms of that concern. I am speaking vaguely because, as I said earlier, I don’t think that I or others really understand what is going on. This has been an important process for many of the people I know who are trying to make a large positive impact on the world. I believe it was part of the process for you as well. When you say “I want to do what I want to do” I think it mostly just serves as a conversation-stopper, rather than something that contributes to a valuable process of reflection and exchange of ideas.

  I personally suspect your error lies in not considering the problem from perspectives other than “what does Brian Tomasik care about right now?”.

  Sure, but this is not a factual error, just an error in being a reasonable person or something. :)

  I think it is a missed opportunity to engage in a process of reflection and exchange of ideas that I don’t fully understand but seems to deliver valuable results.

• Nick_BecksteadAug 13, 2013, 11:44 AM

  3 points

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  I don’t have a lot to add to my comments on religious authorities, apart from what I said in the post and what I said in response to Luke’s Muslim theology case here.

  One thing I’d say is that many of the Christian moral teachings that are most celebrated are actually pretty good, though I’d admit that many others are not. Examples of good ones include:

  • Love your neighbor as yourself (I’d translate this as “treat others as you would like to be treated”)

  • Focus on identifying and managing your own personal weaknesses rather than criticizing others for their weaknesses

  • Prioritize helping poor and disenfranchised people

  • Don’t let your acts of charity be motivated by finding approval from others

  These are all drawn from Jesus’s Sermon on the Mount, which is arguably his most celebrated set of moral teachings.

• Nick_BecksteadAug 13, 2013, 9:29 AM

  1 point

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”

  If by short-term you mean “what happens in the next 100 years or so,” I think there is something to this idea, even for people who care primarily about very long-term considerations. I suspect it is true that the expected value of very long-run outcomes is primarily dominated by totally unforeseeable weird stuff that could happen in the distant future. But I believe that the best way deal with this challenge is to empower humanity to deal with the relatively foreseeable and unforeseeable challenges and opportunities that it will face over the next few generations. This doesn’t mean “let’s just look only at short-run well-being boosts,” but something more like “let’s broadly improve cooperation, motives, access to certain types of information, narrow and broad technological capabilities, and intelligence and rationality to deal with the problems we can’t foresee, and let’s rely on the best evidence we can to prepare for the problems we can foresee.” I say a few things about this issue here. I hope to say more about it in the future.

  An analogy would be that if you were a 5-year-old kid and you primarily cared about how successful you were later in life, you should focus on self-improvement activities (like developing good habits, gaining knowledge, and learning how to interact with other people) and health and safety issues (like getting adequate nutrition, not getting hit by cars, not poisoning yourself, not falling off of tall objects, and not eating lead-based paint). You should not try to anticipate fine-grained challenges in the labor market when you graduate from college or disputes you might have with your spouse. I realize that this analogy may not be compelling, but perhaps it illuminates my perspective.

• Nick_BecksteadAug 12, 2013, 11:52 PM

  3 points

  in reply to: RobinHanson’s comment on: Com­mon sense as a prior

  I think it is evidence that thinking about it carefully wouldn’t advance their current concerns, so they don’t bother or use the thinking/​talking for other purposes. Here are some possibilities that come to mind:

  • they might not care about the outcomes that you think are decision-relevant and associated with your claim

  • they may care about the outcomes, but your claim may not actually be decision-relevant if you were to find out the truth about the claim

  • it may not be a claim which, if thought about carefully, would contribute enough additional evidence to change your probability in the claim enough to change decisions

  • it may be that you haven’t framed your arguments in a way that suggests to people that there is a promising enough path to getting info that would become decision-relevant

  • it may be because of a signalling hypothesis that you would come up with; if you’re talking about the distant future, maybe people mostly talk about such stuff as part of a system of behavior that signals support for certain perspectives. If this is happening more in this kind of case, it may be in part because of the other considerations.

• Nick_BecksteadAug 12, 2013, 11:14 PM

  1 point

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/​or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?

  I disagree with the claim that the argument for shaping the far future is a Pascalian wager. In my opinion, there is a reasonably high, reasonably non-idiosyncratic probability that humanity will survive for a very long time, that there will be a lot of future people, and/​or that future people will have a very high quality of life. Though I have not yet defended this claim as well as I would like, I also believe that many conventionally good things people can do push toward future generations facing future challenges and opportunities better than they otherwise would, which with a high enough and conventional enough probability makes the future go better. I think that these are claims which elite common sense would be convinced of, if in possession of my evidence. If elite common sense would not be so convinced, I would consider abandoning these assumptions.

  Regarding the more purely moral claims, I suspect there are a wide variety of considerations which elite common sense would give weight to, and that very long-term considerations are one time of important consideration which would get weight according to elite common sense. It may also be, in part, a fundamental difference of values, where I am part of a not-too-small contingent of people who have distinctive concerns. However, in genuinely altruistic contexts, I think many people would give these considerations substantially more weight if they thought about the issue carefully.

  Near the beginning of my dissertation, I actually speak about the level of confidence I have in my thesis quite tentatively:

  How convinced should you be by the arguments I’m going to give? I’m defending an unconventional thesis and my support for that thesis comes from highly speculative arguments. I don’t have great confidence in my thesis, or claim that others should. But I am convinced that it could well be true, that the vast majority of thoughtful people give the claim less credence that they should, and that it is worth thinking about more carefully. I aim to make the reader justified in taking a similar attitude. (p. 3, Beckstead 2013)

  I stand by this tentative stance.

• Nick_BecksteadAug 12, 2013, 10:57 PM

  6 points

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  My current meta-ethical view says I care about factual but not necessarily moral disagreements with respect to elites. One’s choice of meta-ethics is itself a moral decision, not a factual one, so this disagreement doesn’t much concern me.

  I’m a bit flabbergasted by the confidence with which you speak about this issue. In my opinion, the history of philosophy is filled with a lot of people often smarter than you and me going around saying that their perspective is the unique one that solves everything and that other people are incoherent and so on. As far as I can tell, you are another one of these people.

  Like Luke Muehlhauser, I believe that we don’t even know what we’re asking when we ask ethical questions, and I suspect we don’t really know what we’re asking when we asking meta-ethical questions either. As far as I can tell, you’ve picked one possible candidate thing we could be asking—”what do I care about right now?”—among a broad class of possible questions, and then you are claiming that whatever you want right now is right because that’s what you’re asking.

  Of course, there are some places where I could be factually wrong in my meta-ethics, like with the logical reasoning in this comment, but I think most elites don’t think there’s something wrong with my logic, just something (ethically) wrong with my moral stance. Let me know if you disagree with this.

  I think most people would just think you had made an error somewhere and not be able to say where it was, and add that you were talking about completely murky issue that people aren’t good at thinking about.

  I personally suspect your error lies in not considering the problem from perspectives other than “what does Brian Tomasik care about right now?”.

  [Edited to reduce rhetoric.]

• Nick_BecksteadAug 12, 2013, 10:37 PM

  2 points

  in reply to: A1987dM’s comment on: Com­mon sense as a prior

  Yes, thank you for catching.

• Nick_BecksteadAug 12, 2013, 5:52 PM

  1 point

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  I agree that in principle, you don’t want some discontinuous distinction between elites and non-elites. I also agree with your points (a) - (c). Something like PageRank seems good to me, though of course I would want to be tentative about the details.

  In practice, my suspicion is that most of what’s relevant here comes from the very elite people’s thinking, so that not much is lost by just focusing on their opinions. But I hold this view pretty tentatively. I presented the ideas the way I did partly because of this hunch and partly for ease of exposition.

• Nick_BecksteadAug 12, 2013, 1:17 PM

  0 points

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  Insofar as my own actions are atypical, I intend for it to result from atypical moral beliefs rather than atypical factual beliefs. (If you can think of instances of clearly atypical factual beliefs on my part, let me know.) Of course, you could claim, elite common sense should apply also as a prior to what my own moral beliefs actually are, given the fallibility of introspection. This is true, but its importance depends on how abstractly I view my own moral values. If I ask questions about what an extrapolated Brian would think upon learning more, having more experiences, etc., then the elite prior has a lot to say on this question. But if I’m more concerned with my very immediate emotional reaction, then there’s less room for error and less that the common-sense prior has to say. The fact that my moral values are sometimes not strongly affected by common-sense moral values comes from my favoring immediate emotions rather than what (one of many possible) extrapolated Brians would feel upon having further and different life experiences. (Of course, there are many possible further life experiences I could have, which would push me in lots of random directions. This is why I’m not so gung ho about what my extrapolated selves would think on some questions.)

  As you point out, one choice point is how much idealization to introduce. At one extreme, you might introduce no idealization at all, so that whatever you presently approve of is what you’ll assume is right. On the other extreme you might have a great deal of idealization. You may assume that a better guide is what you would approve of if you knew much more, had experienced much more, were much more intelligent, made no cognitive errors in your reasoning, and had much more time to think. I lean in favor of the other extreme, as I believe most people who have considered this question do, though recognize that you want to specify your procedure in a way that leaves some core part of your values unchanged. Still, I think this is a choice that turns on many tricky cognitive steps, any of which could easily be taken in the wrong direction. So I would urge that insofar as you are making a very unusual decision at this step, you should try to very carefully understand the process that others are going through.

  ETA: I’d also caution against just straight-out assuming a particular meta-ethical perspective. This is not a case where you are an expert in the sense of someone who elite common sense would defer to, and I don’t think your specific version of anti-realism, or your philosophical perspective which says there is no real question here, are views which can command the assent of a broad coalition of trustworthy people.

• Nick_BecksteadAug 12, 2013, 1:05 PM

  1 point

  in reply to: Brian_Tomasik’s comment on: Com­mon sense as a prior

  It can be murky to infer what people believe based on actions or commitments, because this mixes two quantities: Probabilities and values. For example, the reason most elites don’t seem to take seriously efforts like shaping trajectories for strong AI is not because they think the probabilities of making a difference are astronomically small but because they don’t bite Pascalian bullets. Their utility functions are not linear. If your utility function is linear, this is a reason that your actions (if not your beliefs) will diverge from those of most elites. In any event, many elites are not even systematic or consequentialist in translating utilities times probabilities into actions.

  I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell. In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments. I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.

  I also endorse Jonah’s point about some people caring about what you care about, but for different reasons. Just as we are weird, there can be other people who are weird in different ways that make them obsessed with the things we’re obsessed with for totally different reasons. Just as some scientists are obsessed with random stuff like dung beetles, I think a lot of asteroids were tracked because there were some scientists who are really obsessed with asteroids in particular, and want to ensure that all asteroids are carefully tracked far beyond the regular value that normal people place on tracking all the asteroids. I think this can include some borderline Pascalian issues. For example, important government agencies that care about speculative threats to national security. Dick Cheney famously said, “If there’s a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response.” Similarly, there can be people that are obsessed with many issues far out of proportion with what most ordinary people care about. Looking at what “most people” care about is less robust a way to find gaps in a market than it can appear at first. (I know you don’t think it would be good to save the world, but I think the example still illustrates the point to some extent. An example more relevant to would be that some scientists might just be really interested in insects and do a lot of the research that you’d think would be valuable, even though if you had just thought “no one cares about insects so this research will never happen” you’d be wrong.)