Nick_Beckstead

Karma: 798

Nick_Beckstead Aug 13, 2013, 7:32 PM
3 points
in reply to: Lumifer’s comment on: Common sense as a prior
I think other people are significantly more responsive to values disagreements than Brian is, and that this suggests they are significantly more open to the possibility that their idiosyncratic personal values judgments are mistaken. You can get a sense of how unusual Brian’s perspectives are by examining his website, where his discussions of negative utilitarianism and insect suffering stand out.

Nick_Beckstead Aug 13, 2013, 7:09 PM
3 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

I’m not always as unreasonable as suggested there, but I was mainly trying to point out that if I refuse to go along with certain ideas, it’s not dependent on a controversial theory of meta-ethics. It’s just that I intuitively don’t like the ideas and so reject them out of hand. Most people do this with ideas they find too unintuitive to countenance.

Whether you want to call it a theory of meta-ethics or not, and whether it is a factual error or not, you have an unusual approach to dealing with moral questions that places an unusual amount of emphasis on Brian Tomasik’s present concerns. Maybe this is because there is something very different about you that justifies it, or maybe it is some idiosyncratic blind spot or bias of yours. I think you should put weight on both possibilities, and that this pushes in favor of more moderation in the face of values disagreements. Hope that helps articulate where I’m coming from in your language. This is hard to write and think about.

Nick_Beckstead Aug 13, 2013, 12:00 PM
2 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

Here I would say, “Screw ethics and meta-ethics. All I’m saying is I want to do what I feel like doing, even if you and other elites don’t agree with it.”

I think that there is a genuine concern that many people have when they try to ask ethical questions and discuss them with others, and that this process can lead to doing better in terms of that concern. I am speaking vaguely because, as I said earlier, I don’t think that I or others really understand what is going on. This has been an important process for many of the people I know who are trying to make a large positive impact on the world. I believe it was part of the process for you as well. When you say “I want to do what I want to do” I think it mostly just serves as a conversation-stopper, rather than something that contributes to a valuable process of reflection and exchange of ideas.

I personally suspect your error lies in not considering the problem from perspectives other than “what does Brian Tomasik care about right now?”.

Sure, but this is not a factual error, just an error in being a reasonable person or something. :)

I think it is a missed opportunity to engage in a process of reflection and exchange of ideas that I don’t fully understand but seems to deliver valuable results.

Nick_Beckstead Aug 13, 2013, 11:44 AM
3 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior
I don’t have a lot to add to my comments on religious authorities, apart from what I said in the post and what I said in response to Luke’s Muslim theology case here.

One thing I’d say is that many of the Christian moral teachings that are most celebrated are actually pretty good, though I’d admit that many others are not. Examples of good ones include:
- Love your neighbor as yourself (I’d translate this as “treat others as you would like to be treated”)
- Focus on identifying and managing your own personal weaknesses rather than criticizing others for their weaknesses
- Prioritize helping poor and disenfranchised people
- Don’t let your acts of charity be motivated by finding approval from others
These are all drawn from Jesus’s Sermon on the Mount, which is arguably his most celebrated set of moral teachings.

Nick_Beckstead Aug 13, 2013, 9:29 AM
1 point
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

It’s very common for people to say, “Predictions are hard, especially about the future, so let’s focus on the short term where we can be more confident we’re at least making a small positive impact.”

If by short-term you mean “what happens in the next 100 years or so,” I think there is something to this idea, even for people who care primarily about very long-term considerations. I suspect it is true that the expected value of very long-run outcomes is primarily dominated by totally unforeseeable weird stuff that could happen in the distant future. But I believe that the best way deal with this challenge is to empower humanity to deal with the relatively foreseeable and unforeseeable challenges and opportunities that it will face over the next few generations. This doesn’t mean “let’s just look only at short-run well-being boosts,” but something more like “let’s broadly improve cooperation, motives, access to certain types of information, narrow and broad technological capabilities, and intelligence and rationality to deal with the problems we can’t foresee, and let’s rely on the best evidence we can to prepare for the problems we can foresee.” I say a few things about this issue here. I hope to say more about it in the future.

An analogy would be that if you were a 5-year-old kid and you primarily cared about how successful you were later in life, you should focus on self-improvement activities (like developing good habits, gaining knowledge, and learning how to interact with other people) and health and safety issues (like getting adequate nutrition, not getting hit by cars, not poisoning yourself, not falling off of tall objects, and not eating lead-based paint). You should not try to anticipate fine-grained challenges in the labor market when you graduate from college or disputes you might have with your spouse. I realize that this analogy may not be compelling, but perhaps it illuminates my perspective.

Nick_Beckstead Aug 12, 2013, 11:52 PM
3 points
in reply to: RobinHanson’s comment on: Common sense as a prior
I think it is evidence that thinking about it carefully wouldn’t advance their current concerns, so they don’t bother or use the thinking/talking for other purposes. Here are some possibilities that come to mind:
- they might not care about the outcomes that you think are decision-relevant and associated with your claim
- they may care about the outcomes, but your claim may not actually be decision-relevant if you were to find out the truth about the claim
- it may not be a claim which, if thought about carefully, would contribute enough additional evidence to change your probability in the claim enough to change decisions
- it may be that you haven’t framed your arguments in a way that suggests to people that there is a promising enough path to getting info that would become decision-relevant
- it may be because of a signalling hypothesis that you would come up with; if you’re talking about the distant future, maybe people mostly talk about such stuff as part of a system of behavior that signals support for certain perspectives. If this is happening more in this kind of case, it may be in part because of the other considerations.

Nick_Beckstead Aug 12, 2013, 11:14 PM
1 point
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

However, there are some Pascalian wagers that seem genuinely compelling even after looking for alternatives, like “the Overwhelming Importance of Shaping the Far Future.” My impression is that most elites do not agree that the far future is overwhelmingly important even after hearing your arguments because they don’t have linear utility functions and/or don’t like Pascalian wagers. Do you think most elites would agree with you about shaping the far future?

I disagree with the claim that the argument for shaping the far future is a Pascalian wager. In my opinion, there is a reasonably high, reasonably non-idiosyncratic probability that humanity will survive for a very long time, that there will be a lot of future people, and/or that future people will have a very high quality of life. Though I have not yet defended this claim as well as I would like, I also believe that many conventionally good things people can do push toward future generations facing future challenges and opportunities better than they otherwise would, which with a high enough and conventional enough probability makes the future go better. I think that these are claims which elite common sense would be convinced of, if in possession of my evidence. If elite common sense would not be so convinced, I would consider abandoning these assumptions.

Regarding the more purely moral claims, I suspect there are a wide variety of considerations which elite common sense would give weight to, and that very long-term considerations are one time of important consideration which would get weight according to elite common sense. It may also be, in part, a fundamental difference of values, where I am part of a not-too-small contingent of people who have distinctive concerns. However, in genuinely altruistic contexts, I think many people would give these considerations substantially more weight if they thought about the issue carefully.

Near the beginning of my dissertation, I actually speak about the level of confidence I have in my thesis quite tentatively:

How convinced should you be by the arguments I’m going to give? I’m defending an unconventional thesis and my support for that thesis comes from highly speculative arguments. I don’t have great confidence in my thesis, or claim that others should. But I am convinced that it could well be true, that the vast majority of thoughtful people give the claim less credence that they should, and that it is worth thinking about more carefully. I aim to make the reader justified in taking a similar attitude. (p. 3, Beckstead 2013)

I stand by this tentative stance.

Nick_Beckstead Aug 12, 2013, 10:57 PM
6 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

My current meta-ethical view says I care about factual but not necessarily moral disagreements with respect to elites. One’s choice of meta-ethics is itself a moral decision, not a factual one, so this disagreement doesn’t much concern me.

I’m a bit flabbergasted by the confidence with which you speak about this issue. In my opinion, the history of philosophy is filled with a lot of people often smarter than you and me going around saying that their perspective is the unique one that solves everything and that other people are incoherent and so on. As far as I can tell, you are another one of these people.

Like Luke Muehlhauser, I believe that we don’t even know what we’re asking when we ask ethical questions, and I suspect we don’t really know what we’re asking when we asking meta-ethical questions either. As far as I can tell, you’ve picked one possible candidate thing we could be asking—”what do I care about right now?”—among a broad class of possible questions, and then you are claiming that whatever you want right now is right because that’s what you’re asking.

Of course, there are some places where I could be factually wrong in my meta-ethics, like with the logical reasoning in this comment, but I think most elites don’t think there’s something wrong with my logic, just something (ethically) wrong with my moral stance. Let me know if you disagree with this.

I think most people would just think you had made an error somewhere and not be able to say where it was, and add that you were talking about completely murky issue that people aren’t good at thinking about.

I personally suspect your error lies in not considering the problem from perspectives other than “what does Brian Tomasik care about right now?”.

[Edited to reduce rhetoric.]

Nick_Beckstead Aug 12, 2013, 10:37 PM
2 points
in reply to: A1987dM’s comment on: Common sense as a prior
Yes, thank you for catching.

Nick_Beckstead Aug 12, 2013, 5:52 PM
1 point
in reply to: Brian_Tomasik’s comment on: Common sense as a prior
I agree that in principle, you don’t want some discontinuous distinction between elites and non-elites. I also agree with your points (a) - (c). Something like PageRank seems good to me, though of course I would want to be tentative about the details.

In practice, my suspicion is that most of what’s relevant here comes from the very elite people’s thinking, so that not much is lost by just focusing on their opinions. But I hold this view pretty tentatively. I presented the ideas the way I did partly because of this hunch and partly for ease of exposition.

Nick_Beckstead Aug 12, 2013, 1:17 PM
0 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

Insofar as my own actions are atypical, I intend for it to result from atypical moral beliefs rather than atypical factual beliefs. (If you can think of instances of clearly atypical factual beliefs on my part, let me know.) Of course, you could claim, elite common sense should apply also as a prior to what my own moral beliefs actually are, given the fallibility of introspection. This is true, but its importance depends on how abstractly I view my own moral values. If I ask questions about what an extrapolated Brian would think upon learning more, having more experiences, etc., then the elite prior has a lot to say on this question. But if I’m more concerned with my very immediate emotional reaction, then there’s less room for error and less that the common-sense prior has to say. The fact that my moral values are sometimes not strongly affected by common-sense moral values comes from my favoring immediate emotions rather than what (one of many possible) extrapolated Brians would feel upon having further and different life experiences. (Of course, there are many possible further life experiences I could have, which would push me in lots of random directions. This is why I’m not so gung ho about what my extrapolated selves would think on some questions.)

As you point out, one choice point is how much idealization to introduce. At one extreme, you might introduce no idealization at all, so that whatever you presently approve of is what you’ll assume is right. On the other extreme you might have a great deal of idealization. You may assume that a better guide is what you would approve of if you knew much more, had experienced much more, were much more intelligent, made no cognitive errors in your reasoning, and had much more time to think. I lean in favor of the other extreme, as I believe most people who have considered this question do, though recognize that you want to specify your procedure in a way that leaves some core part of your values unchanged. Still, I think this is a choice that turns on many tricky cognitive steps, any of which could easily be taken in the wrong direction. So I would urge that insofar as you are making a very unusual decision at this step, you should try to very carefully understand the process that others are going through.

ETA: I’d also caution against just straight-out assuming a particular meta-ethical perspective. This is not a case where you are an expert in the sense of someone who elite common sense would defer to, and I don’t think your specific version of anti-realism, or your philosophical perspective which says there is no real question here, are views which can command the assent of a broad coalition of trustworthy people.

Nick_Beckstead Aug 12, 2013, 1:05 PM
1 point
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

It can be murky to infer what people believe based on actions or commitments, because this mixes two quantities: Probabilities and values. For example, the reason most elites don’t seem to take seriously efforts like shaping trajectories for strong AI is not because they think the probabilities of making a difference are astronomically small but because they don’t bite Pascalian bullets. Their utility functions are not linear. If your utility function is linear, this is a reason that your actions (if not your beliefs) will diverge from those of most elites. In any event, many elites are not even systematic or consequentialist in translating utilities times probabilities into actions.

I don’t endorse biting Pascalian bullets, in part for reasons argued in this post, which I think give further support to some considerations identified by GiveWell. In Pascalian cases, we have claims that people in general aren’t good at thinking about and which people generally assign low weight when they are acquainted with the arguments. I believe that Pascalian estimates of expected value that differ greatly from elite common sense and aren’t persuasive to elite common sense should be treated with great caution.

I also endorse Jonah’s point about some people caring about what you care about, but for different reasons. Just as we are weird, there can be other people who are weird in different ways that make them obsessed with the things we’re obsessed with for totally different reasons. Just as some scientists are obsessed with random stuff like dung beetles, I think a lot of asteroids were tracked because there were some scientists who are really obsessed with asteroids in particular, and want to ensure that all asteroids are carefully tracked far beyond the regular value that normal people place on tracking all the asteroids. I think this can include some borderline Pascalian issues. For example, important government agencies that care about speculative threats to national security. Dick Cheney famously said, “If there’s a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response.” Similarly, there can be people that are obsessed with many issues far out of proportion with what most ordinary people care about. Looking at what “most people” care about is less robust a way to find gaps in a market than it can appear at first. (I know you don’t think it would be good to save the world, but I think the example still illustrates the point to some extent. An example more relevant to would be that some scientists might just be really interested in insects and do a lot of the research that you’d think would be valuable, even though if you had just thought “no one cares about insects so this research will never happen” you’d be wrong.)

Nick_Beckstead Aug 12, 2013, 12:32 PM
4 points
in reply to: Brian_Tomasik’s comment on: Common sense as a prior

I think the focus on only intellectual elites has unclear grounding. Is the reason because elites think most seriously about the questions that you care about most? On a question of which kind of truck was most suitable for garbage collection, you would defer to a different class of people. In such a case, I guess you would regard them as the (question-dependent) “elites.”

This is a question which it seems I wasn’t sufficiently clear about. I count someone as an “expert on X” roughly when they are someone that a broad coalition of trustworthy people would defer to on questions about X. As I explained in another comment, if you don’t know about what the experts on X think, I recommend trying to find out what the experts think (if it’s easy/important enough) and going with what the broad coalition of trustworthy people thinks until then. So it may be that some non-elite garbage guys are experts on garbage collection, and a broad coalition of trustworthy people would defer to them on questions of garbage collection, once the broad coalition of trustworthy people knows about what these people think about garbage collection.

Why focus on people who are regarded as most trustworthy by many people? I think those people are likely to be more trustworthy than ordinary people, as I tried to suggest in my quick Quora experiment.

Nick_Beckstead Aug 12, 2013, 11:38 AM
2 points
in reply to: Eliezer Yudkowsky’s comment on: Common sense as a prior
Great. It sounds like may reasonably be on the same page at this point.

To reiterate and clarify, you can pretty much make the standards as high as you like as long as: (1) you have a good enough grip on how the elite class thinks,(2) you are using clear indicators of trustworthiness that many people would accept, and (3) you make a good-faith effort not to cherry pick and watch out for the No True Scotsman fallacy. The only major limitation on this I can think of is that there is some trade-off to be made between certain levels of diversity and independent judgment. Like, if you could somehow pick the 10 best people in the world by some totally broad standards that everyone would accept (I think this is deeply impossible), that probably wouldn’t be as good as picking the best 100-10,000 people by such standards. And I’d substitute some less trustworthy people for more trustworthy people in some cases where it would increase diversity of perspectives.

Nick_Beckstead Aug 12, 2013, 11:23 AM
0 points
in reply to: tog’s comment on: Common sense as a prior
Yes.

Nick_Beckstead Aug 12, 2013, 11:13 AM
1 point
in reply to: Eliezer Yudkowsky’s comment on: Common sense as a prior
I think we don’t disagree about whether elite common sense should defer to cryptography experts (I assume this is what Bruce Schneier is a stand-in for). Simplifying a bit, we are disagreeing about the much more subtle question of whether, given that elite common sense should defer to cryptography experts, in a situation where the current views of cryptographers are unknown, elite common sense recommends adopting the current views of cryptographers. I say elite common sense recommends adopting their views if you know them, but going with what e.g. the upper crust of Ivy League graduates would say if they had access to your information if you don’t know about the opinions of cryptographers. I also suspect elite common sense recommends finding out about the opinions of elite cryptographers if you can. But Wei Dai’s example was one in which you didn’t know and maybe couldn’t find out, so that’s why I said what I said. Frankly, I’m pretty flummoxed about why you think this is the “No True Scotsman” fallacy. I feel that one of us is probably misunderstanding the other on a basic level.

A possible confusion here is that I doubt the cryptographers have very different epistemic standards as opposed to substantive knowledge and experience about cryptography and tools for thinking about it.

I certainly don’t get the impression that one can grind well-specified rules to get to the answer about polling the upper 10% of Ivy League graduates in this case.

I agree with this, and tried to make this clear in my discussion. I went with a rough guess that would work for a decent chunk of the audience rather than only saying something very abstract. It’s subtle, but I think reasonable epistemic frameworks are subtle if you want them to have much generality.
What links here?
- Nick_Beckstead's comment on Common sense as a prior by Nick_Beckstead (Aug 12, 2013, 12:32 PM; 4 points)

Nick_Beckstead Aug 12, 2013, 10:58 AM
8 points
in reply to: itaibn0’s comment on: Common sense as a prior
Perhaps I should have meant loop quantum gravity. I confess that I am speaking beyond my depth, and was just trying to give an example of a central dispute in current theoretical physics. That is the type of case where I would not like to lean heavily on my own perspective.

Nick_Beckstead Aug 11, 2013, 10:16 PM
3 points
in reply to: Eliezer Yudkowsky’s comment on: Common sense as a prior

Suppose I selected from among all physicists who accept MWI and asked them what they thought about FAI arguments. To me that’s just an obvious sort of reweighting you might try, though anyone who’s had experience with machine learning knows that most clever reweightings you try don’t work. To someone else it might be cherry-picking of gullible physicists, and say, “You have violated Beckstead’s rules!”

Just to be clear: I would count this as violating my rules because you haven’t used a clear indicator of trustworthiness that many people would accept.

ETA: I’d add that people should generally pick their indicators in advance and stick with them, and not add them in to tune the system to their desired bottom lines.

Nick_Beckstead Aug 11, 2013, 10:12 PM
3 points
in reply to: Eliezer Yudkowsky’s comment on: Common sense as a prior
Could you maybe just tell me what you think my framework is supposed to imply about Wei Dai’s case, if not what I said it implies? To be clear: I say it implies that the executives should have used an impartial combination of the epistemic standards used by the upper crust of Ivy League graduates, and that this gives little weight to the cryptographers because, though the cryptographers are included, they are a relatively small portion of all people included. So I think my framework straightforwardly doesn’t say that people should be relying on info they can’t use, which is how I understood Wei Dai’s objection. (I think that if they were able to know what the cryptographers opinions are, then elite common sense would recommend deferring to the cryptographers, but I’m just guessing about that.) What is it you think my framework implies—with no funny business and no instance of the fallacy you think I’m committing—and why do you find it objectionable?

ETA:

I’d be happy with advice along the lines of, “First take your best guess as to who the elites really are and how much they ought to be trusted in this case, then take their opinion as a prior with an appropriate degree of concentrated probability density, then update.”

This is what I think I am doing and am intending to do.

Nick_Beckstead Aug 11, 2013, 9:17 PM
4 points
in reply to: Eliezer Yudkowsky’s comment on: Common sense as a prior
It seems the “No True Elite” fallacy would involve:

(1) Elite common sense seeming to say that I should believe X because on my definition of “elites,” elites generally believe X. (2) X being an embarrassing thing to believe (3) Me replying that someone who believed X wouldn’t count as an “elite,” but doing so in a way that couldn’t be justified by my framework

In this example I am actually saying we should defer to the cryptographers if we know their opinions, but that they don’t get to count as part of elite common sense immediately because their opinions are too hard to access. And I’m actually saying that elite common sense supports a claim which it is embarrassing to believe.

So I don’t understand how this is supposed to be an instance of the “No True Scotsman” fallacy.