rk

• rkJan 7, 2019, 2:51 PM

  3 points

  in reply to: Douglas_Knight’s comment on: Does anti-malaria char­ity de­stroy the lo­cal anti-malaria in­dus­try?

  Probably the most valuable nets are those deployed on people who already have malaria, to prevent it from spreading to mosquitoes, and thus to more people

  I hadn’t thought about this! I’d be interested in learning more about this. Do you have a suggested place to start reading or more search term suggestions (on top of Ewald)?

  Also, can animals harbour malaria pathogens that harm humans? This section of the wiki page on malaria makes me think not, but it’s not explicitly stated

• rkDec 21, 2018, 1:04 PM

  1 point

  in reply to: Chris_Leong’s comment on: An­thropic para­doxes trans­posed into An­thropic De­ci­sion Theory

  your decision theory maps from decisions to situations

  Could you say a little more about what a situation is? One thing I thought is maybe that a situation is a result of a choice? But then it sounds like your decision theory decides whether you should, for example, take an offered piece of chocolate, regardless of whether you like chocolate or not. So I guess that’s not it

  But the point is that each theory should be capable of standing on its own

  Can you say a little more about how ADT doesn’t stand on its own? After all, ADT is just defined as:

  An ADT agent is an agent that would implement a self-confirming linking with any agent that would do the same. It would then maximises its expected utility, conditional on that linking, and using the standard non-anthropic probabilities of the various worlds.

  Is the problem that it mentions expected utility, but it should be agnostic over values not expressible as utilities?

• rkDec 21, 2018, 11:10 AM

  1 point

  in reply to: Chris_Leong’s comment on: An­thropic para­doxes trans­posed into An­thropic De­ci­sion Theory

  So I think an account of anthropics that says “give me your values/​morality and I’ll tell you what to do” is not an account of morality + anthropics, but has actually pulled out morality from an account of anthropics that shouldn’t have had it. (Schematically, rather than define adt(decisionProblem) = chooseBest(someValues, decisionProblem), you now have define adt(values, decisionProblem) = chooseBest(values, decisionProblem))

  Perhaps you think that an account that makes mention of morality ends up being (partly) a theory of morality? And that also we should be able to understand anthropic situations apart from values?

  To try and give some intuition for my way of thinking about things, suppose I flip a fair coin and ask agent A if it came up heads. If it guesses heads and is correct, it gets $100. If it guesses tails and is correct, both agents B and C get $100. Agents B and C are not derived from A in any special way and will not be offered similar problems—there is not supposed to be anything anthropic here.

  What should agent A do? Well that depends on A’s values! This is going to be true for a non-anthropic decision theory so I don’t see why we should expect an anthropic decision theory to be free of this dependency.

  Here’s another guess at something you might think: “anthropics is about probabilities. It’s cute that you can parcel up value-laden decisions and anthropics, but it’s not about decisions.”

  Maybe that’s the right take. But even if so, ADT is useful! It says that in several anthropic situations, even if you’ve not sorted your anthropic probabilities out, you can still know what to do.

• rkDec 21, 2018, 12:07 AM

  1 point

  in reply to: Chris_Leong’s comment on: An­thropic para­doxes trans­posed into An­thropic De­ci­sion Theory

  It seems to me that ADT separates anthropics and morality. For example, Bayesianism doesn’t tell you what you should do, just how to update your beliefs. Given your beliefs, what you value decides what you should do. Similarly, ADT gives you an anthropic decision procedure. What exactly does it tell you to do? Well, that depends on your morality!

• rkDec 1, 2018, 1:35 PM

  5 points

  on: Over­con­fi­dent talk­ing down, hum­ble or hos­tile talk­ing up

  As I read through, the core model fit well with my intuition. But then I was surprised when I got to the section on religious schisms! I wondered why we should model the adherents of a religion as trying to join the school with the most ‘accurate’ claims about the religion.

  On reflection, it appears to me that the model probably holds roughly as well in the religion case as the local radio intellectual case. Both of those are examples of “hostile” talking up. I wonder if the ways in which those cases diverge from pure information sharing explains the difference between humble and hostile.

  In particular, perhaps some audiences are looking to reduce cognitive dissonance between their self-image as unbiased on the one hand and their particular beliefs and preferences on the other. That leaves an opening for someone to sell reasonableness/​unbiasedness self-image to people holding a given set of beliefs and preferences.

  Someone making reasonable counterarguments is a threat to what you’ve offered, and in that case your job is to provide refutation, counterargument and discredit so it is easy for that person’s arguments to be dismissed (through a mixture of claimed flaws in their arguments and claimed flaws in the person promoting them). This would be a ‘hostile’ talking up.

  Also, we should probably expect to find it hard to distinguish between some hostile talking ups and overconfident talking downs. If we could always distinguish, hostile talking up is a clear signal of defeat.

• rkNov 23, 2018, 5:40 PM

  3 points

  in reply to: Ben Pace’s comment on: On MIRI’s new re­search directions

  When it comes to disclosure policies, if I’m uncertain between the “MIRI view” and the “Paul Christiano” view, should I bite the bullet and back one approach over the other? Or can I aim to support both views, without worrying that they’re defeating each other?

  My current understanding is that it’s coherent to support both at once. That is, I can think that possibly intelligence needs lots of fundamental insights, and that safety needs lots of similar insights (this is supposed to be a characterisation of a MIRI-ish view). I can think that work done on figuring out more about intelligence and how to control it should only be shared cautiously, because it may accelerate the creation of AGI.

  I can also think that prosaic AGI is possible, and fundamental insights aren’t needed. Then I might think that I could do research that would help align prosaic AGIs but couldn’t possibly align (or contribute to) an agent-based AGI.

  Is the above consistent? Also do people (with better emulators of people) who worry about disclosure think that this makes sense from their point of view?

Believ­ing oth­ers’ priors

• rkNov 20, 2018, 5:37 PM

  3 points

  in reply to: Chris_Leong’s comment on: Ro­bust Delegation

  I think you’ve got a lot of the core idea. But it’s not important that we know that the data point has some ranking within a distribution. Let me try and explain the ideas as I understand them.

  The unbiased estimator is unbiased in the sense that for any actual value of the thing being estimated, the expected value of the estimation across the possible data is the true value.

  To be concrete, suppose I tell you that I will generate a true value, and then add either +1 or −1 to it with equal probability. An unbiased estimator is just to report back the value you get:

  E[estimate(x)] = estimate(x + 1)/​2 + estimate(x − 1)/​2

  If the estimate function is identity, we have (x + x +1 −1)/​2 = x. So its unbiased.

  Now suppose I tell you that I will generate the true value by drawing from a normal distribution with mean 0 and variance 1, and then I tell you 23,000 as the reported value. Via Bayes, you can see that it is more likely that the true value is 22,999 than 23,001. But the unbiased estimator blithely reports 23,000.

  So, though the asymmetry is doing some work here (the further we move above 0, the more likely that +1 rather than −1 is doing some of the work), it could still be that 23,000 is the smallest of the values I sampled.

• rkNov 18, 2018, 4:12 PM

  3 points

  in reply to: Wei Dai’s comment on: Act of Charity

  I am also pretty interested in 2 (ex-post giving). In 2015, there was impactpurchase.org. I got in contact with them about it, and the major updates Paul reported were a) being willing to buy partial contributions (not just for people who were claiming full responsibility for things) and b) more focused on what’s being funded (like for example, only asking for people to submit claims on blog posts and articles).

  I realise that things like impactpurchase is possibly framed in terms of a slightly divergent reason for 2 (it seems more focused on changing the incentive landscape, whereas the posts above include thinking about whether giving slack to people with track records will lead those people to be counterfactually more effective in future).

• rkNov 17, 2018, 2:53 PM

  2 points

  on: Pre­dic­tion-Aug­mented Eval­u­a­tion Systems

  I’m interested in the predictors’ incentives.

  One problem with decision markets is that you only get paid for your information about an option if the decision is taken, which can incentivise you to overstate the case for an option (if you see its predicted benefit X, its true benefit is X+k and it would have to be at X+k+l to be chosen, if l < k, you will want to move the predicted benefit to X+k+l and make a k-l profit).

  Maybe you avoid this if you pay for participation in PAES, but then you might risk people piling on to obvious judgments to get paid. Maybe you evaluate the counterfactual shift in confidence from someone making a judgment, and reward accordingly? But then it seems possible that the problems in the previous paragraph would appear again.

• rkNov 17, 2018, 2:43 PM

  2 points

  in reply to: ozziegooen’s comment on: Pre­dic­tion-Aug­mented Eval­u­a­tion Systems

  On estimating expected value, I’m reminded by some of Hanson’s work where he suggests predicting later evaluation (recent example: http://​​www.overcomingbias.com/​​2018/​​11/​​how-to-fund-prestige-science.html). I think this is an interesting subcase of the evaluating subprocess. It also fits nicely with this post by PC

• rkNov 16, 2018, 12:55 PM

  3 points

  on: Pre­dic­tion-Aug­mented Eval­u­a­tion Systems

  Thanks for the video! I had already skimmed this post when I noticed it, and then I watched it and reread the post. Perhaps my favourite thing about it was that it was slightly non-linear (skipping ahead to the diagram, non-linearity when covering sections).

  Could you say a bit more about your worries with (scaling) prediction markets?

  Do you have any thoughts about which experiments have the best expected information value per $?

• rkNov 13, 2018, 10:19 AM

  1 point

  in reply to: mako yass’s comment on: An­nihilat­ing aliens & Rare Earth sug­gest early filter

  This was really interesting. I’ve thought of this comment on-and-off for the last month.

  You raised an interesting reason for thinking that transhumans would have high anthropic measure. But if you have a reference-class based anthropic theory, couldn’t transhumans have a lot of anthropic measure, but not be in our reference class (that is, for SSA, we shouldn’t reason as if we were selected from a class containing all humans and transhumans)?

  Even if we think that the reference class should contain transhumans, do we have positive reasons for thinking that it should contain organisations?

  One thought is that you might reject reference classes in anthropic reasoning (even under SSA). Is that the case?

• rkNov 13, 2018, 10:15 AM

  1 point

  in reply to: habryka’s comment on: AI de­vel­op­ment in­cen­tive gra­di­ents are not com­pletely terrible

  Yes, that seems an important case to consider.

  You might still think the analysis in the post is relevant if there are actors that can shape the incentive gradients you talk about: Google might be able to focus its sub-entities in a particular way while maintaining profit or a government might choose to implement more or less oversight over tech companies.

  Even with the above paragraph, it seems like the relative change-over-time in resources and power of the strategic entities would be important to consider, as you point out. In this case, it seems like (known) fast takeoffs might be safer!

• rkNov 12, 2018, 4:32 PM

  2 points

  in reply to: rk’s comment on: AI de­vel­op­ment in­cen­tive gra­di­ents are not com­pletely terrible

  I talked to a couple of people in relevant organisations about possible info hazards for talking about races (not because this model is sophisticated or non-obvious, but because it contributes to general self-fulfilling chattering). Amongst those I talked to, they were not worried about (a) simple pieces with at least some nuance in general and (b) this post in particular

• rkNov 12, 2018, 4:29 PM

  2 points

  on: AI de­vel­op­ment in­cen­tive gra­di­ents are not com­pletely terrible

  Comment here if you have structure/​writing complaints for the post

• rkNov 12, 2018, 4:29 PM

  2 points

  on: AI de­vel­op­ment in­cen­tive gra­di­ents are not com­pletely terrible

  Comment here if you are worried about info-hazard-y-ness of talking about AI races

• rkNov 12, 2018, 4:28 PM

  2 points

  on: AI de­vel­op­ment in­cen­tive gra­di­ents are not com­pletely terrible

  Comment here if there are maths problems

AI de­vel­op­ment in­cen­tive gra­di­ents are not uniformly terrible

• rkNov 8, 2018, 9:46 PM

  5 points

  in reply to: ryan_b’s comment on: Up­date the best text­books on ev­ery sub­ject list

  Having the rules in the post made me think you wanted new suggestions in this thread. The rest of the post and habryka’s comment point towards new comments in the old thread.

  If you want people to update the old thread, I would either remove the rules from this post, or add a caveat like “Remember, when you go to post in that thread, you should follow the rules below”