yrimon

[Question] Why might Gen­eral In­tel­li­gences have long term goals?

• yrimonJul 23, 2023, 9:36 AM

  5 points

  0

  in reply to: Radford Neal’s comment on: Ex­am­ples of Prompts that Make GPT-4 Out­put Falsehoods

  I wrote the following in custom instructions, under “How would you like ChatGPT to respond?”:

  
  After every answer, give a short reflection, like “I looked at this answer again and it looks good” or “I thought about this again and I think it could be better if …” or “After looking this over again, this makes no sense because...”.
  
  I then had the following conversation:
  
  Q: What is denser, gold or uranium?

  A: Uranium is denser than gold. The density of gold is about 19.32 grams per cubic centimeter, while the density of uranium is about 19.1 grams per cubic centimeter.

  Upon reflection, I see that I made an error in my initial response. The correct densities are indeed 19.32 g/​cm³ for gold and 19.1 g/​cm³ for uranium, but these values indicate that gold is denser than uranium, not the other way around. I apologize for the confusion, and I appreciate your patience.

• yrimonJun 21, 2023, 3:38 PM

  4 points

  0

  on: Causal­ity: A Brief Introduction

  I feel like there is a major explaining paragraph missing here, explaining the difference between causality and probability. Something like:

  Armed with knowledge of the future, we could know exactly what will happen. (e.g. The other doctor will give Alice medicine, and she will get better.) Given a full probability distribution over events we could make optimal predictions. (e.g. There is a ²⁄₃ chance the other doctor will give Alice medicine, ¹⁄₄ chance of her getting better if he doesn’t and ²⁄₃ chance of her getting better if he does.) Causality gives us a way to combine a partial probability distribution with additional knowledge of the world to make predictions about events that are out of distribution. (e.g. Since I understand that the medicine works mostly by placebo, I can intervene and give Alice a placebo when the other doctor doesn’t give her the medicine, raising her chances. Furthermore, if I have a distribution of how effective a placebo is relative to the medicine, I can quantify how helpful my intervention is.)

  
  An intervention is a really important example of an out of distribution generalization; but if I gave you the full probability distribution of the outcomes that your interventions would achieve it would no longer be out of distribution (and you’d need to deal with paradoxes involving seemingly not having chocies about certain things).

[Question] Im­pact of ” ‘Let’s think step by step’ is all you need”?

• yrimonJul 24, 2022, 2:31 PM

  2 points

  0

  on: What’s next for in­stru­men­tal ra­tio­nal­ity?

  No Free Lunch means that optimization requires taking advantage of underlying structure in the set of possible environments. In the case epistemics, we all share close-to-the-same-environment (including having similar minds), so there are a lot of universally-useful optimizations for learning about the environment.

  Optimizations over the space of “how-to-behave instructions” requires some similar underlying structure. Such structure can emerge for two reasons: (1) because of the shared environment, or (2) because of shared goals. (Yeah, I’m thinking about agents as cartesian, in the sense of separating the goals and the environment, but to be fair so do L+P+S+C.)

  On the environment side, this leads to convergent behaviours (which can also be thought of as behaviours resulting from selection theorems), like good epistemics, or gaining power over resources.

  When it comes to goals, on the other hand, it is both possible (by the orthogonality thesis) and the case that different peole have vastly different goals (e.g. some people want to live forever, some want to commit suicide, and these two groups probably require mostly different strategies). Less in common between different people’s goals means less universally-useful how-to-behave instructions. Nonetheless, optimizing behaviours that are commonly prioritized is close enough to universally useful, e.g. doing relationships well.

  Perhaps an “Instrumental Sequences” would include the above categories as major chapters. In such a case, as indicated in the post, current reseaerch being posted on Lesswrong gives an approximate idea of what such sequences could look like.