bargo

• bargo 12 Jun 2023 19:28 UTC

  4 points

  on: In­fra-Bayesian phys­i­cal­ism: a for­mal the­ory of nat­u­ral­ized induction

  A theory of physics is mathematically quite similar to a cellular automaton. This theory will usually be incomplete, something that we can represent in infra-Bayesianism by Knightian uncertainty. So, the “cellular automaton” has underspecified time evolution.

  What evidence is there that incomplete models with Knightian uncertainty are a way to turn rough models of physics into loss functions? Can the ideas behind it be applied to regular Bayesianism?

• bargo 19 May 2023 17:55 UTC

  1 point

  0

  on: Unify­ing Bar­gain­ing No­tions (2/​2)

  bolution

  solution

• bargo 14 May 2023 8:33 UTC

  3 points

  0

  on: Log­i­cal Prob­a­bil­ity of Gold­bach’s Con­jec­ture: Prov­able Rule or Coin­ci­dence?

  https://​​arxiv.org/​​abs/​​2211.06738 is related

• bargo 2 Mar 2023 9:12 UTC

  3 points

  2

  in reply to: lsusr’s comment on: Self-Refer­ence Breaks the Orthog­o­nal­ity Thesis

  Sure, I mean that it is an implementation of what you mentioned in the third-to-last paragraph.

• bargo 1 Mar 2023 18:31 UTC

  5 points

  0

  on: Self-Refer­ence Breaks the Orthog­o­nal­ity Thesis

  Congratulations, you discovered [Active Inference]!

• bargo 28 Feb 2023 2:42 UTC

  2 points

  1

  in reply to: Harsha G.’s comment on: Cu­ri­os­ity as a Solu­tion to AGI Alignment

  Solomonoff Induction and Machine Learning. How would you formulate this in terms of a machine that can only predict future observations?

• bargo 27 Feb 2023 12:30 UTC

  1 point

  0

  on: Cu­ri­os­ity as a Solu­tion to AGI Alignment

  I want to provide feedback, but can’t see the actual definition of the objective function in either of the cases. Can you write down a sketch of how this would be implemented using existing primitives (SI, ML) so I can argue against what you’re really intending?

  Some preliminary thoughts:

  1. Curiosity (obtaining information) is an instrumental goal, so I’m not sure if making it more important will produce more or less aligned systems. How will you trade off curiosity and satisfaction of human values?

  2. It’s difficult to specify correctly—depending on what you mean by curiosity, the AI can start showing itself random unpredictable data (the Noisy TV problem for RL curiosity) or manipulating humans to give it self-affirming instructions (if the goal is to minimize prediction error). So far I haven’t seen a solution that isn’t a hack that won’t generalize to superintelligent agents.

• bargo 4 Feb 2023 19:48 UTC

  1 point

  0

  in reply to: Ericf’s comment on: What would it look like if it looked like AGI was far?

  Hmm, can you elaborate on what you mean in the last sentence?