Daniel C

• Daniel CSep 8, 2024, 9:14 PM

  3 points

  0

  in reply to: Seth Herd’s comment on: My de­com­po­si­tion of the al­ign­ment problem

  Thanks! I recall reading the steering subsystems post a while ago & it matched a lot of my thinking on the topic. The idea of using variables in the world model to determine the optimization target also seems similar to your “Goals selected from learned knowledge” approach (the targeting process is essentially a mapping from learned knowledge to goals).

  
  Another motivation for the targeting process (which might also be an advantage of GLSK) I forgot to mention is that we can allow the AI to update their goals as they update their knowledge (eg about what the current human values are), which might help us avoid value lock-in.

• Daniel CSep 7, 2024, 7:05 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  I intended ‘local’ (aka not global) to be a necessary but not sufficient condition for predictions made by smaller maps within it to be possible (cuz global predictions runs into problems of embedded agency)

  I’m mostly agnostic about what the other necessary conditions are & what the sufficient conditions are

• Daniel CSep 7, 2024, 6:25 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  Yes, there’s no upper bound for what counts as “local” (except global), but there is an upper bound for the scale at which agents’ predictions can outpace the territory (eg humans can’t predict everything in the galaxy)

  I meant upper bound in the second sense

• Daniel CSep 7, 2024, 5:59 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  Yes, by “locally outpace” I simply meant outpace at some non-global scale, there will of course be some tighter upper bound for that scale when it comes to real world agents

• Daniel CSep 7, 2024, 5:11 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  I don’t think we disagree?

  The point was exactly that although we can’t outpace the territory globally, we can still do it locally(by throwing out info we don’t care about like solar flares)

  That by itself is not that interesting. The interesting part is given that different embedded maps throw out different info & retain some info, is there any info that’s convergently retained by a wide variety of maps? (aka natural latents)

  The rest of the disagreement seems to boil down to terminology

• Daniel CSep 7, 2024, 4:37 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  The claim was that a subprogram (map) embedded within a program(territory) cannot predict the *entire execution trace of that program faster than the program itself given computational irreducibility

  “there are many ways of escaping them in principle, or even in practice (by focusing on abstract behavior of computers).”

  Yes, I think this is the same point as my point about coarse graining (outpacing the territory “locally” by throwing away some info)

• Daniel CSep 7, 2024, 3:38 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  I agree, just changed the wording of that part

  Embedded agency & computational irreducibility implies that the smaller map cannot outpace the full time evolution of the territory because it is a part of it, which may or may not be important for real world agents.

  In the case where the response time of the map does matter to some extent, embedded maps often need to coarse grain over the territory to “locally” outpace the territory

  We may think of natural latents as coarse grainings that are convergent for a wide variety of embedded maps

• Daniel CSep 7, 2024, 2:42 PM

  4 points

  2

  in reply to: Vladimir_Nesov’s comment on: Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

  Presumably there can be a piece of paper somewhere with the laws of physics & initial conditions of our universe written on it. That piece of paper can “fully capture” the entire territory in that sense.

  But no agents within our universe can compute all consequences of the territory using that piece of paper (given computational irreducibility), because that computation would be part of the time evolution of the universe itself

  I think that bit is about embedded agency

Jonothan Go­rard:The ter­ri­tory is iso­mor­phic to an equiv­alence class of its maps

What pro­gram struc­tures en­able effi­cient in­duc­tion?

My de­com­po­si­tion of the al­ign­ment problem

• Daniel C 21 Dec 2021 12:39 UTC

  1 point

  in reply to: JBlack’s comment on: Ma­nipu­la­tion re­sis­tance of futarchy

  Would there be a problem when speculators can create stocks in the conditional case? As in if a decision C harms me, can i create and sell loads and loads of C stock, and not having to actually go through the trade when C is not enforced (due to the low price i’ve caused)?