Roman Leventov

From Con­cep­tual Spaces to Quan­tum Con­cepts: For­mal­is­ing and Learn­ing Struc­tured Con­cep­tual Models

• Roman LeventovFeb 5, 2024, 10:11 PM

  2 points

  −2

  in reply to: ryan_greenblatt’s comment on: Value learn­ing in the ab­sence of ground truth

  Well, yes, it also includes learning weak agent’s models more generally, not just the “values”. But I think the point stands. It’s elaborated better in the linked post. As AIs will receive most of the same information that humans receive through always-on wearable sensors, there won’t be much to learn for AIs from humans. Rather, it’s humans that will need to do their homework, to increase the quality of their value judgements.

• Roman LeventovFeb 5, 2024, 9:30 PM

  3 points

  0

  on: Value learn­ing in the ab­sence of ground truth

  I agree with the core problem statement and most assumptions of the Pursuit of Happiness/​Conventions Approach, but suggest a different solution: https://​​www.lesswrong.com/​​posts/​​rZWNxrzuHyKK2pE65/​​ai-alignment-as-a-translation-problem

  I agree with OpenAI folks that generalisation is the key concept for understanding alignment process. But I think that with their weak-to-strong generalisation agenda, they (as well as almost everyone else) apply it I’m the reverse direction: learning values of weak agents (humans) doesn’t make sense. Rather, weak agents should learn the causal models that strong agents employ to be able to express an informed value judgement. This is the way to circumvent the “absence of the ground truth for values” problem: instead, agent try to generalise their respective world models so that they sufficiently overlap, and then choose actions that seem net beneficial to both sides, without knowing how this value judgement way made by the other side.

  In order to be able to generalise to shared world models with AIs, we must also engineer AIs to have human inductive biases from the beginning. Otherwise, this won’t be feasible. This observation makes “brain-like AGI” one of the most important alignment agendas in my view.

• Roman LeventovFeb 5, 2024, 6:44 PM

  6 points

  4

  in reply to: Algon’s comment on: AI al­ign­ment as a trans­la­tion problem

  If I understand correctly, by “discreteness” you mean that it simply says that one agent can know neither the meaning of symbols used by another agent nor the “degree” of grokking the meaning. Just cannot say anything.

  This is correct, but the underlying reason why this is correct is the same as why solipsism or the simulation hypothesis cannot be disproven (or proven!).

  So yeah, I think there is no tangible relationship to the alignment problem, except that it corroborates that we couldn’t have 100% (literally, probability=1) certainty of alignment or safety of whatever we create, but it was obvious even without this philosophical argument.

  So, I removed that paragraph about Quine’s argument from the post.

AI al­ign­ment as a trans­la­tion problem

• Roman LeventovJan 27, 2024, 3:55 PM

  2 points

  0

  in reply to: Dirichlet-to-Neumann’s comment on: Mak­ing ev­ery re­searcher seek grants is a bro­ken model

  That also was, naturally, the model in the Soviet Union, with orgs called “scientific research institutes”. https://​​www.jstor.org/​​stable/​​284836

• Roman LeventovJan 26, 2024, 11:24 PM

  2 points

  0

  in reply to: the gears to ascension’s comment on: Work­shop (hackathon, res­i­dence pro­gram, etc.) about for-profit AI Safety pro­jects?

  See a discussion of this point here with Marius Hobbhahn and others.

• Roman LeventovJan 26, 2024, 9:54 AM

  3 points

  0

  on: This might be the last AI Safety Camp

  This post has led me to this idea: Workshop (hackathon, residence program, etc.) about for-profit AI Safety projects?

[Question] Work­shop (hackathon, res­i­dence pro­gram, etc.) about for-profit AI Safety pro­jects?

• Roman LeventovJan 23, 2024, 8:18 PM

  3 points

  0

  in reply to: Nathan Helm-Burger’s comment on: Gaia Net­work: An Illus­trated Primer

  Collusion detection and prevention and trust modelling don’t trivially follow from the basic architecture of the system described on the level of this article. Some specific mechanisms should be implemented in the Protocol to have collusion detection and trust modelling. And we don’t have these mechanisms actually developed yet, but we think that they should be doable (though this is still a research bet, not a 100% certainty) because the Gaia Network directly embodies (or is amenable to) all six general principles for anti-collusion mechanism design (agency architecture) proposed by Eric Drexler (and these principles themselves should be further validated via formalisation and proving theorems about the collusion properties of the systems of distributed intelligence).

  Of course, there should also be (at least initially, but practically for a very long time, if not forever) “traditional” governance mechanisms of the Gaia Network, nodes, model and data ownership, etc. So, there are a lot of open questions about interfacing GN with existing codes of law, judicial and law enforcement practice, intellectual property, political and governance processes, etc. Some of these interfaces and connections with existing institutions should in practice deal with bad actors and certain types of malicious behaviour on GN.

In­sti­tu­tional eco­nomics through the lens of scale-free reg­u­la­tive de­vel­op­ment, mor­pho­gen­e­sis, and cog­ni­tive science

Gaia Net­work: An Illus­trated Primer

• Roman LeventovJan 18, 2024, 10:49 AM

  2 points

  0

  in reply to: Mitchell_Porter’s comment on: Wor­ri­some mi­s­un­der­stand­ing of the core is­sues with AI transition

  Fair, I edited the post.

Wor­ri­some mi­s­un­der­stand­ing of the core is­sues with AI transition

• Roman LeventovJan 16, 2024, 2:18 AM

  2 points

  0

  on: AI do­ing philos­o­phy = AI gen­er­at­ing hands?

  Apart from the view on philosophy as “cohesive stories that bind together and infuse meaning into scientific models”, which I discussed with you earlier and you was not very satisfied with, another interpretation of philosophy (natural phil, phil of science, phil of mathematics, and metaphil, at least) is “apex generalisation/​abstraction”. Think Bengio’s “AI scientist”, but the GM should be even deeper to first sample a plausible “philosophy of science” given all the observations about the world up to the moment, then sample plausible scientific theory given the philosophy and all observations up to the moment on a specific level or coarse-graining/​scientific abstraction (quantum, chemical, bio, physio, psycho, socio, etc.), then sample mechanistic model that describes the situation/​system of interest at hand (e.g., a morphology of a particular organism, given the laws of biology, or morphology of the particular society), given the observations of the system up to the moment, and then finally sample plausible variables values that describe the particular situation at a particular point in time given all the above.

  If this interpretation is correct, then doing philosophy well and not deluding ourselves is far off. And there is a huge risk in thinking we can do it well before we actually can.

• Roman LeventovJan 14, 2024, 11:48 AM

  2 points

  −4

  in reply to: Roko’s comment on: An even deeper atheism

  Extrapolated volition is a non-sensical concept altogether, as demonstrated in the OP. There is no extrapolated volition outside of it unfolding in real life in a specific context, which affects the trajectory of values/​volition in a specific way. And which will be this context is unknown and unknowable (maybe aliens will visit Earth tomorrow, maybe not).

• Roman LeventovJan 5, 2024, 8:50 PM

  4 points

  2

  in reply to: Chipmonk’s comment on: What tech­ni­cal top­ics could help with bound­aries/​mem­branes?

  Related, consciousness frame: where is the boundary of it? Is our brain conscious, or the whole nervous system, or the whole human, or the whole human + the entire microbiome populating them, or human + robotic prosthetic limbs, or human + web search + chat AI + personal note taking app, or the whole human group (collective consciousness), etc.

  Some computational theories of consciousness attempt to give a specific, mathematically formalised answer to this question.

• Roman LeventovJan 5, 2024, 8:45 PM

  3 points

  0

  in reply to: Chipmonk’s comment on: What tech­ni­cal top­ics could help with bound­aries/​mem­branes?

  Psychology may not be “technical enough” because an adequate mathematical science or process theory is not developed for it, yet, but it’s ultimately very important, perhaps critically important: see the last paragraph of https://​​www.lesswrong.com/​​posts/​​AKBkDNeFLZxaMqjQG/​​gaia-network-a-practical-incremental-pathway-to-open-agency. Davidad apparently thinks that it can be captured with an Infra-Bayesian model of a person/​​human.

  Also on psychology: what is the boundary of personality, where just a “role” (spouse, worker, etc) turns into multiple-personality disorder?

• Roman LeventovJan 5, 2024, 8:36 PM

  4 points

  0

  in reply to: Chipmonk’s comment on: What tech­ni­cal top­ics could help with bound­aries/​mem­branes?

  In the most recent episode of his podcast show, Jim Rutt (former president of SFI) and his guest talk about membranes a lot, the word appears 30 times on a transcript page: https://​​www.jimruttshow.com/​​cody-moser/​​

• Roman LeventovJan 5, 2024, 7:32 PM

  7 points

  1

  in reply to: Chipmonk’s comment on: What tech­ni­cal top­ics could help with bound­aries/​mem­branes?

  Related, quantum information theory:

  • The physical meaning of the Holographic Principle., https://​​chrisfieldsresearch.com/​​holo-pub-Quanta-2022.pdf, and many other related papers on Quantum Free Energy Principle at https://​​chrisfieldsresearch.com/​​