Roman Leventov

• Roman Leventov 14 Oct 2024 23:58 UTC

  5 points

  0

  on: AI re­search as­sis­tants com­pe­ti­tion 2024Q3: Tie be­tween Elicit and You.com

  Undermind.ai I think is much more useful for searching concepts and ideas in papers rather than extracting tabular info a la Elicit. Nominally Elicit can do the former, too, but is quite bad in my experience.

• Roman Leventov 12 Oct 2024 14:50 UTC

  4 points

  0

  on: The Great Data In­te­gra­tion Schlep

  https://​​openmined.org/​​ develops Syft, a framework for “private computation” in secure enclaves. It potentially reduces the barriers for data integration both within particularly bureaucratic orgs and across orgs.

• Roman Leventov 12 Oct 2024 13:04 UTC

  13 points

  0

  on: My the­ory of change for work­ing in AI healthtech

  Thanks for the post, I agree with it!

  I just wrote a post with differential knowledge interconnection thesis, where I argue that it is on net beneficial to develop AI capabilities such as

  • Federated learning, privacy-preserving multi-party computation, and privacy-preserving machine learning.

  • Federated inference and belief sharing.

  • Protocols and file formats for data, belief, or claim exchange and validation.

  • Semantic knowledge mining and hybrid reasoning on (federated) knowledge graphs and multimodal data.

  • Structured or semantic search.

  • Datastore federation for retrieval-based LMs.

  • Cross-language (such as, English/​French) retrieval, search, and semantic knowledge integration. This is especially important for low-online-presence languages.

  I discuss whether knowledge interconnection exacerbates or abates the risk if industrial dehumanization on net in a section. It’s a challenging question, but I reach the tentative conclusion that AI capabilities that favor obtaining and leveraging “interconnected” rather than “isolated” knowledge are on net risk-reducing. This is because the “human economy” is more complex than the hypothetical “pure machine-industrial economy”, and “knowledge interconnection” capabilities support that greater complexity.

  Would you agree or disagree with this?

Differ­en­tial knowl­edge interconnection

• Roman Leventov 1 Jun 2024 15:47 UTC

  5 points

  0

  on: There Should Be More Align­ment-Driven Star­tups!

  I think the model of commercial R&D lab would often suit alignment work better than a “classical” startup company. Conjecture and AE Studio come to mind. Answer.AI, founded by Jeremy Howard (of Fast.ai and Kaggle) and Eric Ries (Lean Startup) elaborates on this business and organisational model here: https://​​www.answer.ai/​​posts/​​2023-12-12-launch.html.

The AI Revolu­tion in Biology

• Roman Leventov 14 May 2024 5:50 UTC

  2 points

  0

  in reply to: Roman Leventov’s comment on: The two-tiered society

  But I should add, I agree that 1-3 poses challenging political and coordination problems. Nobody assumes it will be easy, including Acemoglu. It’s just another one in the row of hard political challenges posed by AI, along with the questions of “aligned with whom?”, considering/​accounting for people’s voice past dysfunctional governments and political elites in general, etc.

• Roman Leventov 14 May 2024 5:41 UTC

  2 points

  0

  in reply to: FlorianH’s comment on: The two-tiered society

  Separately, I at least spontaneously wonder: How would one even want to go about differentiating what is the ‘bad automation’ to be discouraged, from legit automation without which no modern economy could competitively run anyway? For a random example, say if Excel wouldn’t yet exist (or, for its next update..), we’d have to say: Sorry, cannot do such software, as any given spreadsheet has the risk of removing thousands of hours of work...?! Or at least: Please, Excel, ask the human to manually confirm each cell’s calculation...?? So I don’t know how we’d in practice enforce non-automation. Just ‘it uses a large LLM’ feels weirdly arbitrary condition—though, ok, I could see how, due to a lack of alternatives, one might use something like that as an ad-hoc criterion, with all the problems it brings. But again, I think points 1. & 2. mean this is unrealistic or unsuccessful anyway.

  Clearly, specific rule-based regulation is a dumb strategy. Acemoglu’s suggestions: tax incentives to keep employment and “labour voice” to let people decide in the context of specific company and job how they want to work with AI. I like this self-governing strategy. Basically, the idea is that people will want to keep influencing things and will resist “job bullshittification” done to them, if they have the political power (“labour voice”). But they should also have alternative choice of technology and work arrangement/​method that doesn’t turn their work into rubber-stamping bullshit, but also alleviates the burden (“machine usefulness”). Because if they only have the choice between rubber-stamping bullshit job and burdensome job without AI, they may choose rubber-stamping.

• Roman Leventov 14 May 2024 5:33 UTC

  2 points

  0

  in reply to: FlorianH’s comment on: The two-tiered society

  If you’d really be able to coordinate globally to enable 1. or 2. globally—extremely unlikely in the current environment and given the huge incentives for individual countries to remain weak in enforcement—then it seems you might as well try to impose directly the economic first best solution w.r.t. robots vs. labor: high global tax rates and redistribution.

  If anything, this problem seems more pernicious wrt. climate change mitigation and environmental damage: it’s much more distributed, not only in US and China, but Russia and India are also big emitters, big leverage in Brazil, Congo, and Indonesia with their forests, overfishing and ocean pollution everywhere, etc.

  With AI, it’s basically the question of regulating US and UK companies: EU is always eager to over-regulate relative to the US, and China is already successfully and closely regulating their AI for a variety of reasons (which Acemoglu points out). The big problem of the Chinese economy is weak internal demand, and automating jobs and therefore increasing inequality and decreasing the local purchasing power is the last thing that China wants.

• Roman Leventov 13 May 2024 17:20 UTC

  2 points

  0

  in reply to: Gordon Seidoh Worley’s comment on: The two-tiered society

  What levels of automation does the AI provide and at what rate is what he suggests to influence directly (specifically, slow down), through economic and political measures. So it’s not fair to list that as an assumption.

• Roman Leventov 13 May 2024 17:09 UTC

  2 points

  0

  in reply to: Viliam’s comment on: The two-tiered society

  It would depend on exact details, but if a machine can do something as well or better than a human, then the machine should do it.

  It’s a question of how to design work. Machine can cultivate better than a human a monoculture mega-farm, but not a small permaculture garden (at least, yet). Is a monoculture mega-farm more “effective”? Maybe, if we take the pre-AI opportunity cost of human labour, but also maybe not with the post-AI opportunity cost of human labour. And this is before factoring in the “economic value” of better psychological and physical health of people who work on small farms vs. those who eat processed food on their couches that is done from the crops grown on monoculture mega-farms, and do nothing.

  As I understand, Acemoglu rougly suggests to look for ways to apply this logic in other domain of economy, including the knowledge economy. Yes, it’s not guaranteed that such arrangements will stay economical for a long time (but it’s also not beyond my imagination, especially if we factor in the economic value of physical and psychological health), but it may set the economy and the society on a different trajectory with higher chances of eventualities that we would consider “not doom”.

  What does “foster labour voice” even mean?

  Unions 2.0, or something like holacracy?

  Especially in companies where everything is automated.

  Not yet. Clearly, what he suggests could only remain effective for a limited time.

  You can give more power to current employees of current companies, but soon there will be new startups with zero employees (or where, for tax reasons, owners will formally employ their friends or family members).

  Not that soon at all, if we speak about the real economy. In IT sector, I suspect that Big Techs will win big in the AI race because only they have deep enough pockets (you already see Inflection AI quasi-acquired by MS, Stability essentially bust, etc.). And Big Techs still have huge workforces and it won’t be just Nadella or just Pichai anytime soon. Many other knowledge sectors (banks, law) are regulated and also won’t shed employees that fast.

  Human-complementary AI technologies again sounds like a bullshit job, only mostly did by a machine, where a human is involved somewhere in the loop, but the machine could still do his part better, too.

  In my gardening example, a human may wear AI goggles that tell them which plants or animal species do their see or what disease a plant has.

  Tax on media platforms—solves a completely different problem. Yes, it is important to care about public mental health. But that is separate from the problem of technological unemployment. (You could have technological unemployment even in the universe where all social media are banned.)

  Tax on media platforms is just a concrete example of how “reforming business models” could be done in practice, maybe not the best one (but it’s not my example). I will carry on with my gardening example and suggest “tax on fertiliser”: make it so huge that megafarms (which require a lot of fertiliser) become less economical than permaculture gardens. Because without such a push, permaculture gardens won’t magically materialise. Acemoglu underscores this point multiple times: it’s not a matter of pure technological invention and application of it in a laissez-faire market to switch to a different socioeconomic trajectory. Inventing AI goggles for gardening (or any other technology which makes permaculture gardening arbitrarily convenient) won’t make the economy to switch from monoculture mega-farms without an extra push.

  Perhaps, Acemoglu also has something in his mind about attention/​creator economy and the automation that may happen to them (AI influencers can replace human influencers) when he talks about “digital ad tax”, but I don’t see it.

The two-tiered society

• Roman Leventov 30 Mar 2024 13:08 UTC

  2 points

  0

  on: On attunement

  John Vervaeke calls attunement “relevance realization”.

• Roman Leventov 27 Mar 2024 3:16 UTC

  14 points

  9

  on: Modern Trans­form­ers are AGI, and Hu­man-Level

  Cf. DeepMind’s “Levels of AGI” paper (https://​​arxiv.org/​​abs/​​2311.02462), calling modern transformers “emerging AGI” there, but also defining “expert”, “virtuoso”, and “superhuman” AGI.

• Roman Leventov 24 Mar 2024 13:24 UTC

  2 points

  0

  in reply to: Mateusz Bagiński’s comment on: AI Align­ment Metastrategy

  Humane/​acc, https://​​twitter.com/​​AndrewCritchPhD

From Con­cep­tual Spaces to Quan­tum Con­cepts: For­mal­is­ing and Learn­ing Struc­tured Con­cep­tual Models

• Roman Leventov 5 Feb 2024 22:11 UTC

  2 points

  −2

  in reply to: ryan_greenblatt’s comment on: Value learn­ing in the ab­sence of ground truth

  Well, yes, it also includes learning weak agent’s models more generally, not just the “values”. But I think the point stands. It’s elaborated better in the linked post. As AIs will receive most of the same information that humans receive through always-on wearable sensors, there won’t be much to learn for AIs from humans. Rather, it’s humans that will need to do their homework, to increase the quality of their value judgements.

• Roman Leventov 5 Feb 2024 21:30 UTC

  3 points

  0

  on: Value learn­ing in the ab­sence of ground truth

  I agree with the core problem statement and most assumptions of the Pursuit of Happiness/​Conventions Approach, but suggest a different solution: https://​​www.lesswrong.com/​​posts/​​rZWNxrzuHyKK2pE65/​​ai-alignment-as-a-translation-problem

  I agree with OpenAI folks that generalisation is the key concept for understanding alignment process. But I think that with their weak-to-strong generalisation agenda, they (as well as almost everyone else) apply it I’m the reverse direction: learning values of weak agents (humans) doesn’t make sense. Rather, weak agents should learn the causal models that strong agents employ to be able to express an informed value judgement. This is the way to circumvent the “absence of the ground truth for values” problem: instead, agent try to generalise their respective world models so that they sufficiently overlap, and then choose actions that seem net beneficial to both sides, without knowing how this value judgement way made by the other side.

  In order to be able to generalise to shared world models with AIs, we must also engineer AIs to have human inductive biases from the beginning. Otherwise, this won’t be feasible. This observation makes “brain-like AGI” one of the most important alignment agendas in my view.

• Roman Leventov 5 Feb 2024 18:44 UTC

  6 points

  4

  in reply to: Algon’s comment on: AI al­ign­ment as a trans­la­tion problem

  If I understand correctly, by “discreteness” you mean that it simply says that one agent can know neither the meaning of symbols used by another agent nor the “degree” of grokking the meaning. Just cannot say anything.

  This is correct, but the underlying reason why this is correct is the same as why solipsism or the simulation hypothesis cannot be disproven (or proven!).

  So yeah, I think there is no tangible relationship to the alignment problem, except that it corroborates that we couldn’t have 100% (literally, probability=1) certainty of alignment or safety of whatever we create, but it was obvious even without this philosophical argument.

  So, I removed that paragraph about Quine’s argument from the post.

AI al­ign­ment as a trans­la­tion problem