Mild Optimization

Soft op­ti­miza­tion makes the value tar­get bigger

When to use quantilization

Satis­ficers want to be­come maximisers

Re­quire­ments for a STEM-ca­pa­ble AGI Value Learner (my Case for Less Doom)

Stable Poin­t­ers to Value III: Re­cur­sive Quantilization

Quan­tiliz­ers max­i­mize ex­pected util­ity sub­ject to a con­ser­va­tive cost constraint

Quan­tilal con­trol for finite MDPs

[Question] Why don’t quan­tiliz­ers also cut off the up­per end of the dis­tri­bu­tion?

Steam

Op­ti­miza­tion Reg­u­lariza­tion through Time Penalty

Thoughts on Quantilizers

Re­ward is not Ne­c­es­sary: How to Create a Com­po­si­tional Self-Pre­serv­ing Agent for Life-Long Learning

Val­ida­tor mod­els: A sim­ple ap­proach to de­tect­ing goodharting

Break­ing the Op­ti­mizer’s Curse, and Con­se­quences for Ex­is­ten­tial Risks and Value Learning

The Op­ti­mizer’s Curse and How to Beat It

[Aspira­tion-based de­signs] 2. For­mal frame­work, ba­sic algorithm

How to safely use an optimizer

Think­ing about max­i­miza­tion and corrigibility

Aspira­tion-based Q-Learning

AISC pro­ject: Satis­fIA – AI that satis­fies with­out over­do­ing it

[Aspira­tion-based de­signs] 1. In­for­mal in­tro­duc­tion

AISC team re­port: Soft-op­ti­miza­tion, Bayes and Goodhart

Ex­plor­ing Mild Be­havi­our in Embed­ded Agents