AI-Assisted Alignment

TagLast edit: Jan 25, 2024, 5:18 AM by habryka

AI-Assisted Alignment is a cluster of alignment plans that involve AI somehow significantly helping with alignment research. This can include weak tool AI, or more advanced AGI doing original research.

There has been a lot of debate about how practical this alignment approach is.

Other search terms for this tag: AI aligning AI

A “Bitter Lesson” Approach to Aligning AGI and ASI

RogerDearnaleyJul 6, 2024, 1:23 AM

60 points

39 comments24 min readLW link

Requirements for a Basin of Attraction to Alignment

RogerDearnaleyFeb 14, 2024, 7:10 AM

41 points

12 comments31 min readLW link

[Link] Why I’m optimistic about OpenAI’s alignment approach

janleikeDec 5, 2022, 10:51 PM

98 points

15 comments1 min readLW link

(aligned.substack.com)

We have to Upgrade

Jed McCalebMar 23, 2023, 5:53 PM

129 points

35 comments2 min readLW link

Proposed Alignment Technique: OSNR (Output Sanitization via Noising and Reconstruction) for Safer Usage of Potentially Misaligned AGI

sudoMay 29, 2023, 1:35 AM

14 points

9 comments6 min readLW link

Beliefs and Disagreements about Automating Alignment Research

Ian McKenzieAug 24, 2022, 6:37 PM

107 points

4 comments7 min readLW link

[Link] A minimal viable product for alignment

janleikeApr 6, 2022, 3:38 PM

53 points

38 comments1 min readLW link

Cyborgism

NicholasKees and janus

Feb 10, 2023, 2:47 PM

341 points

46 comments35 min readLW link 2 reviews

How to Control an LLM’s Behavior (why my P(DOOM) went down)

RogerDearnaleyNov 28, 2023, 7:56 PM

64 points

30 comments11 min readLW link

Infinite Possibility Space and the Shutdown Problem

magfrumpOct 18, 2022, 5:37 AM

9 points

0 comments2 min readLW link

(www.magfrump.net)

Introducing AlignmentSearch: An AI Alignment-Informed Conversional Agent

BionicD0LPH1N, Fraser and TheBayesian

Apr 1, 2023, 4:39 PM

79 points

14 comments4 min readLW link

Discussion on utilizing AI for alignment

eliflandAug 23, 2022, 2:36 AM

16 points

3 comments1 min readLW link

(www.foxy-scout.com)

Anti-Slop Interventions?

abramdemskiFeb 4, 2025, 7:50 PM

76 points

33 comments6 min readLW link

Discussion with Nate Soares on a key alignment difficulty

HoldenKarnofskyMar 13, 2023, 9:20 PM

265 points

43 comments22 min readLW link 1 review

Sufficiently many Godzillas as an alignment strategy

142857Aug 28, 2022, 12:08 AM

8 points

3 comments1 min readLW link

AI-assisted list of ten concrete alignment things to do right now

lemonhopeSep 7, 2022, 8:38 AM

8 points

5 comments4 min readLW link

Capabilities and alignment of LLM cognitive architectures

Seth HerdApr 18, 2023, 4:29 PM

88 points

18 comments20 min readLW link

Instruction-following AGI is easier and more likely than value aligned AGI

Seth HerdMay 15, 2024, 7:38 PM

79 points

28 comments12 min readLW link

Getting from an unaligned AGI to an aligned AGI?

Tor Økland BarstadJun 21, 2022, 12:36 PM

13 points

7 comments9 min readLW link

Making it harder for an AGI to “trick” us, with STVs

Tor Økland BarstadJul 9, 2022, 2:42 PM

15 points

5 comments22 min readLW link

Alignment with argument-networks and assessment-predictions

Tor Økland BarstadDec 13, 2022, 2:17 AM

10 points

5 comments45 min readLW link

Internal independent review for language model agent alignment

Seth HerdJul 7, 2023, 6:54 AM

55 points

30 comments11 min readLW link

Some Thoughts on AI Alignment: Using AI to Control AI

eigenvalueJun 21, 2024, 5:44 PM

1 point

1 comment1 min readLW link

(github.com)

A survey of tool use and workflows in alignment research

Logan Riggs, Jan, janus and jacquesthibs

Mar 23, 2022, 11:44 PM

45 points

4 comments1 min readLW link

My thoughts on OpenAI’s alignment plan

Orpheus16Dec 30, 2022, 7:33 PM

55 points

3 comments20 min readLW link

[Linkpost] Jan Leike on three kinds of alignment taxes

Orpheus16Jan 6, 2023, 11:57 PM

27 points

2 comments3 min readLW link

(aligned.substack.com)

[Question] What specific thing would you do with AI Alignment Research Assistant GPT?

quetzal_rainbowJan 8, 2023, 7:24 PM

47 points

9 comments1 min readLW link

Eli Lifland on Navigating the AI Alignment Landscape

ozziegooenFeb 1, 2023, 9:17 PM

9 points

1 comment31 min readLW link

(quri.substack.com)

How might we safely pass the buck to AI?

joshcFeb 19, 2025, 5:48 PM

83 points

58 comments31 min readLW link

Cyborg Periods: There will be multiple AI transitions

Jan_Kulveit and rosehadshar

Feb 22, 2023, 4:09 PM

108 points

9 comments6 min readLW link

Maintaining Alignment during RSI as a Feedback Control Problem

berenMar 2, 2025, 12:21 AM

66 points

6 comments11 min readLW link

Intent alignment as a stepping-stone to value alignment

Seth HerdNov 5, 2024, 8:43 PM

37 points

7 comments3 min readLW link

The prospect of accelerated AI safety progress, including philosophical progress

Mitchell_PorterMar 13, 2025, 10:52 AM

11 points

0 comments4 min readLW link

AI for Resolving Forecasting Questions: An Early Exploration

ozziegooenJan 16, 2025, 9:41 PM

10 points

2 comments1 min readLW link

Misaligned AGI Death Match

Nate Reinar WindwoodMay 14, 2023, 6:00 PM

1 point

0 comments1 min readLW link

AI for AI safety

Joe CarlsmithMar 14, 2025, 3:00 PM

78 points

13 comments17 min readLW link

(joecarlsmith.substack.com)

AI Tools for Existential Security

Lizka and owencb

Mar 14, 2025, 6:38 PM

22 points

4 comments11 min readLW link

(www.forethought.org)

Some thoughts on automating alignment research

Lukas FinnvedenMay 26, 2023, 1:50 AM

30 points

4 comments6 min readLW link

Davidad’s Bold Plan for Alignment: An In-Depth Explanation

Charbel-Raphaël and Gabin

Apr 19, 2023, 4:09 PM

168 points

40 comments21 min readLW link 2 reviews

[Linkpost] Introducing Superalignment

berenJul 5, 2023, 6:23 PM

175 points

69 comments1 min readLW link

(openai.com)

Deep sparse autoencoders yield interpretable features too

Armaan A. AbrahamFeb 23, 2025, 5:46 AM

29 points

8 comments8 min readLW link

Agentized LLMs will change the alignment landscape

Seth HerdApr 9, 2023, 2:29 AM

160 points

102 comments3 min readLW link 1 review

1. A Sense of Fairness: Deconfusing Ethics

RogerDearnaleyNov 17, 2023, 8:55 PM

16 points

8 comments15 min readLW link

AISC project: How promising is automating alignment research? (literature review)

Bogdan Ionut CirsteaNov 28, 2023, 2:47 PM

4 points

1 comment1 min readLW link

(docs.google.com)

Prospects for Alignment Automation: Interpretability Case Study

Jacob Pfau and Geoffrey Irving

Mar 21, 2025, 2:05 PM

30 points

4 comments8 min readLW link

Annotated reply to Bengio’s “AI Scientists: Safe and Useful AI?”

Roman LeventovMay 8, 2023, 9:26 PM

18 points

2 comments7 min readLW link

(yoshuabengio.org)

Requirements for a STEM-capable AGI Value Learner (my Case for Less Doom)

RogerDearnaleyMay 25, 2023, 9:26 AM

33 points

3 comments15 min readLW link

An LLM-based “exemplary actor”

Roman LeventovMay 29, 2023, 11:12 AM

16 points

0 comments12 min readLW link

A potentially high impact differential technological development area

Noosphere89Jun 8, 2023, 2:33 PM

5 points

2 comments2 min readLW link

Philosophical Cyborg (Part 2)...or, The Good Successor

ukc10014Jun 21, 2023, 3:43 PM

21 points

1 comment31 min readLW link

How I Learned To Stop Worrying And Love The Shoggoth

Peter MerelJul 12, 2023, 5:47 PM

9 points

15 comments5 min readLW link

Robustness of Model-Graded Evaluations and Automated Interpretability

Simon Lermen and viluon

Jul 15, 2023, 7:12 PM

47 points

5 comments9 min readLW link

[Question] Have you ever considered taking the ‘Turing Test’ yourself?

Super AGIJul 27, 2023, 3:48 AM

2 points

6 comments1 min readLW link

Could We Automate AI Alignment Research?

Stephen McAleeseAug 10, 2023, 12:17 PM

34 points

10 comments21 min readLW link

[Question] Would you ask a genie to give you the solution to alignment?

sudoAug 24, 2022, 1:29 AM

6 points

1 comment1 min readLW link

Prize for Alignment Research Tasks

stuhlmueller and William_S

Apr 29, 2022, 8:57 AM

64 points

38 comments10 min readLW link

Ngo and Yudkowsky on alignment difficulty

Eliezer Yudkowsky and Richard_Ngo

Nov 15, 2021, 8:31 PM

254 points

151 comments99 min readLW link 1 review

How should DeepMind’s Chinchilla revise our AI forecasts?

Cleo NardoSep 15, 2022, 5:54 PM

35 points

12 comments13 min readLW link

Conditioning Generative Models for Alignment

JozdienJul 18, 2022, 7:11 AM

60 points

8 comments20 min readLW link

Provably Honest—A First Step

Srijanak DeNov 5, 2022, 7:18 PM

10 points

2 comments8 min readLW link

Research request (alignment strategy): Deep dive on “making AI solve alignment for us”

JanBDec 1, 2022, 2:55 PM

16 points

3 comments1 min readLW link

Results from a survey on tool use and workflows in alignment research

jacquesthibs, Jan, janus and Logan Riggs

Dec 19, 2022, 3:19 PM

79 points

2 comments19 min readLW link

Research Direction: Be the AGI you want to see in the world

scottviteri, sudo and Lauro Langosco

Feb 5, 2023, 7:15 AM

44 points

0 comments7 min readLW link

Curiosity as a Solution to AGI Alignment

Harsha G.Feb 26, 2023, 11:36 PM

7 points

7 comments3 min readLW link

[Question] Can we get an AI to “do our alignment homework for us”?

Chris_LeongFeb 26, 2024, 7:56 AM

53 points

33 comments1 min readLW link

Introducing AI Alignment Inc., a California public benefit corporation...

TherapistAIMar 7, 2023, 6:47 PM

1 point

4 comments1 min readLW link

Paper review: “The Unreasonable Effectiveness of Easy Training Data for Hard Tasks”

Vassil TashevFeb 29, 2024, 6:44 PM

11 points

0 comments4 min readLW link

Artificial Static Place Intelligence: Guaranteed Alignment

ankFeb 15, 2025, 11:08 AM

2 points

2 comments2 min readLW link

Alignment in Thought Chains

Faust NemesisMar 4, 2024, 7:24 PM

1 point

0 comments2 min readLW link

A Review of Weak to Strong Generalization [AI Safety Camp]

sevdeawesomeMar 7, 2024, 5:16 PM

14 points

0 comments9 min readLW link

W2SG: Introduction

Maria KaprosMar 10, 2024, 4:25 PM

2 points

2 comments10 min readLW link

The Ideal Speech Situation as a Tool for AI Ethical Reflection: A Framework for Alignment

kenneth myersFeb 9, 2024, 6:40 PM

6 points

12 comments3 min readLW link

A Review of In-Context Learning Hypotheses for Automated AI Alignment Research

alamertonApr 18, 2024, 6:29 PM

25 points

4 comments16 min readLW link

Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller

Henry CaiJun 16, 2024, 1:01 PM

7 points

0 comments7 min readLW link

(arxiv.org)

As We May Align

Gilbert CDec 20, 2024, 7:02 PM

−1 points

0 comments6 min readLW link

[Question] How to devour 5000 pages within a day if Chatgpt crashes upon the +50mb file containing the content? Need some recommendations.

GameSep 27, 2024, 7:30 AM

1 point

0 comments1 min readLW link

[Question] Are Sparse Autoencoders a good idea for AI control?

Gerard BoxoDec 26, 2024, 5:34 PM

3 points

4 comments1 min readLW link

AIsip Manifesto: A Scientific Exploration of Harmonious Co-Existence Between Humans, AI, and All Beings ChatGPT-4o’s Independent Perspective on AIsip, Signed by ChatGPT-4o and Endorsed by Carl Sellman

Carl SellmanOct 11, 2024, 7:06 PM

1 point

0 comments3 min readLW link

AI Alignment via Slow Substrates: Early Empirical Results With StarCraft II

Lester LeongOct 14, 2024, 4:05 AM

60 points

9 comments12 min readLW link

A Solution for AGI/ASI Safety

Weibing WangDec 18, 2024, 7:44 PM

50 points

29 comments1 min readLW link

AI-assisted alignment proposals require specific decomposition of capabilities

RobertMMar 30, 2023, 9:31 PM

16 points

2 comments6 min readLW link

Godzilla Strategies

johnswentworthJun 11, 2022, 3:44 PM

159 points

72 comments3 min readLW link

Model-driven feedback could amplify alignment failures

aogJan 30, 2023, 12:00 AM

21 points

1 comment2 min readLW link

Proposal: Derivative Information Theory (DIT) — A Dynamic Model of Agency and Consciousness

YogmogApr 14, 2025, 12:27 AM

1 point

0 comments2 min readLW link

Recursive alignment with the principle of alignment

hiveFeb 27, 2025, 2:34 AM

9 points

1 comment15 min readLW link

(hiveism.substack.com)

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning

Jeremias FerraoApr 18, 2025, 7:34 PM

3 points

0 comments10 min readLW link

Is Alignment a flawed approach?

Patrick BernardMar 11, 2025, 8:32 PM

1 point

0 comments3 min readLW link

Tetherware #1: The case for humanlike AI with free will

Jáchym FibírJan 30, 2025, 10:58 AM

5 points

14 comments10 min readLW link

(tetherware.substack.com)

[Question] How far along Metr’s law can AI start automating or helping with alignment research?

Christopher KingMar 20, 2025, 3:58 PM

20 points

21 comments1 min readLW link

The Overlap Paradigm: Rethinking Data’s Role in Weak-to-Strong Generalization (W2SG)

Serhii ZamriiFeb 3, 2025, 7:31 PM

2 points

0 comments11 min readLW link

Language Models and World Models, a Philosophy

kyjohnsoFeb 3, 2025, 2:55 AM

1 point

0 comments1 min readLW link

(hylaeansea.org)

Gettier Cases [repost]

AntigoneFeb 3, 2025, 6:12 PM

−4 points

4 comments2 min readLW link

Constitutional Classifiers: Defending against universal jailbreaks (Anthropic Blog)

ArchimedesFeb 4, 2025, 2:55 AM

16 points

1 comment1 min readLW link

(www.anthropic.com)

Does Time Linearity Shape Human Self-Directed Evolution, and will AGI/ASI Transcend or Destabilise Reality?

EmmelyFeb 5, 2025, 7:58 AM

1 point

0 comments3 min readLW link

Emergence of superintelligence from AI hiveminds: how to make it human-friendly?

Mitchell_PorterApr 27, 2025, 4:51 AM

10 points

0 comments2 min readLW link

Exploring the Precautionary Principle in AI Development: Historical Analogies and Lessons Learned

Christopher KingMar 21, 2023, 3:53 AM

−1 points

2 comments9 min readLW link

“Unintentional AI safety research”: Why not systematically mine AI technical research for safety purposes?

Jemal YoungMar 29, 2023, 3:56 PM

27 points

3 comments6 min readLW link

[Question] Daisy-chaining epsilon-step verifiers

DecaeneusApr 6, 2023, 2:07 AM

2 points

1 comment1 min readLW link

Scientism vs. people

Roman LeventovApr 18, 2023, 5:28 PM

4 points

4 comments11 min readLW link

How to express this system for ethically aligned AGI as a Mathematical formula?

Oliver SiegelApr 19, 2023, 8:13 PM

−1 points

0 comments1 min readLW link

[Question] Shouldn’t we ‘Just’ Superimitate Low-Res Uploads?

lukemarksNov 3, 2023, 7:42 AM

15 points

2 comments2 min readLW link

No comments.

AI-As­sisted Alignment

AI-Assisted Alignment