RSS

Align­ment Tax

TagLast edit: Dec 30, 2024, 8:40 PM by Dakara

Alignment Tax (sometimes called a safety tax) is the extra cost of ensuring that an AI system is aligned, relative to the cost of building an unaligned alternative. The term ‘tax’ can be misleading: in the safety literature, ‘alignment/​safety tax’ or ‘alignment cost’ is meant to refer to increased developer time, extra compute, or decreased performance, and not only to the financial cost/​tax required to build an aligned system.

In order to get a better idea of what the alignment tax is, consider some of the cases that lie at the edges. The best case scenario is No Tax: This means we lose no performance by aligning the system, so there is no reason to deploy an AI that is not aligned, i.e., we might as well align it. The worst case scenario is Max Tax: This means that we lose all performance by aligning the system, so alignment is functionally impossible. So you either deploy an unaligned system, or you don’t get any benefit from AI systems at all. We expect something in between these two scenarios to be the case.

Paul Christiano distinguishes two main approaches for dealing with the alignment tax.[1][2]

Further reading

The case for a nega­tive al­ign­ment tax

Sep 18, 2024, 6:33 PM
75 points
20 comments7 min readLW link

Safety tax functions

owencbOct 20, 2024, 2:08 PM
30 points
0 comments6 min readLW link
(strangecities.substack.com)

AI safety tax dynamics

owencbOct 23, 2024, 12:18 PM
22 points
0 comments6 min readLW link
(strangecities.substack.com)

[Linkpost] Jan Leike on three kinds of al­ign­ment taxes

AkashJan 6, 2023, 11:57 PM
27 points
2 comments3 min readLW link
(aligned.substack.com)

Against ubiquitous al­ign­ment taxes

berenMar 6, 2023, 7:50 PM
56 points
10 comments2 min readLW link

The case for re­mov­ing al­ign­ment and ML re­search from the train­ing dataset

berenMay 30, 2023, 8:54 PM
48 points
8 comments5 min readLW link

How difficult is AI Align­ment?

Sammy MartinSep 13, 2024, 3:47 PM
44 points
6 comments23 min readLW link

Ten Levels of AI Align­ment Difficulty

Sammy MartinJul 3, 2023, 8:20 PM
129 points
24 comments12 min readLW link1 review

La­bor Par­ti­ci­pa­tion is a High-Pri­or­ity AI Align­ment Risk

alexJun 17, 2024, 6:09 PM
6 points
0 comments17 min readLW link

Se­cu­rity Mind­set and the Lo­gis­tic Suc­cess Curve

Eliezer YudkowskyNov 26, 2017, 3:58 PM
106 points
49 comments20 min readLW link

The com­mer­cial in­cen­tive to in­ten­tion­ally train AI to de­ceive us

Derek M. JonesDec 29, 2022, 11:30 AM
5 points
1 comment4 min readLW link
(shape-of-code.com)

On the Im­por­tance of Open Sourc­ing Re­ward Models

elandgreJan 2, 2023, 7:01 PM
18 points
5 comments6 min readLW link
No comments.