RSS

Ad­ver­sar­ial Ex­am­ples (AI)

TagLast edit: 14 Dec 2024 1:56 UTC by Ruby

SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

Jessica Rumbelow and mwatkins
5 Feb 2023 22:02 UTC
679 points
206 comments12 min readLW link1 review

Iron­ing Out the Squiggles

Zack_M_Davis29 Apr 2024 16:13 UTC
157 points
36 comments11 min readLW link

AI Safety in a World of Vuln­er­a­ble Ma­chine Learn­ing Systems

AdamGleave and EuanMcLean
8 Mar 2023 2:40 UTC
70 points
28 comments29 min readLW link
(far.ai)

If I were a well-in­ten­tioned AI… I: Image classifier

Stuart_Armstrong26 Feb 2020 12:39 UTC
35 points
4 comments5 min readLW link

Solv­ing ad­ver­sar­ial at­tacks in com­puter vi­sion as a baby ver­sion of gen­eral AI alignment

Stanislav Fort29 Aug 2024 17:17 UTC
87 points
8 comments7 min readLW link

[Question] What progress have we made on au­to­mated au­dit­ing?

LawrenceC6 Jul 2024 1:49 UTC
38 points
1 comment1 min readLW link

AXRP Epi­sode 1 - Ad­ver­sar­ial Poli­cies with Adam Gleave

DanielFilan29 Dec 2020 20:41 UTC
12 points
5 comments34 min readLW link

Deep For­get­ting & Un­learn­ing for Safely-Scoped LLMs

scasper5 Dec 2023 16:48 UTC
123 points
30 comments13 min readLW link

The Good­hart Game

John_Maxwell18 Nov 2019 23:22 UTC
13 points
5 comments5 min readLW link

Ad­ver­sar­ial Ro­bust­ness Could Help Prevent Catas­trophic Misuse

aogara11 Dec 2023 19:12 UTC
30 points
18 comments9 min readLW link

There are (prob­a­bly) no su­per­hu­man Go AIs: strong hu­man play­ers beat the strongest AIs

Taran19 Feb 2023 12:25 UTC
124 points
34 comments4 min readLW link

RAIN: Your Lan­guage Models Can Align Them­selves with­out Fine­tun­ing—Microsoft Re­search 2023 - Re­duces the ad­ver­sar­ial prompt at­tack suc­cess rate from 94% to 19%!

Singularian250124 Sep 2023 16:48 UTC
5 points
0 comments1 min readLW link

Ad­ver­sar­ial Poli­cies Beat Pro­fes­sional-Level Go AIs

sanxiyn3 Nov 2022 13:27 UTC
31 points
35 comments1 min readLW link
(goattack.alignmentfund.org)

Hu­man beats SOTA Go AI by learn­ing an ad­ver­sar­ial policy

Vanessa Kosoy19 Feb 2023 9:38 UTC
58 points
32 comments1 min readLW link
(goattack.far.ai)

Ad­ver­sar­ial at­tacks and op­ti­mal control

Jan22 May 2022 18:22 UTC
17 points
7 comments8 min readLW link
(universalprior.substack.com)

A Search for More ChatGPT /​ GPT-3.5 /​ GPT-4 “Un­speak­able” Glitch Tokens

Martin Fell9 May 2023 14:36 UTC
26 points
9 comments6 min readLW link

EIS X: Con­tinual Learn­ing, Mo­du­lar­ity, Com­pres­sion, and Biolog­i­cal Brains

scasper21 Feb 2023 16:59 UTC
14 points
4 comments3 min readLW link

EIS XII: Sum­mary

scasper23 Feb 2023 17:45 UTC
18 points
0 comments6 min readLW link

[AN #62] Are ad­ver­sar­ial ex­am­ples caused by real but im­per­cep­ti­ble fea­tures?

Rohin Shah22 Aug 2019 17:10 UTC
28 points
10 comments9 min readLW link
(mailchi.mp)

Even Su­per­hu­man Go AIs Have Sur­pris­ing Failure Modes

AdamGleave, EuanMcLean, Tony Wang, Kellin Pelrine, Tom Tseng, Yawen Duan, Joseph Miller and MichaelDennis
20 Jul 2023 17:31 UTC
129 points
22 comments10 min readLW link
(far.ai)

Analysing Ad­ver­sar­ial At­tacks with Lin­ear Probing

Yoann Poupart, Imene Kerboua, Clement Neo and Jason Hoelscher-Obermaier
17 Jun 2024 14:16 UTC
9 points
0 comments8 min readLW link

The Achilles Heel Hy­poth­e­sis for AI

scasper13 Oct 2020 14:35 UTC
20 points
6 comments1 min readLW link

An ad­ver­sar­ial ex­am­ple for Direct Logit At­tri­bu­tion: mem­ory man­age­ment in gelu-4l

Can, Yeu-Tong Lau, James Dao and Jett Janiak
30 Aug 2023 17:36 UTC
17 points
0 comments8 min readLW link
(arxiv.org)

Image Hi­jacks: Ad­ver­sar­ial Images can Con­trol Gen­er­a­tive Models at Runtime

Scott Emmons, Luke Bailey and Euan Ong
20 Sep 2023 15:23 UTC
58 points
9 comments1 min readLW link
(arxiv.org)

EIS IX: In­ter­pretabil­ity and Adversaries

scasper20 Feb 2023 18:25 UTC
30 points
8 comments8 min readLW link

Beyond the Board: Ex­plor­ing AI Ro­bust­ness Through Go

AdamGleave19 Jun 2024 16:40 UTC
41 points
2 comments1 min readLW link
(far.ai)

Ev­i­dence Sets: Towards In­duc­tive-Bi­ases based Anal­y­sis of Pro­saic AGI

bayesian_kitten16 Dec 2021 22:41 UTC
22 points
10 comments21 min readLW link

Does ro­bust­ness im­prove with scale?

ChengCheng, niki.h, Ian McKenzie, Oskar Hollinsworth, Tom Tseng and AdamGleave
25 Jul 2024 20:55 UTC
14 points
0 comments1 min readLW link
(far.ai)

High-stakes al­ign­ment via ad­ver­sar­ial train­ing [Red­wood Re­search re­port]

dmz, LawrenceC and Nate Thomas
5 May 2022 0:59 UTC
142 points
29 comments9 min readLW link

Smar­tyHead­erCode: anoma­lous to­kens for GPT3.5 and GPT-4

AdamYedidia15 Apr 2023 22:35 UTC
71 points
18 comments6 min readLW link

Fea­tures and Ad­ver­saries in MemoryDT

Joseph Bloom and Jay Bailey
20 Oct 2023 7:32 UTC
31 points
6 comments25 min readLW link

Arte­facts gen­er­ated by mode col­lapse in GPT-4 Turbo serve as ad­ver­sar­ial at­tacks.

Sohaib Imran10 Nov 2023 15:23 UTC
11 points
0 comments2 min readLW link
No comments.