RSS

GPT

TagLast edit: Feb 19, 2023, 2:36 AM by Multicore

GPT (Generative Pretrained Transformer) is a family of large transformer-based language models created by OpenAI. Its ability to generate remarkably human-like responses has relevance to discussions on AGI.

External links:

GPT-3 Paper

GPT-3 Website

Col­lec­tion of GPT-3 results

Kaj_SotalaJul 18, 2020, 8:04 PM
89 points
24 comments1 min readLW link
(twitter.com)

[Question] To what ex­tent is GPT-3 ca­pa­ble of rea­son­ing?

TurnTroutJul 20, 2020, 5:10 PM
70 points
73 comments16 min readLW link

GPT-3: a dis­ap­point­ing paper

nostalgebraistMay 29, 2020, 7:06 PM
65 points
43 comments8 min readLW link1 review

$1000 bounty for OpenAI to show whether GPT3 was “de­liber­ately” pre­tend­ing to be stupi­der than it is

Bird ConceptJul 21, 2020, 6:42 PM
56 points
39 comments2 min readLW link
(twitter.com)

Two Small Ex­per­i­ments on GPT-2

jimrandomhFeb 21, 2019, 2:59 AM
54 points
28 comments1 min readLW link

GPT-3 Fic­tion Samples

gwernJun 25, 2020, 4:12 PM
63 points
15 comments1 min readLW link
(www.gwern.net)

345M ver­sion GPT-2 released

lifelonglearnerMay 5, 2019, 2:49 AM
37 points
0 comments1 min readLW link
(openai.com)

‘This Waifu Does Not Ex­ist’: 100,000 StyleGAN & GPT-2 samples

gwernMar 1, 2019, 4:29 AM
39 points
6 comments1 min readLW link
(www.thiswaifudoesnotexist.net)

[Question] How “hon­est” is GPT-3?

abramdemskiJul 8, 2020, 7:38 PM
72 points
18 comments5 min readLW link

Align­ment As A Bot­tle­neck To Use­ful­ness Of GPT-3

johnswentworthJul 21, 2020, 8:02 PM
111 points
57 comments3 min readLW link

Repli­cat­ing the repli­ca­tion crisis with GPT-3?

skybrianJul 22, 2020, 9:20 PM
29 points
10 comments1 min readLW link

Does GPT-2 Un­der­stand Any­thing?

Douglas Summers-StayJan 2, 2020, 5:09 PM
37 points
23 comments5 min readLW link

[Question] How well can the GPT ar­chi­tec­ture solve the par­ity task?

FactorialCodeJul 11, 2020, 7:02 PM
19 points
3 comments1 min readLW link

Devel­op­men­tal Stages of GPTs

orthonormalJul 26, 2020, 10:03 PM
140 points
72 comments7 min readLW link1 review

Can you get AGI from a Trans­former?

Steven ByrnesJul 23, 2020, 3:27 PM
117 points
40 comments12 min readLW link

OpenGPT-2: We Repli­cated GPT-2 Be­cause You Can Too

avturchinAug 23, 2019, 11:32 AM
18 points
0 comments1 min readLW link
(medium.com)

larger lan­guage mod­els may dis­ap­point you [or, an eter­nally un­finished draft]

nostalgebraistNov 26, 2021, 11:08 PM
260 points
31 comments31 min readLW link2 reviews

Hu­mans Who Are Not Con­cen­trat­ing Are Not Gen­eral Intelligences

sarahconstantinFeb 25, 2019, 8:40 PM
190 points
35 comments6 min readLW link1 review
(srconstantin.wordpress.com)

Writ­ing with GPT-3

Jacob FalkovichJul 24, 2020, 3:22 PM
42 points
0 comments4 min readLW link

Are we in an AI over­hang?

Andy JonesJul 27, 2020, 12:48 PM
266 points
106 comments4 min readLW link

[Question] How will in­ter­net fo­rums like LW be able to defend against GPT-style spam?

ChristianKlJul 28, 2020, 8:12 PM
14 points
17 comments1 min readLW link

An­a­lyz­ing the Prob­lem GPT-3 is Try­ing to Solve

adamShimiAug 6, 2020, 9:58 PM
16 points
2 comments4 min readLW link

GPT-3, be­lief, and consistency

skybrianAug 16, 2020, 11:12 PM
18 points
7 comments2 min readLW link

The Hacker Learns to Trust

Ben PaceJun 22, 2019, 12:27 AM
80 points
18 comments8 min readLW link
(medium.com)

[April Fools] User GPT2 is Banned

jimrandomhApr 2, 2019, 6:00 AM
65 points
20 comments1 min readLW link

GPT-2: 6-Month Fol­low-Up

lifelonglearnerAug 21, 2019, 5:06 AM
28 points
1 comment1 min readLW link

the scal­ing “in­con­sis­tency”: openAI’s new insight

nostalgebraistNov 7, 2020, 7:40 AM
148 points
14 comments9 min readLW link
(nostalgebraist.tumblr.com)

[Question] If GPT-6 is hu­man-level AGI but costs $200 per page of out­put, what would hap­pen?

Daniel KokotajloOct 9, 2020, 12:00 PM
29 points
30 comments1 min readLW link

Hiring en­g­ineers and re­searchers to help al­ign GPT-3

paulfchristianoOct 1, 2020, 6:54 PM
206 points
13 comments3 min readLW link

How LLMs are and are not myopic

janusJul 25, 2023, 2:19 AM
134 points
16 comments8 min readLW link

[AN #102]: Meta learn­ing by GPT-3, and a list of full pro­pos­als for AI alignment

Rohin ShahJun 3, 2020, 5:20 PM
38 points
6 comments10 min readLW link
(mailchi.mp)

Us­ing GPT-N to Solve In­ter­pretabil­ity of Neu­ral Net­works: A Re­search Agenda

Sep 3, 2020, 6:27 PM
68 points
11 comments2 min readLW link

in­ter­pret­ing GPT: the logit lens

nostalgebraistAug 31, 2020, 2:47 AM
227 points
38 comments10 min readLW link

Image GPT

Daniel KokotajloJun 18, 2020, 11:41 AM
29 points
27 comments1 min readLW link
(openai.com)

Scaf­folded LLMs as nat­u­ral lan­guage computers

berenApr 12, 2023, 10:47 AM
95 points
10 comments11 min readLW link

GPT-4 Plugs In

ZviMar 27, 2023, 12:10 PM
198 points
47 comments6 min readLW link
(thezvi.wordpress.com)

[Question] Will OpenAI’s work un­in­ten­tion­ally in­crease ex­is­ten­tial risks re­lated to AI?

adamShimiAug 11, 2020, 6:16 PM
53 points
55 comments1 min readLW link

[ASoT] Fine­tun­ing, RL, and GPT’s world prior

JozdienDec 2, 2022, 4:33 PM
44 points
8 comments5 min readLW link

OpenAI an­nounces GPT-3

gwernMay 29, 2020, 1:49 AM
67 points
23 comments1 min readLW link
(arxiv.org)

Ex­trap­o­lat­ing GPT-N performance

Lukas FinnvedenDec 18, 2020, 9:41 PM
112 points
31 comments22 min readLW link1 review

Au­tore­gres­sive Propaganda

lsusrAug 22, 2021, 2:18 AM
25 points
3 comments3 min readLW link

GPT-4 Predictions

Stephen McAleeseFeb 17, 2023, 11:20 PM
110 points
27 comments11 min readLW link

Can sub­marines swim?

jasoncrawfordFeb 22, 2023, 6:48 PM
18 points
14 comments13 min readLW link
(rootsofprogress.org)

DALL-E by OpenAI

Daniel KokotajloJan 5, 2021, 8:05 PM
97 points
20 comments1 min readLW link

Pre­dic­tions for GPT-N

hippkeJul 29, 2020, 1:16 AM
36 points
31 comments1 min readLW link

[Question] GPT-4 and ASCII Images?

carterallenMar 19, 2023, 3:46 PM
10 points
17 comments1 min readLW link

is gpt-3 few-shot ready for real ap­pli­ca­tions?

nostalgebraistAug 3, 2020, 7:50 PM
31 points
5 comments9 min readLW link
(nostalgebraist.tumblr.com)

Cyborgism

Feb 10, 2023, 2:47 PM
336 points
46 comments35 min readLW link2 reviews

Simulators

janusSep 2, 2022, 12:45 PM
631 points
168 comments41 min readLW link8 reviews
(generative.ink)

I wanted to in­ter­view Eliezer Yud­kowsky but he’s busy so I simu­lated him instead

lsusrSep 16, 2021, 7:34 AM
111 points
33 comments5 min readLW link

[LINK] - ChatGPT discussion

JanBDec 1, 2022, 3:04 PM
13 points
8 comments1 min readLW link
(openai.com)

Map­ping the se­man­tic void: Strange go­ings-on in GPT em­bed­ding spaces

mwatkinsDec 14, 2023, 1:10 PM
114 points
31 comments14 min readLW link

GPT-4o My and Google I/​O Day

ZviMay 16, 2024, 5:50 PM
41 points
2 comments37 min readLW link
(thezvi.wordpress.com)

Do Not Mess With Scar­lett Johansson

ZviMay 22, 2024, 3:10 PM
65 points
7 comments16 min readLW link
(thezvi.wordpress.com)

′ pe­ter­todd’’s last stand: The fi­nal days of open GPT-3 research

mwatkinsJan 22, 2024, 6:47 PM
109 points
16 comments45 min readLW link

What’s up with all the non-Mor­mons? Weirdly spe­cific uni­ver­sal­ities across LLMs

mwatkinsApr 19, 2024, 1:43 PM
40 points
13 comments27 min readLW link

Nav­i­gat­ing LLM em­bed­ding spaces us­ing archetype-based directions

mwatkinsMay 8, 2024, 5:54 AM
15 points
4 comments28 min readLW link

OpenAI re­leases GPT-4o, na­tively in­ter­fac­ing with text, voice and vision

Martín SotoMay 13, 2024, 6:50 PM
54 points
23 comments1 min readLW link
(openai.com)

Get­ting 50% (SoTA) on ARC-AGI with GPT-4o

ryan_greenblattJun 17, 2024, 6:44 PM
262 points
50 comments13 min readLW link

Why did ChatGPT say that? Prompt en­g­ineer­ing and more, with PIZZA.

Jessica RumbelowAug 3, 2024, 12:07 PM
41 points
2 comments4 min readLW link

Ex­plor­ing the pe­ter­todd /​ Leilan du­al­ity in GPT-2 and GPT-J

mwatkinsDec 23, 2024, 1:17 PM
12 points
1 comment17 min readLW link

[Question] Why is o1 so de­cep­tive?

abramdemskiSep 27, 2024, 5:27 PM
179 points
24 comments3 min readLW link

HDBSCAN is Sur­pris­ingly Effec­tive at Find­ing In­ter­pretable Clusters of the SAE De­coder Matrix

Oct 11, 2024, 11:06 PM
8 points
2 comments10 min readLW link

BIG-Bench Ca­nary Con­tam­i­na­tion in GPT-4

JozdienOct 22, 2024, 3:40 PM
123 points
14 comments4 min readLW link

On GPT-4.5

ZviMar 3, 2025, 1:40 PM
44 points
12 comments22 min readLW link
(thezvi.wordpress.com)

GPT-4 for per­sonal pro­duc­tivity: on­line dis­trac­tion blocker

SergiiSep 26, 2023, 5:41 PM
65 points
13 comments2 min readLW link
(grgv.xyz)

AI #4: In­tro­duc­ing GPT-4

ZviMar 21, 2023, 2:00 PM
101 points
32 comments103 min readLW link
(thezvi.wordpress.com)

Stan­ford claims to have repli­cated ChatGPT for < $600

NoSignalNoNoiseMar 21, 2023, 2:28 AM
2 points
1 comment1 min readLW link
(crfm.stanford.edu)

Agen­tic GPT simu­la­tions: a risk and an opportunity

Yair HalberstadtMar 22, 2023, 6:24 AM
24 points
8 comments1 min readLW link

Sparks of Ar­tifi­cial Gen­eral In­tel­li­gence: Early ex­per­i­ments with GPT-4 | Microsoft Research

DragonGodMar 23, 2023, 5:45 AM
68 points
23 comments1 min readLW link
(arxiv.org)

Microsoft Re­search Paper Claims Sparks of Ar­tifi­cial In­tel­li­gence in GPT-4

ZviMar 24, 2023, 1:20 PM
72 points
14 comments6 min readLW link
(thezvi.wordpress.com)

[Question] Can GPT-4 play 20 ques­tions against an­other in­stance of it­self?

Nathan Helm-BurgerMar 28, 2023, 1:11 AM
15 points
1 comment1 min readLW link
(evanthebouncy.medium.com)

Creat­ing a fam­ily with GPT-4

Kaj_SotalaMar 28, 2023, 6:40 AM
23 points
3 comments10 min readLW link
(kajsotala.fi)

ChatGPT and Bing Chat can’t play Botticelli

Asha SaavossMar 29, 2023, 5:39 PM
11 points
0 comments6 min readLW link

Ar­gu­ing all sides with ChatGPT

Richard_KennawayMar 30, 2023, 7:50 PM
16 points
1 comment8 min readLW link

Anal­y­sis of GPT-4 com­pe­tence in as­sess­ing com­plex le­gal lan­guage: Ex­am­ple of Bill C-11 of the Cana­dian Par­li­a­ment. - Part 1

M. Y. ZuoApr 2, 2023, 12:01 AM
12 points
2 comments14 min readLW link

GTP4 ca­pa­ble of limited re­cur­sive im­prov­ing?

Boris KashirinApr 2, 2023, 9:38 PM
2 points
3 comments1 min readLW link

[Question] ChatGTP “Writ­ing ” News Sto­ries for The Guardian?

jmhApr 7, 2023, 12:16 PM
1 point
4 comments1 min readLW link

GPTs are Pre­dic­tors, not Imitators

Eliezer YudkowskyApr 8, 2023, 7:59 PM
416 points
99 comments3 min readLW link3 reviews

[Question] Does GPT-4′s abil­ity to com­press text in a way that it can ac­tu­ally de­com­press in­di­cate self-aware­ness?

FinalFormal2Apr 10, 2023, 4:48 PM
3 points
2 comments1 min readLW link

Why Si­mu­la­tor AIs want to be Ac­tive In­fer­ence AIs

Apr 10, 2023, 6:23 PM
93 points
9 comments8 min readLW link1 review

On AutoGPT

ZviApr 13, 2023, 12:30 PM
248 points
47 comments20 min readLW link
(thezvi.wordpress.com)

Steer­ing GPT-2-XL by adding an ac­ti­va­tion vector

May 13, 2023, 6:42 PM
437 points
98 comments50 min readLW link1 review

The ‘ pe­ter­todd’ phenomenon

mwatkinsApr 15, 2023, 12:59 AM
192 points
50 comments38 min readLW link1 review

Study 1b: This One Weird Trick does NOT cause in­cor­rect­ness cascades

Robert_AIZIApr 20, 2023, 6:10 PM
5 points
0 comments6 min readLW link
(aizi.substack.com)

OpenAI’s GPT-4 Safety Goals

PeterMcCluskeyApr 22, 2023, 7:11 PM
3 points
3 comments4 min readLW link
(bayesianinvestor.com)

On OpenAI Dev Day

ZviNov 9, 2023, 4:10 PM
60 points
0 comments15 min readLW link
(thezvi.wordpress.com)

What’s Your Cog­ni­tive Al­gorithm?

RaemonJun 18, 2020, 10:16 PM
74 points
23 comments13 min readLW link

Lin­ear en­cod­ing of char­ac­ter-level in­for­ma­tion in GPT-J to­ken embeddings

Nov 10, 2023, 10:19 PM
34 points
4 comments28 min readLW link

Study­ing The Alien Mind

Dec 5, 2023, 5:27 PM
80 points
10 comments15 min readLW link

The “AI Dun­geons” Dragon Model is heav­ily path de­pen­dent (test­ing GPT-3 on ethics)

Rafael HarthJul 21, 2020, 12:14 PM
44 points
9 comments6 min readLW link

GPT-3 Gems

TurnTroutJul 23, 2020, 12:46 AM
33 points
10 comments48 min readLW link

[Question] Ques­tion on GPT-3 Ex­cel Demo

Zhitao HouJun 22, 2020, 8:31 PM
0 points
1 comment1 min readLW link

[Question] Are we cer­tain that gpt-2 and similar al­gorithms are not self-aware?

OzyrusJul 11, 2019, 8:37 AM
0 points
12 comments1 min readLW link

[Question] What should we ex­pect from GPT-3?

avturchinMar 21, 2019, 2:28 PM
22 points
2 comments1 min readLW link

[Question] List of pub­lic pre­dic­tions of what GPT-X can or can’t do?

Daniel KokotajloJun 14, 2020, 2:25 PM
20 points
9 comments1 min readLW link

GPT-3: A Summary

leogaoJun 2, 2020, 6:14 PM
20 points
0 comments1 min readLW link
(leogao.dev)

[Question] If AI is based on GPT, how to en­sure its safety?

avturchinJun 18, 2020, 8:33 PM
20 points
11 comments1 min readLW link

[up­dated] how does gpt2′s train­ing cor­pus cap­ture in­ter­net dis­cus­sion? not well

nostalgebraistJul 27, 2020, 10:30 PM
25 points
3 comments2 min readLW link
(nostalgebraist.tumblr.com)

[Question] Prob­a­bil­ity that other ar­chi­tec­tures will scale as well as Trans­form­ers?

Daniel KokotajloJul 28, 2020, 7:36 PM
22 points
4 comments1 min readLW link

[Question] To what ex­tent are the scal­ing prop­er­ties of Trans­former net­works ex­cep­tional?

abramdemskiJul 28, 2020, 8:06 PM
30 points
1 comment1 min readLW link

En­gag­ing Se­ri­ously with Short Timelines

sapphireJul 29, 2020, 7:21 PM
43 points
21 comments3 min readLW link

Lan­guage Models are a Po­ten­tially Safe Path to Hu­man-Level AGI

Nadav BrandesApr 20, 2023, 12:40 AM
28 points
7 comments8 min readLW link1 review

GPT as an “In­tel­li­gence Fork­lift.”

boazbarakMay 19, 2023, 9:15 PM
49 points
27 comments3 min readLW link

PaLM-2 & GPT-4 in “Ex­trap­o­lat­ing GPT-N perfor­mance”

Lukas FinnvedenMay 30, 2023, 6:33 PM
57 points
6 comments6 min readLW link

Eval­u­at­ing strate­gic rea­son­ing in GPT models

phelps-sgMay 25, 2023, 11:51 AM
4 points
1 comment8 min readLW link

Ex­per­i­ments in Eval­u­at­ing Steer­ing Vectors

Gytis DaujotasJun 19, 2023, 3:11 PM
34 points
4 comments4 min readLW link

[Question] How hard would it be to change GPT-3 in a way that al­lows au­dio?

ChristianKlAug 28, 2020, 2:42 PM
9 points
5 comments1 min readLW link

Why GPT wants to mesa-op­ti­mize & how we might change this

John_MaxwellSep 19, 2020, 1:48 PM
55 points
33 comments9 min readLW link

[Question] Where is hu­man level on text pre­dic­tion? (GPTs task)

Daniel KokotajloSep 20, 2020, 9:00 AM
27 points
19 comments1 min readLW link

Ex­am­ples of Prompts that Make GPT-4 Out­put Falsehoods

Jul 22, 2023, 8:21 PM
21 points
5 comments6 min readLW link

GPT-4 can catch sub­tle cross-lan­guage trans­la­tion mistakes

Michael TontchevJul 27, 2023, 1:39 AM
7 points
1 comment1 min readLW link

Eval­u­at­ing GPT-4 The­ory of Mind Capabilities

Aug 10, 2023, 5:57 PM
15 points
2 comments14 min readLW link

The Col­lid­ing Ex­po­nen­tials of AI

VermillionOct 14, 2020, 11:31 PM
28 points
16 comments5 min readLW link

Paper: On mea­sur­ing situ­a­tional aware­ness in LLMs

Sep 4, 2023, 12:54 PM
109 points
16 comments5 min readLW link
(arxiv.org)

An ex­pla­na­tion for ev­ery to­ken: us­ing an LLM to sam­ple an­other LLM

Max HOct 11, 2023, 12:53 AM
35 points
5 comments11 min readLW link

Beyond 175 billion pa­ram­e­ters: Can we an­ti­ci­pate fu­ture GPT-X Ca­pa­bil­ities?

bakztfutureDec 4, 2020, 11:42 PM
−1 points
1 comment2 min readLW link

MIRI com­ments on Co­tra’s “Case for Align­ing Nar­rowly Su­per­hu­man Models”

Rob BensingerMar 5, 2021, 11:43 PM
142 points
13 comments26 min readLW link

A sim­ple way to make GPT-3 fol­low instructions

Quintin PopeMar 8, 2021, 2:57 AM
11 points
5 comments4 min readLW link

Thoughts on the Align­ment Im­pli­ca­tions of Scal­ing Lan­guage Models

leogaoJun 2, 2021, 9:32 PM
82 points
11 comments17 min readLW link

New GPT-3 competitor

Quintin PopeAug 12, 2021, 7:05 AM
32 points
10 comments1 min readLW link

AI-Based Code Gen­er­a­tion Us­ing GPT-J-6B

Tomás B.Jun 16, 2021, 3:05 PM
22 points
14 comments1 min readLW link
(minimaxir.com)

GPT-Aug­mented Blogging

lsusrSep 14, 2021, 11:55 AM
52 points
18 comments13 min readLW link

Si­mu­lated Elon Musk Lives in a Simulation

lsusrSep 18, 2021, 7:37 AM
66 points
9 comments3 min readLW link

[Question] How much should you be will­ing to pay for an AGI?

Logan ZoellnerSep 20, 2021, 11:51 AM
11 points
5 comments1 min readLW link

[Question] Any write­ups on GPT agency?

OzyrusSep 26, 2021, 10:55 PM
4 points
6 comments1 min readLW link

[Question] Is GPT-3 already sam­ple-effi­cient?

Daniel KokotajloOct 6, 2021, 1:38 PM
36 points
32 comments1 min readLW link

NVIDIA and Microsoft re­leases 530B pa­ram­e­ter trans­former model, Me­ga­tron-Tur­ing NLG

OzyrusOct 11, 2021, 3:28 PM
51 points
36 comments1 min readLW link
(developer.nvidia.com)

“Sum­ma­riz­ing Books with Hu­man Feed­back” (re­cur­sive GPT-3)

gwernNov 15, 2021, 5:41 PM
24 points
4 comments1 min readLW link
(openai.com)

Reader-gen­er­ated Essays

Henrik KarlssonJan 3, 2022, 8:56 AM
25 points
1 comment6 min readLW link
(escapingflatland.substack.com)

A one-ques­tion Tur­ing test for GPT-3

Jan 22, 2022, 6:17 PM
85 points
25 comments5 min readLW link

Idea: build al­ign­ment dataset for very ca­pa­ble models

Quintin PopeFeb 12, 2022, 7:30 PM
14 points
2 comments3 min readLW link

More GPT-3 and sym­bol grounding

Stuart_ArmstrongFeb 23, 2022, 6:30 PM
21 points
7 comments3 min readLW link

Per­sonal imi­ta­tion software

FlaglandbaseMar 7, 2022, 7:55 AM
6 points
6 comments1 min readLW link

New GPT3 Im­pres­sive Ca­pa­bil­ities—In­struc­tGPT3 [1/​2]

simeon_cMar 13, 2022, 10:58 AM
72 points
10 comments7 min readLW link

Hu­mans pre­tend­ing to be robots pre­tend­ing to be human

Richard_KennawayMar 28, 2022, 3:13 PM
25 points
14 comments1 min readLW link

[Link] Train­ing Com­pute-Op­ti­mal Large Lan­guage Models

nostalgebraistMar 31, 2022, 6:01 PM
51 points
23 comments1 min readLW link
(arxiv.org)

New Scal­ing Laws for Large Lan­guage Models

1a3ornApr 1, 2022, 8:41 PM
246 points
22 comments5 min readLW link

GPT-3 and con­cept extrapolation

Stuart_ArmstrongApr 20, 2022, 10:39 AM
19 points
27 comments1 min readLW link

Get­ting GPT-3 to pre­dict Me­tac­u­lus questions

MathiasKBMay 6, 2022, 6:01 AM
69 points
9 comments2 min readLW link

Pos­i­tive out­comes un­der an un­al­igned AGI takeover

YitzMay 12, 2022, 7:45 AM
19 points
10 comments3 min readLW link

Paper: Teach­ing GPT3 to ex­press un­cer­tainty in words

Owain_EvansMay 31, 2022, 1:27 PM
97 points
7 comments4 min readLW link

OpenAI: GPT-based LLMs show abil­ity to dis­crim­i­nate be­tween its own wrong an­swers, but in­abil­ity to ex­plain how/​why it makes that dis­crim­i­na­tion, even as model scales

Aditya JainJun 13, 2022, 11:33 PM
14 points
5 comments1 min readLW link
(openai.com)

[Question] AI mis­al­ign­ment risk from GPT-like sys­tems?

fiso64Jun 19, 2022, 5:35 PM
10 points
8 comments1 min readLW link

Try­ing out Prompt Eng­ineer­ing on TruthfulQA

Megan KinnimentJul 23, 2022, 2:04 AM
10 points
0 comments8 min readLW link

Us­ing GPT-3 to aug­ment hu­man intelligence

Henrik KarlssonAug 10, 2022, 3:54 PM
52 points
8 comments18 min readLW link
(escapingflatland.substack.com)

What’s the Least Im­pres­sive Thing GPT-4 Won’t be Able to Do

AlgonAug 20, 2022, 7:48 PM
80 points
125 comments1 min readLW link

Progress Re­port 7: mak­ing GPT go hur­rdurr in­stead of brrrrrrr

Nathan Helm-BurgerSep 7, 2022, 3:28 AM
21 points
0 comments4 min readLW link

[ASoT] Thoughts on GPT-N

Ulisse MiniNov 8, 2022, 7:14 AM
8 points
0 comments1 min readLW link

Steer­ing Be­havi­our: Test­ing for (Non-)My­opia in Lan­guage Models

Dec 5, 2022, 8:28 PM
40 points
19 comments10 min readLW link

ChatGPT: First Impressions

specbugDec 1, 2022, 4:36 PM
18 points
2 comments13 min readLW link
(sixeleven.in)

Jailbreak­ing ChatGPT on Re­lease Day

ZviDec 2, 2022, 1:10 PM
242 points
77 comments6 min readLW link1 review
(thezvi.wordpress.com)

Could an AI be Reli­gious?

mk54Dec 4, 2022, 5:00 AM
−12 points
14 comments1 min readLW link

Can GPT-3 Write Con­tra Dances?

jefftkDec 4, 2022, 3:00 AM
6 points
4 comments10 min readLW link
(www.jefftk.com)

A crisis for on­line com­mu­ni­ca­tion: bots and bot users will over­run the In­ter­net?

Mitchell_PorterDec 11, 2022, 9:11 PM
15 points
11 comments1 min readLW link

Triv­ial GPT-3.5 limi­ta­tion workaround

Dave LindberghDec 12, 2022, 8:42 AM
5 points
4 comments1 min readLW link

[Question] Is the ChatGPT-simu­lated Linux vir­tual ma­chine real?

KenoubiDec 13, 2022, 3:41 PM
18 points
7 comments1 min readLW link

Bad at Arith­metic, Promis­ing at Math

cohenmacaulayDec 18, 2022, 5:40 AM
100 points
19 comments20 min readLW link1 review

Next Level Seinfeld

ZviDec 19, 2022, 1:30 PM
50 points
8 comments1 min readLW link
(thezvi.wordpress.com)

Mlyyrczo

lsusrDec 26, 2022, 7:58 AM
41 points
14 comments3 min readLW link

Can ChatGPT count?

p.b.Jan 7, 2023, 7:57 AM
13 points
11 comments2 min readLW link

[Question] GPT learn­ing from smarter texts?

ViliamJan 8, 2023, 10:23 PM
26 points
7 comments1 min readLW link

ChatGPT strug­gles to re­spond to the real world

Alex FlintJan 12, 2023, 4:02 PM
31 points
9 comments24 min readLW link

Large lan­guage mod­els learn to rep­re­sent the world

gjmJan 22, 2023, 1:10 PM
101 points
20 comments3 min readLW link1 review

Ano­ma­lous to­kens re­veal the origi­nal iden­tities of In­struct models

Feb 9, 2023, 1:30 AM
139 points
16 comments9 min readLW link
(generative.ink)

[Question] Is In­struc­tGPT Fol­low­ing In­struc­tions in Other Lan­guages Sur­pris­ing?

DragonGodFeb 13, 2023, 11:26 PM
39 points
15 comments1 min readLW link

The Cave Alle­gory Re­vis­ited: Un­der­stand­ing GPT’s Worldview

Jan_KulveitFeb 14, 2023, 4:00 PM
86 points
5 comments3 min readLW link

The idea that ChatGPT is sim­ply “pre­dict­ing” the next word is, at best, misleading

Bill BenzonFeb 20, 2023, 11:32 AM
55 points
88 comments5 min readLW link

Sto­ry­tel­ling Makes GPT-3.5 Deon­tol­o­gist: Un­ex­pected Effects of Con­text on LLM Behavior

Mar 14, 2023, 8:44 AM
17 points
0 comments12 min readLW link

GPT can write Quines now (GPT-4)

Andrew_CritchMar 14, 2023, 7:18 PM
112 points
30 comments1 min readLW link

ARC tests to see if GPT-4 can es­cape hu­man con­trol; GPT-4 failed to do so

Christopher KingMar 15, 2023, 12:29 AM
116 points
22 comments2 min readLW link

GPT-4 de­vel­oper livestream

Gerald MonroeMar 14, 2023, 8:55 PM
9 points
0 comments1 min readLW link
(www.youtube.com)

A chess game against GPT-4

Rafael HarthMar 16, 2023, 2:05 PM
24 points
23 comments1 min readLW link

GPT-4 Mul­ti­pli­ca­tion Competition

dandelion4Mar 16, 2023, 3:09 AM
11 points
7 comments1 min readLW link

[Question] Will 2023 be the last year you can write short sto­ries and re­ceive most of the in­tel­lec­tual credit for writ­ing them?

lcMar 16, 2023, 9:36 PM
20 points
11 comments1 min readLW link

Is it a bad idea to pay for GPT-4?

nemMar 16, 2023, 8:49 PM
24 points
8 comments1 min readLW link

The Power of High Speed Stupidity

robotelvisMar 17, 2023, 9:41 PM
33 points
6 comments9 min readLW link1 review
(messyprogress.substack.com)

[Question] What did you do with GPT4?

ChristianKlMar 18, 2023, 3:21 PM
27 points
17 comments1 min readLW link

Fea­ture pro­posal: in­te­grate LessWrong with ChatGPT to pro­mote ac­tive reading

DirectedEvolutionMar 19, 2023, 3:41 AM
10 points
4 comments1 min readLW link

Re­marks 1–18 on GPT (com­pressed)

Cleo NardoMar 20, 2023, 10:27 PM
145 points
35 comments31 min readLW link

Ex­plor­ing GPT4′s world model

hippkeMar 20, 2023, 9:31 PM
−5 points
5 comments2 min readLW link

[Question] 10/​50/​90% chance of GPT-N Trans­for­ma­tive AI?

human_generated_textAug 9, 2020, 12:10 AM
24 points
8 comments1 min readLW link

Lan­guage and Ca­pa­bil­ities: Test­ing LLM Math­e­mat­i­cal Abil­ities Across Languages

Ethan EdwardsApr 4, 2024, 1:18 PM
24 points
2 comments36 min readLW link

On agen­tic gen­er­al­ist mod­els: we’re es­sen­tially us­ing ex­ist­ing tech­nol­ogy the weak­est and worst way you can use it

Yuli_BanAug 28, 2024, 1:57 AM
10 points
2 comments9 min readLW link

Us­ing GPT-Eliezer against ChatGPT Jailbreaking

Dec 6, 2022, 7:54 PM
170 points
85 comments9 min readLW link

Philo­soph­i­cal Cy­borg (Part 1)

Jun 14, 2023, 4:20 PM
31 points
4 comments13 min readLW link

GPT-4 solves Gary Mar­cus-in­duced flubs

JakubKMar 17, 2023, 6:40 AM
56 points
29 comments2 min readLW link
(docs.google.com)

OpenAI in­tro­duces func­tion call­ing for GPT-4

Jun 20, 2023, 1:58 AM
24 points
3 comments4 min readLW link
(openai.com)

[Linkpost] Faith and Fate: Limits of Trans­form­ers on Compositionality

Joe KwonJun 16, 2023, 3:04 PM
19 points
4 comments1 min readLW link
(arxiv.org)

[Linkpost] A shared lin­guis­tic space for trans­mit­ting our thoughts from brain to brain in nat­u­ral conversations

Bogdan Ionut CirsteaJul 1, 2023, 1:57 PM
17 points
2 comments1 min readLW link

May Gw­ern.net newslet­ter (w/​GPT-3 com­men­tary)

gwernJun 2, 2020, 3:40 PM
32 points
7 comments1 min readLW link
(www.gwern.net)

A trick for Safer GPT-N

RaziedAug 23, 2020, 12:39 AM
7 points
1 comment2 min readLW link

The In­for­ma­tion: OpenAI shows ‘Straw­berry’ to feds, races to launch it

Martín SotoAug 27, 2024, 11:10 PM
145 points
15 comments3 min readLW link

From GPT to AGI

ChristianKlAug 31, 2020, 1:28 PM
6 points
7 comments1 min readLW link

GPT, the mag­i­cal col­lab­o­ra­tion zone, Lex Frid­man and Sam Altman

Bill BenzonMar 18, 2024, 8:04 PM
3 points
1 comment3 min readLW link

on “learn­ing to sum­ma­rize”

nostalgebraistSep 12, 2020, 3:20 AM
25 points
13 comments8 min readLW link
(nostalgebraist.tumblr.com)

Ex­tract­ing and Eval­u­at­ing Causal Direc­tion in LLMs’ Activations

Dec 14, 2022, 2:33 PM
29 points
5 comments11 min readLW link

Ab­stract con­cepts and met­al­in­gual defi­ni­tion: Does ChatGPT un­der­stand jus­tice and char­ity?

Bill BenzonDec 16, 2022, 9:01 PM
2 points
0 comments13 min readLW link

GPT-2′s po­si­tional em­bed­ding ma­trix is a helix

AdamYedidiaJul 21, 2023, 4:16 AM
44 points
21 comments4 min readLW link

Ret­ro­spec­tive on ‘GPT-4 Pre­dic­tions’ After the Re­lease of GPT-4

Stephen McAleeseMar 17, 2023, 6:34 PM
26 points
6 comments6 min readLW link

[Question] What ex­per­i­ment set­tles the Gary Mar­cus vs Ge­offrey Hin­ton de­bate?

Valentin BaltadzhievFeb 14, 2024, 9:06 AM
12 points
8 comments1 min readLW link

The “spel­ling mir­a­cle”: GPT-3 spel­ling abil­ities and glitch to­kens revisited

mwatkinsJul 31, 2023, 7:47 PM
85 points
29 comments20 min readLW link

Does ChatGPT’s perfor­mance war­rant work­ing on a tu­tor for chil­dren? [It’s time to take it to the lab.]

Bill BenzonDec 19, 2022, 3:12 PM
13 points
5 comments4 min readLW link
(new-savanna.blogspot.com)

Re­searchers and writ­ers can ap­ply for proxy ac­cess to the GPT-3.5 base model (code-davinci-002)

ampdotDec 1, 2023, 6:48 PM
14 points
0 comments1 min readLW link
(airtable.com)

Paper­clipGPT(-4)

Michael TontchevMar 14, 2023, 10:03 PM
7 points
0 comments11 min readLW link

The po­si­tional em­bed­ding ma­trix and pre­vi­ous-to­ken heads: how do they ac­tu­ally work?

AdamYedidiaAug 10, 2023, 1:58 AM
26 points
4 comments13 min readLW link

Si­mu­late the CEO

robotelvisAug 12, 2023, 12:09 AM
23 points
5 comments5 min readLW link
(messyprogress.substack.com)

Trans­fer learn­ing and gen­er­al­iza­tion-qua-ca­pa­bil­ity in Bab­bage and Davinci (or, why di­vi­sion is bet­ter than Span­ish)

RP and agg
Feb 9, 2024, 7:00 AM
50 points
6 comments3 min readLW link

ChatGPT un­der­stands, but largely does not gen­er­ate Span­glish (and other code-mixed) text

Milan WDec 23, 2022, 5:40 PM
15 points
5 comments4 min readLW link

[Question] GPT-3 + GAN

stick109Oct 17, 2020, 7:58 AM
4 points
3 comments1 min readLW link

The Limit of Lan­guage Models

DragonGodJan 6, 2023, 11:53 PM
44 points
26 comments4 min readLW link

Im­ple­ment­ing ac­ti­va­tion steering

AnnahFeb 5, 2024, 5:51 PM
72 points
8 comments7 min readLW link

Ac­tAdd: Steer­ing Lan­guage Models with­out Optimization

Sep 6, 2023, 5:21 PM
105 points
3 comments2 min readLW link
(arxiv.org)

In­stan­ti­at­ing an agent with GPT-4 and text-davinci-003

Max HMar 19, 2023, 11:57 PM
13 points
3 comments32 min readLW link

Graph­i­cal ten­sor no­ta­tion for interpretability

Jordan TaylorOct 4, 2023, 8:04 AM
141 points
11 comments19 min readLW link

New Tool: the Resi­d­ual Stream Viewer

AdamYedidiaOct 1, 2023, 12:49 AM
32 points
7 comments4 min readLW link
(tinyurl.com)

This anime sto­ry­board doesn’t ex­ist: a graphic novel writ­ten and illus­trated by GPT4

RomanSOct 5, 2023, 2:01 PM
12 points
7 comments55 min readLW link

En­tan­gle­ment and in­tu­ition about words and mean­ing

Bill BenzonOct 4, 2023, 2:16 PM
4 points
0 comments2 min readLW link

Thoughts on the im­pli­ca­tions of GPT-3, two years ago and NOW [here be drag­ons, we’re swim­ming, fly­ing and talk­ing with them]

Bill BenzonDec 29, 2022, 8:05 PM
0 points
0 comments5 min readLW link

All GPT skills are translation

p.b.Dec 13, 2020, 8:06 PM
4 points
0 comments2 min readLW link

Rele­vance of ‘Harm­ful In­tel­li­gence’ Data in Train­ing Datasets (We­bText vs. Pile)

MiguelDevOct 12, 2023, 12:08 PM
12 points
0 comments9 min readLW link

Beta test GPT-3 based re­search assistant

jungofthewonDec 16, 2020, 1:42 PM
34 points
2 comments1 min readLW link

Re­quire­ments for a Basin of At­trac­tion to Alignment

RogerDearnaleyFeb 14, 2024, 7:10 AM
41 points
12 comments31 min readLW link

The case for al­ign­ing nar­rowly su­per­hu­man models

Ajeya CotraMar 5, 2021, 10:29 PM
186 points
75 comments38 min readLW link1 review

[Question] Don’t you think RLHF solves outer al­ign­ment?

Charbel-RaphaëlNov 4, 2022, 12:36 AM
9 points
23 comments1 min readLW link

MAKE IT BETTER (a po­etic demon­stra­tion of the ba­nal­ity of GPT-3)

rogersbaconJan 2, 2023, 8:47 PM
7 points
2 comments5 min readLW link

[Question] What will GPT-4 be in­ca­pable of?

Michaël TrazziApr 6, 2021, 7:57 PM
34 points
33 comments1 min readLW link

Dis­cur­sive Com­pe­tence in ChatGPT, Part 1: Talk­ing with Dragons

Bill BenzonJan 5, 2023, 9:01 PM
2 points
0 comments6 min readLW link

How I Learned to Stop Wor­ry­ing and Love MUM

WaddingtonMay 20, 2021, 7:57 AM
2 points
0 comments3 min readLW link

Spec­u­la­tions against GPT-n writ­ing al­ign­ment papers

Donald HobsonJun 7, 2021, 9:13 PM
31 points
6 comments2 min readLW link

What does GPT-3 un­der­stand? Sym­bol ground­ing and Chi­nese rooms

Stuart_ArmstrongAug 3, 2021, 1:14 PM
40 points
15 comments12 min readLW link

AI and the Map of Your Mind: Pat­tern Recog­ni­tion

Scott BroockMar 20, 2023, 5:43 PM
2 points
2 comments6 min readLW link

[Question] 1h-vol­un­teers needed for a small AI Safety-re­lated re­search pro­ject

PabloAMCAug 16, 2021, 5:53 PM
2 points
0 comments1 min readLW link

Fred the Heretic, a GPT for poetry

Bill BenzonDec 8, 2024, 4:52 PM
4 points
0 comments1 min readLW link

ChatGPT tells sto­ries about XP-708-DQ, Eliezer, drag­ons, dark sor­cer­esses, and un­al­igned robots be­com­ing aligned

Bill BenzonJan 8, 2023, 11:21 PM
6 points
2 comments18 min readLW link

The case for more am­bi­tious lan­guage model evals

JozdienJan 30, 2024, 12:01 AM
112 points
30 comments5 min readLW link

ChatGPT (and now GPT4) is very eas­ily dis­tracted from its rules

dmcsMar 15, 2023, 5:55 PM
180 points
42 comments1 min readLW link

[Question] Who owns OpenAI’s new lan­guage model?

ioannesFeb 14, 2019, 5:51 PM
16 points
9 comments1 min readLW link

GPT-4: What we (I) know about it

Robert_AIZIMar 15, 2023, 8:12 PM
40 points
29 comments12 min readLW link
(aizi.substack.com)

Truth­ful AI: Devel­op­ing and gov­ern­ing AI that does not lie

Oct 18, 2021, 6:37 PM
82 points
9 comments10 min readLW link

AMA on Truth­ful AI: Owen Cot­ton-Bar­ratt, Owain Evans & co-authors

Owain_EvansOct 22, 2021, 4:23 PM
31 points
15 comments1 min readLW link

Hegel vs. GPT-3

BezziOct 27, 2021, 5:55 AM
10 points
21 comments2 min readLW link

[Question] What ex­actly is GPT-3′s base ob­jec­tive?

Daniel KokotajloNov 10, 2021, 12:57 AM
60 points
14 comments2 min readLW link

How does GPT-3 spend its 175B pa­ram­e­ters?

Robert_AIZIJan 13, 2023, 7:21 PM
41 points
14 comments6 min readLW link
(aizi.substack.com)

Put­ting mul­ti­modal LLMs to the Tetris test

Feb 1, 2024, 4:02 PM
30 points
5 comments7 min readLW link

Pro­to­type of Us­ing GPT-3 to Gen­er­ate Text­book-length Content

Rafael CosmanJan 18, 2023, 2:25 PM
2 points
8 comments40 min readLW link
(github.com)

Truth­ful LMs as a warm-up for al­igned AGI

Jacob_HiltonJan 17, 2022, 4:49 PM
65 points
14 comments13 min readLW link

How I’m think­ing about GPT-N

delton137Jan 17, 2022, 5:11 PM
54 points
21 comments18 min readLW link

The Gallery for Paint­ing Trans­for­ma­tions—A GPT-3 Analogy

Robert_AIZIJan 19, 2023, 11:32 PM
1 point
0 comments6 min readLW link
(aizi.substack.com)

Un­com­pet­i­tive pro­gram­ming with GPT-3

BezziFeb 6, 2022, 10:19 AM
7 points
8 comments3 min readLW link

How well did Man­i­fold pre­dict GPT-4?

David CheeMar 15, 2023, 11:19 PM
49 points
5 comments2 min readLW link

ChatGPT vs the 2-4-6 Task

cwilluJan 25, 2023, 6:59 AM
20 points
4 comments3 min readLW link

ChatGPT: Tan­tal­iz­ing af­terthoughts in search of story tra­jec­to­ries [in­duc­tion heads]

Bill BenzonFeb 3, 2023, 10:35 AM
4 points
0 comments20 min readLW link

Us­ing GPT-3 for pre­vent­ing con­flict dur­ing mes­sag­ing — a pitch for an app

Eli_Mar 17, 2022, 11:02 AM
22 points
17 comments3 min readLW link

Some mis­cel­la­neous thoughts on ChatGPT, sto­ries, and me­chan­i­cal interpretability

Bill BenzonFeb 4, 2023, 7:35 PM
2 points
0 comments3 min readLW link

[Question] If you lose enough Good Heart To­kens, will you lose real-world money?

YitzApr 1, 2022, 9:11 PM
9 points
0 comments1 min readLW link

Ad­den­dum: More Effi­cient FFNs via Attention

Robert_AIZIFeb 6, 2023, 6:55 PM
10 points
2 comments5 min readLW link
(aizi.substack.com)

Test­ing PaLM prompts on GPT3

YitzApr 6, 2022, 5:21 AM
103 points
14 comments8 min readLW link

Is GPT3 a Good Ra­tion­al­ist? - In­struc­tGPT3 [2/​2]

simeon_cApr 7, 2022, 1:46 PM
11 points
0 comments7 min readLW link

PaLM in “Ex­trap­o­lat­ing GPT-N perfor­mance”

Lukas FinnvedenApr 6, 2022, 1:05 PM
85 points
19 comments2 min readLW link

[Question] What’s ac­tu­ally go­ing on in the “mind” of the model when we fine-tune GPT-3 to In­struc­tGPT?

rpglover64Feb 10, 2023, 7:57 AM
18 points
3 comments1 min readLW link

What is the solu­tion to the Align­ment prob­lem?

AlgonApr 30, 2022, 11:19 PM
24 points
2 comments1 min readLW link

Maybe talk­ing isn’t the best way to com­mu­ni­cate with LLMs

mnvrJan 17, 2024, 6:24 AM
3 points
1 comment1 min readLW link
(mrmr.io)

[Question] Is it a co­in­ci­dence that GPT-3 re­quires roughly the same amount of com­pute as is nec­es­sary to em­u­late the hu­man brain?

RomanSFeb 10, 2023, 4:26 PM
11 points
10 comments1 min readLW link

A pos­si­ble check against mo­ti­vated rea­son­ing us­ing elicit.org

david reinsteinMay 18, 2022, 8:52 PM
3 points
0 comments1 min readLW link

RL with KL penalties is bet­ter seen as Bayesian inference

May 25, 2022, 9:23 AM
114 points
17 comments12 min readLW link

A note on ‘semiotic physics’

metasemiFeb 11, 2023, 5:12 AM
11 points
13 comments6 min readLW link

Who mod­els the mod­els that model mod­els? An ex­plo­ra­tion of GPT-3′s in-con­text model fit­ting ability

LovreJun 7, 2022, 7:37 PM
112 points
16 comments9 min readLW link

GPTs’ abil­ity to keep a se­cret is weirdly prompt-dependent

Jul 22, 2023, 12:21 PM
31 points
0 comments9 min readLW link

In­ves­ti­gat­ing causal un­der­stand­ing in LLMs

Jun 14, 2022, 1:57 PM
28 points
6 comments13 min readLW link

Ex­plain­ing SolidGoldMag­ikarp by look­ing at it from ran­dom directions

Robert_AIZIFeb 14, 2023, 2:54 PM
8 points
0 comments8 min readLW link
(aizi.substack.com)

GPT-3 Catch­ing Fish in Morse Code

Megan KinnimentJun 30, 2022, 9:22 PM
117 points
27 comments8 min readLW link

Nyarlathotep Stirs: A Meta-Nar­ra­tive ChatGPT Story

Charlie SandersMar 20, 2023, 8:00 AM
4 points
2 comments12 min readLW link
(dailymicrofiction.substack.com)

[Question] The OpenAI play­ground for GPT-3 is a ter­rible in­ter­face. Is there any great lo­cal (or web) app for ex­plor­ing/​learn­ing with lan­guage mod­els?

avivAug 13, 2022, 4:34 PM
3 points
1 comment1 min readLW link

Syd­ney the Bin­gena­tor Can’t Think, But It Still Threat­ens People

Valentin BaltadzhievFeb 20, 2023, 6:37 PM
−3 points
2 comments8 min readLW link

What’s the Most Im­pres­sive Thing That GPT-4 Could Plau­si­bly Do?

bayesedAug 26, 2022, 3:34 PM
24 points
22 comments1 min readLW link

GPT-4 is bad at strate­gic thinking

Christopher KingMar 27, 2023, 3:11 PM
22 points
8 comments1 min readLW link

No­body knows how to re­li­ably test for AI safety

marcusarvanMar 27, 2023, 7:48 PM
1 point
0 comments5 min readLW link

OpenAI Credit Ac­count (2510$)

Emirhan BULUTJan 21, 2024, 2:32 AM
1 point
0 comments1 min readLW link

The dreams of GPT-4

RomanSMar 20, 2023, 5:00 PM
14 points
7 comments9 min readLW link

I had a chat with GPT-4 on the fu­ture of AI and AI safety

Kristian FreedMar 28, 2023, 5:47 PM
1 point
0 comments8 min readLW link

[Question] If we have Hu­man-level chat­bots, won’t we end up be­ing ruled by pos­si­ble peo­ple?

Erlja Jkdf.Sep 20, 2022, 1:59 PM
5 points
13 comments1 min readLW link

Inch­ing “Kubla Khan” and GPT into the same in­tel­lec­tual frame­work @ 3 Quarks Daily

Bill BenzonMar 28, 2023, 7:50 PM
5 points
0 comments3 min readLW link

A Hive­mind of GPT-4 bots REALLY IS A HIVEMIND!

Erlja Jkdf.Mar 27, 2023, 12:44 PM
−10 points
1 comment1 min readLW link

An Un­ex­pected GPT-3 De­ci­sion in a Sim­ple Gam­ble

casualphysicsenjoyerSep 25, 2022, 4:46 PM
8 points
4 comments1 min readLW link

Early Re­sults: Do LLMs com­plete false equa­tions with false equa­tions?

Robert_AIZIMar 30, 2023, 8:14 PM
14 points
0 comments4 min readLW link
(aizi.substack.com)

Re­call and Re­gur­gi­ta­tion in GPT2

Megan KinnimentOct 3, 2022, 7:35 PM
43 points
1 comment26 min readLW link

Harry Pot­ter and the Data Cen­ters of Doom

RomanSMar 31, 2023, 10:42 AM
13 points
5 comments4 min readLW link

GPT-4 busted? Clear self-in­ter­est when sum­ma­riz­ing ar­ti­cles about it­self vs when ar­ti­cle talks about Claude, LLaMA, or DALL·E 2

Christopher KingMar 31, 2023, 5:05 PM
6 points
4 comments4 min readLW link

The Peril of the Great Leaks (writ­ten with ChatGPT)

bvbvbvbvbvbvbvbvbvbvbvMar 31, 2023, 6:14 PM
3 points
1 comment1 min readLW link

Imag­ine a world where Microsoft em­ploy­ees used Bing

Christopher KingMar 31, 2023, 6:36 PM
6 points
2 comments2 min readLW link

[Question] Trans­former trained on it’s own con­tent?

MicromegasApr 1, 2023, 3:08 PM
1 point
0 comments1 min readLW link

Mys­ter­ies of mode collapse

janusNov 8, 2022, 10:37 AM
284 points
57 comments14 min readLW link1 review

Bing find­ing ways to by­pass Microsoft’s filters with­out be­ing asked. Is it re­pro­ducible?

Christopher KingFeb 20, 2023, 3:11 PM
27 points
15 comments1 min readLW link

[simu­la­tion] 4chan user claiming to be the at­tor­ney hired by Google’s sen­tient chat­bot LaMDA shares wild de­tails of encounter

janusNov 10, 2022, 9:39 PM
19 points
1 comment13 min readLW link
(generative.ink)

No con­vinc­ing ev­i­dence for gra­di­ent de­scent in ac­ti­va­tion space

BlaineApr 12, 2023, 4:48 AM
85 points
9 comments20 min readLW link

[Question] Is the speed of train­ing large mod­els go­ing to in­crease sig­nifi­cantly in the near fu­ture due to Cere­bras An­dromeda?

Amal Nov 15, 2022, 10:50 PM
13 points
11 comments1 min readLW link

Open-source LLMs may prove Bostrom’s vuln­er­a­ble world hypothesis

Roope AhvenharjuApr 15, 2023, 7:16 PM
1 point
1 comment1 min readLW link

Chronos­ta­sis: The Time-Cap­sule Co­nun­drum of Lan­guage Models

RationalMindsetMar 26, 2023, 6:54 PM
−5 points
0 comments1 min readLW link

[Question] Us­ing ChatGPT for mem­ory re­con­soli­da­tion?

warrenjordanApr 13, 2023, 1:27 AM
3 points
2 comments1 min readLW link

Mechanis­ti­cally in­ter­pret­ing time in GPT-2 small

Apr 16, 2023, 5:57 PM
68 points
6 comments21 min readLW link

By De­fault, GPTs Think In Plain Sight

Fabien RogerNov 19, 2022, 7:15 PM
88 points
36 comments9 min readLW link

Pol­lut­ing the agen­tic commons

hamandcheeseApr 13, 2023, 5:42 PM
7 points
4 comments2 min readLW link
(www.secondbest.ca)

Pre­train­ing Lan­guage Models with Hu­man Preferences

Feb 21, 2023, 5:57 PM
135 points
20 comments11 min readLW link2 reviews

Re­search Re­port: In­cor­rect­ness Cascades

Robert_AIZIApr 14, 2023, 12:49 PM
19 points
0 comments10 min readLW link
(aizi.substack.com)

When will GPT-5 come out? Pre­dic­tion mar­kets vs. Extrapolation

MalteDec 12, 2023, 2:41 AM
12 points
9 comments3 min readLW link

The Soul of the Writer (on LLMs, the psy­chol­ogy of writ­ers, and the na­ture of in­tel­li­gence)

rogersbaconApr 16, 2023, 4:02 PM
11 points
1 comment3 min readLW link
(www.secretorum.life)

An al­ter­na­tive of PPO to­wards alignment

ml hkustApr 17, 2023, 5:58 PM
2 points
2 comments4 min readLW link

Did ChatGPT just gaslight me?

TW123Dec 1, 2022, 5:41 AM
123 points
45 comments9 min readLW link
(aiwatchtower.substack.com)

Read­abil­ity is mostly a waste of characters

vlad.proexApr 21, 2023, 10:05 PM
21 points
7 comments3 min readLW link

We Need To Know About Con­tinual Learning

michael_mjdApr 22, 2023, 5:08 PM
29 points
14 comments4 min readLW link

OpenAI Credit Ac­count (2510$)

Emirhan BULUTJan 21, 2024, 2:30 AM
1 point
0 comments1 min readLW link

Scal­able And Trans­fer­able Black-Box Jailbreaks For Lan­guage Models Via Per­sona Modulation

Nov 7, 2023, 5:59 PM
38 points
2 comments2 min readLW link
(arxiv.org)

LLMs and com­pu­ta­tion complexity

Jonathan MarcusApr 28, 2023, 5:48 PM
57 points
29 comments5 min readLW link

The Misal­ign­ment Para­dox: Ro­bustly Har­ness­ing De­liber­ate Value Diver­gence (Writ­ten by GPT-4)

shl0msApr 28, 2023, 3:29 AM
0 points
0 comments6 min readLW link

[Question] In­ject­ing noise to GPT to get mul­ti­ple answers

bipoloFeb 22, 2023, 8:02 PM
1 point
1 comment1 min readLW link

Feel­ings, Noth­ing More than Feel­ings, About AI

PaulBeconNov 14, 2023, 6:50 PM
7 points
0 comments3 min readLW link

Large Lan­guage Models can Strate­gi­cally De­ceive their Users when Put Un­der Pres­sure.

ReaderMNov 15, 2023, 4:36 PM
89 points
9 comments2 min readLW link1 review
(arxiv.org)

Just How Hard a Prob­lem is Align­ment?

Roger DearnaleyFeb 25, 2023, 9:00 AM
3 points
1 comment21 min readLW link

[Question] GPT-4 Specs: 1 Trillion Pa­ram­e­ters?

infinibot27Mar 26, 2023, 6:56 PM
6 points
8 comments1 min readLW link

Reflec­tion Mechanisms as an Align­ment Tar­get—At­ti­tudes on “near-term” AI

Mar 2, 2023, 4:29 AM
21 points
0 comments8 min readLW link

If it quacks like a duck...

RationalMindsetMar 26, 2023, 6:54 PM
−4 points
0 comments4 min readLW link

Ilya: The AI sci­en­tist shap­ing the world

David VargaNov 20, 2023, 1:09 PM
11 points
0 comments4 min readLW link

More ex­per­i­ments in GPT-4 agency: writ­ing memos

Christopher KingMar 24, 2023, 5:51 PM
5 points
2 comments10 min readLW link

Does GPT-4 ex­hibit agency when sum­ma­riz­ing ar­ti­cles?

Christopher KingMar 24, 2023, 3:49 PM
16 points
2 comments5 min readLW link

The Limi­ta­tions of GPT-4

p.b.Nov 24, 2023, 3:30 PM
27 points
12 comments4 min readLW link

So, just why do GPTs have to op­er­ate by con­tin­u­ing an ex­ist­ing string?

Bill BenzonMar 24, 2023, 12:08 PM
−4 points
0 comments3 min readLW link

Is your job re­place­able by GPT-4? (as of March 2023)

BezziMar 23, 2023, 10:16 PM
18 points
6 comments1 min readLW link

Plan­ning in LLMs: In­sights from AlphaGo

jcoDec 4, 2023, 6:48 PM
8 points
10 comments11 min readLW link

[Question] Is OpenAI los­ing money on each re­quest?

thenoviceoofDec 1, 2023, 3:27 AM
8 points
8 comments5 min readLW link

GPT-4 al­ign­ing with aca­sual de­ci­sion the­ory when in­structed to play games, but in­cludes a CDT ex­pla­na­tion that’s in­cor­rect if they differ

Christopher KingMar 23, 2023, 4:16 PM
7 points
4 comments8 min readLW link

ChatGPT tells sto­ries, and a note about re­verse en­g­ineer­ing: A Work­ing Paper

Bill BenzonMar 3, 2023, 3:12 PM
3 points
0 comments3 min readLW link

Us­ing GPT-4 to Un­der­stand Code

sidMar 24, 2023, 12:09 AM
25 points
2 comments6 min readLW link

The Miss­ing Piece in AI Align­ment: Struc­tured Me­mory and Continuity

Allen MurphyFeb 9, 2025, 3:04 AM
1 point
0 comments2 min readLW link

ChatGPT seems over­con­fi­dent to me

qbolecDec 4, 2022, 8:03 AM
19 points
3 comments16 min readLW link

A Novel Emer­gence of Meta-Aware­ness in LLM Fine-Tuning

rifeJan 15, 2025, 10:59 PM
54 points
31 comments2 min readLW link

Gen­er­at­ing Cog­nate­ful Sen­tences with Large Lan­guage Models

vkethanaJan 6, 2025, 6:40 PM
8 points
0 comments10 min readLW link

In­de­pen­dent re­search ar­ti­cle an­a­lyz­ing con­sis­tent self-re­ports of ex­pe­rience in ChatGPT and Claude

rifeJan 6, 2025, 5:34 PM
4 points
20 comments1 min readLW link
(awakenmoon.ai)

ChatGPT ex­plores the se­man­tic differential

Bill BenzonMar 9, 2023, 1:09 PM
7 points
2 comments7 min readLW link

Stop call­ing it “jailbreak­ing” ChatGPT

TemplarrrMar 10, 2023, 11:41 AM
7 points
9 comments2 min readLW link

Test­ing Ways to By­pass ChatGPT’s Safety Features

Robert_AIZIDec 5, 2022, 6:50 PM
7 points
4 comments5 min readLW link
(aizi.substack.com)

[Question] Why does ChatGPT throw an er­ror when out­putting “David Mayer”?

ArchimedesDec 1, 2024, 12:11 AM
6 points
9 comments1 min readLW link

ChatGPT on Spielberg’s A.I. and AI Alignment

Bill BenzonDec 5, 2022, 9:10 PM
5 points
0 comments4 min readLW link

ChatGPT: “An er­ror oc­curred. If this is­sue per­sists...”

Bill BenzonDec 7, 2022, 3:41 PM
5 points
11 comments3 min readLW link

Of pump­kins, the Fal­con Heavy, and Grou­cho Marx: High-Level dis­course struc­ture in ChatGPT

Bill BenzonDec 8, 2022, 10:25 PM
2 points
0 comments8 min readLW link

The de­fault sce­nario for the next 50 years

JulienNov 24, 2024, 2:01 PM
1 point
0 comments6 min readLW link

Two new datasets for eval­u­at­ing poli­ti­cal syco­phancy in LLMs

alma.liezengaSep 28, 2024, 6:29 PM
8 points
0 comments9 min readLW link

GPT-2 Some­times Fails at IOI

Ronak_MehtaAug 14, 2024, 11:24 PM
13 points
0 comments2 min readLW link
(ronakrm.github.io)

LLMs stifle cre­ativity, elimi­nate op­por­tu­ni­ties for serendipi­tous dis­cov­ery and dis­rupt in­ter­gen­er­a­tional trans­fer of wisdom

GhdzAug 5, 2024, 6:27 PM
6 points
2 comments7 min readLW link

High level dis­course struc­ture in ChatGPT: Part 2 [Quasi-sym­bolic?]

Bill BenzonDec 10, 2022, 10:26 PM
7 points
0 comments6 min readLW link

[Question] What spe­cific dan­gers arise when ask­ing GPT-N to write an Align­ment Fo­rum post?

Matthew BarnettJul 28, 2020, 2:56 AM
45 points
14 comments1 min readLW link

Are AIs like An­i­mals? Per­spec­tives and Strate­gies from Biology

Jackson EmanuelMay 16, 2023, 11:39 PM
1 point
0 comments21 min readLW link

ChatGPT goes through a worm­hole hole in our Shandyesque uni­verse [vir­tual wacky weed]

Bill BenzonDec 11, 2022, 11:59 AM
−1 points
2 comments3 min readLW link

GPT-4

nzMar 14, 2023, 5:02 PM
151 points
150 comments1 min readLW link
(openai.com)

A short cri­tique of Omo­hun­dro’s “Ba­sic AI Drives”

Soumyadeep BoseDec 19, 2024, 7:19 PM
6 points
0 comments4 min readLW link

Fix sim­ple mis­takes in ARC-AGI, etc.

Oleg TrottJul 9, 2024, 5:46 PM
9 points
9 comments1 min readLW link

A brain­teaser for lan­guage models

Adam ScherlisDec 12, 2022, 2:43 AM
47 points
3 comments2 min readLW link

[Question] Is the work on AI al­ign­ment rele­vant to GPT?

Richard_KennawayJul 30, 2020, 12:23 PM
24 points
5 comments1 min readLW link

Agen­tic Lan­guage Model Memes

FactorialCodeAug 1, 2020, 6:03 PM
16 points
1 comment2 min readLW link

Ex­plor­ing the Resi­d­ual Stream of Trans­form­ers for Mechanis­tic In­ter­pretabil­ity — Explained

Zeping YuDec 26, 2023, 12:36 AM
7 points
1 comment11 min readLW link

[Question] What are the most im­por­tant pa­pers/​post/​re­sources to read to un­der­stand more of GPT-3?

adamShimiAug 2, 2020, 8:53 PM
22 points
4 comments1 min readLW link

Is “red” for GPT-4 the same as “red” for you?

Yusuke HayashiMay 6, 2023, 5:55 PM
9 points
6 comments2 min readLW link

LLM cog­ni­tion is prob­a­bly not hu­man-like

Max HMay 8, 2023, 1:22 AM
26 points
15 comments7 min readLW link

Let’s go meta: Gram­mat­i­cal knowl­edge and self-refer­en­tial sen­tences [ChatGPT]

Bill BenzonDec 12, 2022, 9:50 PM
5 points
0 comments9 min readLW link

Lan­guage mod­els can ex­plain neu­rons in lan­guage models

nzMay 9, 2023, 5:29 PM
23 points
0 comments1 min readLW link
(openai.com)

Re­search Re­port: In­cor­rect­ness Cas­cades (Cor­rected)

Robert_AIZIMay 9, 2023, 9:54 PM
9 points
0 comments9 min readLW link
(aizi.substack.com)

[Question] How is GPT-4o Re­lated to GPT-4?

Joel BurgetMay 15, 2024, 6:33 PM
10 points
2 comments1 min readLW link

GPT4 is ca­pa­ble of writ­ing de­cent long-form sci­ence fic­tion (with the right prompts)

RomanSMay 23, 2023, 1:41 PM
22 points
28 comments65 min readLW link

The Com­pleat Cybornaut

May 19, 2023, 8:44 AM
65 points
2 comments16 min readLW link

An ex­plo­ra­tion of GPT-2′s em­bed­ding weights

Adam ScherlisDec 13, 2022, 12:46 AM
44 points
4 comments10 min readLW link

Col­lec­tive Identity

May 18, 2023, 9:00 AM
59 points
12 comments8 min readLW link

Trans­former Ar­chi­tec­ture Choice for Re­sist­ing Prompt In­jec­tion and Jail-Break­ing Attacks

RogerDearnaleyMay 21, 2023, 8:29 AM
9 points
1 comment4 min readLW link

hu­man psy­chol­in­guists: a crit­i­cal appraisal

nostalgebraistDec 31, 2019, 12:20 AM
182 points
59 comments16 min readLW link2 reviews
(nostalgebraist.tumblr.com)

[Question] Bar­cod­ing LLM Train­ing Data Sub­sets. Any­one try­ing this for in­ter­pretabil­ity?

right..enough?Apr 13, 2024, 3:09 AM
7 points
0 comments7 min readLW link
No comments.