RSS

Sin­gu­lar Learn­ing Theory

TagLast edit: 20 Jun 2023 23:25 UTC by DanielFilan

Singluar learning theory is a theory that applies algebraic geometry to statistical learning theory, developed by Sumio Watanabe. Reference textbooks are “the grey book”, Algebraic Geometry and Statistical Learning Theory, and “the green book”, Mathematical Theory of Bayesian Statistics.

DSLT 2. Why Neu­ral Net­works obey Oc­cam’s Razor

Liam Carroll18 Jun 2023 0:23 UTC
22 points
14 comments17 min readLW link

DSLT 3. Neu­ral Net­works are Singular

Liam Carroll20 Jun 2023 8:20 UTC
29 points
5 comments19 min readLW link

DSLT 1. The RLCT Mea­sures the Effec­tive Di­men­sion of Neu­ral Networks

Liam Carroll16 Jun 2023 9:50 UTC
52 points
10 comments13 min readLW link

DSLT 0. Distill­ing Sin­gu­lar Learn­ing Theory

Liam Carroll16 Jun 2023 9:50 UTC
77 points
7 comments5 min readLW link

Neu­ral net­works gen­er­al­ize be­cause of this one weird trick

Jesse Hoogland18 Jan 2023 0:10 UTC
179 points
34 comments53 min readLW link1 review
(www.jessehoogland.com)

Sin­gu­lar learn­ing the­ory: exercises

Zach Furman30 Aug 2024 20:00 UTC
90 points
5 comments14 min readLW link

In­ves­ti­gat­ing the learn­ing co­effi­cient of mod­u­lar ad­di­tion: hackathon project

17 Oct 2023 19:51 UTC
94 points
5 comments12 min readLW link

Growth and Form in a Toy Model of Superposition

8 Nov 2023 11:08 UTC
89 points
7 comments14 min readLW link

An­nounc­ing Timaeus

22 Oct 2023 11:59 UTC
188 points
15 comments4 min readLW link

Spooky ac­tion at a dis­tance in the loss landscape

28 Jan 2023 0:22 UTC
61 points
4 comments7 min readLW link
(www.jessehoogland.com)

Ti­maeus’s First Four Months

28 Feb 2024 17:01 UTC
172 points
6 comments6 min readLW link

DSLT 4. Phase Tran­si­tions in Neu­ral Networks

Liam Carroll24 Jun 2023 17:22 UTC
30 points
3 comments16 min readLW link

Gra­di­ent sur­fing: the hid­den role of regularization

Jesse Hoogland6 Feb 2023 3:50 UTC
37 points
9 comments14 min readLW link
(www.jessehoogland.com)

Epoch wise crit­i­cal pe­ri­ods, and sin­gu­lar learn­ing theory

Garrett Baker14 Dec 2023 20:55 UTC
9 points
1 comment5 min readLW link

Stage­wise Devel­op­ment in Neu­ral Networks

20 Mar 2024 19:54 UTC
96 points
1 comment11 min readLW link

Sim­ple ver­sus Short: Higher-or­der de­gen­er­acy and er­ror-correction

Daniel Murfet11 Mar 2024 7:52 UTC
109 points
8 comments13 min readLW link

AXRP Epi­sode 31 - Sin­gu­lar Learn­ing The­ory with Daniel Murfet

DanielFilan7 May 2024 3:50 UTC
72 points
4 comments71 min readLW link

Dialogue in­tro­duc­tion to Sin­gu­lar Learn­ing Theory

Olli Järviniemi8 Jul 2024 16:58 UTC
100 points
15 comments8 min readLW link

Ti­maeus is hiring!

12 Jul 2024 23:42 UTC
67 points
6 comments2 min readLW link

AXRP Epi­sode 38.2 - Jesse Hoogland on Sin­gu­lar Learn­ing Theory

DanielFilan27 Nov 2024 6:30 UTC
34 points
0 comments10 min readLW link

Deep Learn­ing is cheap Solomonoff in­duc­tion?

7 Dec 2024 11:00 UTC
44 points
1 comment17 min readLW link

The gen­er­al­iza­tion phase diagram

Dmitry Vaintrob26 Jan 2025 20:30 UTC
26 points
2 comments16 min readLW link

Proof idea: SLT to AIT

Lucius Bushnaq10 Feb 2025 23:14 UTC
39 points
13 comments6 min readLW link

Gen­er­al­iza­tion, from ther­mo­dy­nam­ics to statis­ti­cal physics

Jesse Hoogland30 Nov 2023 21:28 UTC
64 points
9 comments28 min readLW link

Towards Devel­op­men­tal Interpretability

12 Jul 2023 19:33 UTC
192 points
10 comments9 min readLW link1 review

Ap­ply for the 2023 Devel­op­men­tal In­ter­pretabil­ity Con­fer­ence!

25 Aug 2023 7:12 UTC
33 points
0 comments2 min readLW link

You’re Mea­sur­ing Model Com­plex­ity Wrong

11 Oct 2023 11:46 UTC
92 points
17 comments13 min readLW link

[Question] A few Align­ment ques­tions: util­ity op­ti­miz­ers, SLT, sharp left turn and identifiability

Igor Timofeev26 Sep 2023 0:27 UTC
6 points
1 comment2 min readLW link

My hopes for al­ign­ment: Sin­gu­lar learn­ing the­ory and whole brain emulation

Garrett Baker25 Oct 2023 18:31 UTC
61 points
5 comments12 min readLW link

Open Call for Re­search As­sis­tants in Devel­op­men­tal Interpretability

30 Aug 2023 9:02 UTC
55 points
11 comments4 min readLW link

Sin­gu­lar learn­ing the­ory and bridg­ing from ML to brain emulations

1 Nov 2023 21:31 UTC
26 points
16 comments29 min readLW link

Es­ti­mat­ing effec­tive di­men­sion­al­ity of MNIST models

Arjun Panickssery2 Nov 2023 14:13 UTC
41 points
3 comments1 min readLW link

De­gen­era­cies are sticky for SGD

16 Jun 2024 21:19 UTC
56 points
1 comment16 min readLW link

A short ‘deriva­tion’ of Watan­abe’s Free En­ergy Formula

Wuschel Schulz29 Jan 2024 23:41 UTC
13 points
6 comments7 min readLW link

Learn­ing co­effi­cient es­ti­ma­tion: the details

Zach Furman16 Nov 2023 3:19 UTC
36 points
0 comments2 min readLW link
(colab.research.google.com)

My Crit­i­cism of Sin­gu­lar Learn­ing Theory

Joar Skalse19 Nov 2023 15:19 UTC
83 points
56 comments12 min readLW link

In­ter­view Daniel Mur­fet on Univer­sal Phenom­ena in Learn­ing Machines

Alexander Gietelink Oldenziel6 Feb 2023 0:00 UTC
47 points
1 comment16 min readLW link

rough draft on what hap­pens in the brain when you have an insight

Emrik21 May 2024 18:02 UTC
11 points
2 comments1 min readLW link

Sin­gu­lar Learn­ing The­ory for Dummies

Rahul Chand15 Oct 2024 21:13 UTC
1 point
0 comments8 min readLW link

My im­pres­sion of sin­gu­lar learn­ing theory

Ege Erdil18 Jun 2023 15:34 UTC
47 points
30 comments2 min readLW link

Sin­gu­lar­i­ties against the Sin­gu­lar­ity: An­nounc­ing Work­shop on Sin­gu­lar Learn­ing The­ory and Alignment

1 Apr 2023 9:58 UTC
87 points
0 comments1 min readLW link
(singularlearningtheory.com)

Jesse Hoogland on Devel­op­men­tal In­ter­pretabil­ity and Sin­gu­lar Learn­ing Theory

Michaël Trazzi6 Jul 2023 15:46 UTC
42 points
2 comments4 min readLW link
(theinsideview.ai)

Es­ti­mat­ing the Prob­a­bil­ity of Sam­pling a Trained Neu­ral Net­work at Random

1 Mar 2025 2:11 UTC
30 points
7 comments1 min readLW link
(arxiv.org)

The Hes­sian rank bounds the learn­ing coefficient

Lucius Bushnaq8 Aug 2024 20:55 UTC
68 points
9 comments4 min readLW link

Fea­ture Tar­geted LLC Es­ti­ma­tion Dist­in­guishes SAE Fea­tures from Ran­dom Directions

19 Jul 2024 20:32 UTC
59 points
6 comments16 min readLW link
No comments.