RSS

Sin­gu­lar Learn­ing Theory

TagLast edit: 20 Jun 2023 23:25 UTC by DanielFilan

Singluar learning theory is a theory that applies algebraic geometry to statistical learning theory, developed by Sumio Watanabe. Reference textbooks are “the grey book”, Algebraic Geometry and Statistical Learning Theory, and “the green book”, Mathematical Theory of Bayesian Statistics.

Neu­ral net­works gen­er­al­ize be­cause of this one weird trick

Jesse Hoogland18 Jan 2023 0:10 UTC
171 points
28 comments53 min readLW link
(www.jessehoogland.com)

DSLT 1. The RLCT Mea­sures the Effec­tive Di­men­sion of Neu­ral Networks

Liam Carroll16 Jun 2023 9:50 UTC
51 points
9 comments13 min readLW link

DSLT 0. Distill­ing Sin­gu­lar Learn­ing Theory

Liam Carroll16 Jun 2023 9:50 UTC
76 points
6 comments5 min readLW link

Sin­gu­lar learn­ing the­ory: exercises

Zach Furman30 Aug 2024 20:00 UTC
88 points
5 comments14 min readLW link

DSLT 2. Why Neu­ral Net­works obey Oc­cam’s Razor

Liam Carroll18 Jun 2023 0:23 UTC
22 points
14 comments17 min readLW link

DSLT 3. Neu­ral Net­works are Singular

Liam Carroll20 Jun 2023 8:20 UTC
27 points
5 comments19 min readLW link

DSLT 4. Phase Tran­si­tions in Neu­ral Networks

Liam Carroll24 Jun 2023 17:22 UTC
30 points
3 comments16 min readLW link

Spooky ac­tion at a dis­tance in the loss landscape

28 Jan 2023 0:22 UTC
61 points
4 comments7 min readLW link
(www.jessehoogland.com)

In­ves­ti­gat­ing the learn­ing co­effi­cient of mod­u­lar ad­di­tion: hackathon project

17 Oct 2023 19:51 UTC
94 points
5 comments12 min readLW link

Growth and Form in a Toy Model of Superposition

8 Nov 2023 11:08 UTC
87 points
7 comments14 min readLW link

Gra­di­ent sur­fing: the hid­den role of regularization

Jesse Hoogland6 Feb 2023 3:50 UTC
37 points
9 comments14 min readLW link
(www.jessehoogland.com)

An­nounc­ing Timaeus

22 Oct 2023 11:59 UTC
187 points
15 comments4 min readLW link

Ti­maeus’s First Four Months

28 Feb 2024 17:01 UTC
172 points
6 comments6 min readLW link

Epoch wise crit­i­cal pe­ri­ods, and sin­gu­lar learn­ing theory

Garrett Baker14 Dec 2023 20:55 UTC
9 points
1 comment5 min readLW link

Stage­wise Devel­op­ment in Neu­ral Networks

20 Mar 2024 19:54 UTC
89 points
1 comment11 min readLW link

Sim­ple ver­sus Short: Higher-or­der de­gen­er­acy and er­ror-correction

Daniel Murfet11 Mar 2024 7:52 UTC
111 points
6 comments13 min readLW link

AXRP Epi­sode 31 - Sin­gu­lar Learn­ing The­ory with Daniel Murfet

DanielFilan7 May 2024 3:50 UTC
72 points
4 comments71 min readLW link

Dialogue in­tro­duc­tion to Sin­gu­lar Learn­ing Theory

Olli Järviniemi8 Jul 2024 16:58 UTC
97 points
14 comments8 min readLW link

Ti­maeus is hiring!

12 Jul 2024 23:42 UTC
67 points
6 comments2 min readLW link

Gen­er­al­iza­tion, from ther­mo­dy­nam­ics to statis­ti­cal physics

Jesse Hoogland30 Nov 2023 21:28 UTC
63 points
9 comments28 min readLW link

Towards Devel­op­men­tal Interpretability

12 Jul 2023 19:33 UTC
180 points
9 comments9 min readLW link

Ap­ply for the 2023 Devel­op­men­tal In­ter­pretabil­ity Con­fer­ence!

25 Aug 2023 7:12 UTC
33 points
0 comments2 min readLW link

You’re Mea­sur­ing Model Com­plex­ity Wrong

11 Oct 2023 11:46 UTC
87 points
15 comments13 min readLW link

[Question] A few Align­ment ques­tions: util­ity op­ti­miz­ers, SLT, sharp left turn and identifiability

Igor Timofeev26 Sep 2023 0:27 UTC
6 points
1 comment2 min readLW link

Learn­ing co­effi­cient es­ti­ma­tion: the details

Zach Furman16 Nov 2023 3:19 UTC
36 points
0 comments2 min readLW link
(colab.research.google.com)

My Crit­i­cism of Sin­gu­lar Learn­ing Theory

Joar Skalse19 Nov 2023 15:19 UTC
82 points
56 comments12 min readLW link

Fea­ture Tar­geted LLC Es­ti­ma­tion Dist­in­guishes SAE Fea­tures from Ran­dom Directions

19 Jul 2024 20:32 UTC
59 points
6 comments16 min readLW link

De­gen­era­cies are sticky for SGD

16 Jun 2024 21:19 UTC
56 points
1 comment16 min readLW link

In­ter­view Daniel Mur­fet on Univer­sal Phenom­ena in Learn­ing Machines

Alexander Gietelink Oldenziel6 Feb 2023 0:00 UTC
47 points
1 comment16 min readLW link

A short ‘deriva­tion’ of Watan­abe’s Free En­ergy Formula

Wuschel Schulz29 Jan 2024 23:41 UTC
13 points
6 comments7 min readLW link

My im­pres­sion of sin­gu­lar learn­ing theory

Ege Erdil18 Jun 2023 15:34 UTC
47 points
30 comments2 min readLW link

Sin­gu­lar­i­ties against the Sin­gu­lar­ity: An­nounc­ing Work­shop on Sin­gu­lar Learn­ing The­ory and Alignment

1 Apr 2023 9:58 UTC
87 points
0 comments1 min readLW link
(singularlearningtheory.com)

Jesse Hoogland on Devel­op­men­tal In­ter­pretabil­ity and Sin­gu­lar Learn­ing Theory

Michaël Trazzi6 Jul 2023 15:46 UTC
42 points
2 comments4 min readLW link
(theinsideview.ai)

rough draft on what hap­pens in the brain when you have an insight

Emrik21 May 2024 18:02 UTC
11 points
2 comments1 min readLW link

My hopes for al­ign­ment: Sin­gu­lar learn­ing the­ory and whole brain emulation

Garrett Baker25 Oct 2023 18:31 UTC
61 points
5 comments12 min readLW link

Open Call for Re­search As­sis­tants in Devel­op­men­tal Interpretability

30 Aug 2023 9:02 UTC
55 points
11 comments4 min readLW link

Sin­gu­lar learn­ing the­ory and bridg­ing from ML to brain emulations

1 Nov 2023 21:31 UTC
26 points
16 comments29 min readLW link

Es­ti­mat­ing effec­tive di­men­sion­al­ity of MNIST models

Arjun Panickssery2 Nov 2023 14:13 UTC
41 points
3 comments1 min readLW link

Sin­gu­lar Learn­ing The­ory for Dummies

Rahul Chand15 Oct 2024 21:13 UTC
2 points
0 comments8 min readLW link

The Hes­sian rank bounds the learn­ing coefficient

Lucius Bushnaq8 Aug 2024 20:55 UTC
68 points
9 comments4 min readLW link
No comments.