Meta-theory of rationality

Feb 20, 2025, 6:17 PM

Here I speculate about questions such as:

What makes a theory of rationality useful or useless?

When is a theory of rationality useful for building agents, describing agents, or becoming a better agent, and to what extent should the answers be connected?

How elegant should we expect algorithms for intelligence to be?

What concepts deserve to be promoted to the root/core design of an AGI versus discovered by AGI? Perhaps relatedly, does human cognition have such a root/core algorithm, and if so, what is it?

Levels of analysis for thinking about agency

Cole WyethFeb 26, 2025, 4:24 AM

11 points

0 comments7 min readLW link

Action theory is not policy theory is not agent theory

Cole WyethSep 5, 2023, 1:38 AM

19 points

4 comments6 min readLW link

(colewyeth.com)

What makes a theory of intelligence useful?

Cole WyethFeb 20, 2025, 7:22 PM

14 points

0 comments11 min readLW link

Existing UDTs test the limits of Bayesianism (and consistency)

Cole WyethMar 12, 2025, 4:09 AM

23 points

10 comments7 min readLW link

Keyboard shortcuts

Keys shown in yellow (e.g., ]) are accesskeys, and require a browser-specific modifier key (or keys).

Keys shown in grey (e.g., ?) do not require any modifier keys.

General
? Show keyboard shortcuts
Esc Hide keyboard shortcuts

Site navigation
h Go to Home (a.k.a. “Frontpage”) view
f Go to Featured (a.k.a. “Curated”) view
a Go to All (a.k.a. “Community”) view
m Go to Meta view
v Go to Tags view
c Go to Recent Comments view
r Go to Archive view
q Go to Sequences view
t Go to About page
u Go to User or Login page
o Go to Inbox page

Page navigation
, Jump up to top of page
. Jump down to bottom of page
/ Jump to top of comments section
s Search

Page actions
n New post or comment
e Edit current post

Post/comment list views
. Focus next entry in list
, Focus previous entry in list
; Cycle between links in focused entry
Enter Go to currently focused entry
Esc Unfocus currently focused entry
] Go to next page
[ Go to previous page
\ Go to first page
e Edit currently focused post

Editor
k Bold text
i Italic text
l Insert hyperlink
q Blockquote text

Appearance
= Increase text size
- Decrease text size
0 Reset to default text size
′ Cycle through content width settings
1 Switch to default theme [A]
2 Switch to dark theme [B]
3 Switch to grey theme [C]
4 Switch to ultramodern theme [D]
5 Switch to simple theme [E]
6 Switch to brutalist theme [F]
7 Switch to ReadTheSequences theme [G]
8 Switch to classic Less Wrong theme [H]
9 Switch to modern Less Wrong theme [I]
; Open theme tweaker
Enter Save changes and close theme tweaker
Esc Close theme tweaker (without saving)

Slide shows
l Start/resume slideshow
Esc Exit slideshow
→↓ Next slide
←↑ Previous slide
Space Reset slide zoom

Miscellaneous
x Switch to next view on user page
z Switch to previous view on user page
` Toggle compact comment list view
g Toggle anti-kibitzer

Meta-theory of rationality

Levels of anal­y­sis for think­ing about agency

Ac­tion the­ory is not policy the­ory is not agent theory

What makes a the­ory of in­tel­li­gence use­ful?

Ex­ist­ing UDTs test the limits of Bayesi­anism (and con­sis­tency)

Levels of analysis for thinking about agency

Action theory is not policy theory is not agent theory

What makes a theory of intelligence useful?

Existing UDTs test the limits of Bayesianism (and consistency)