Information Theory

Tag

Entropy, and Short Codes

Eliezer Yudkowsky23 Feb 2008 3:16 UTC

82 points

27 comments4 min readLW link

Conditional Independence, and Naive Bayes

Eliezer Yudkowsky1 Mar 2008 1:59 UTC

72 points

16 comments6 min readLW link

[Question] Newbie questions about information theory and transformers

Misaligned-Semi-intelligence21 Jun 2023 22:45 UTC

10 points

1 comment1 min readLW link

The Second Law of Thermodynamics, and Engines of Cognition

Eliezer Yudkowsky27 Feb 2008 0:48 UTC

190 points

76 comments9 min readLW link

[Question] How does one recognize information and differentiate it from noise?

M. Y. Zuo3 Aug 2022 3:57 UTC

4 points

29 comments1 min readLW link

Mutual Information, and Density in Thingspace

Eliezer Yudkowsky23 Feb 2008 19:14 UTC

69 points

28 comments7 min readLW link

Six (and a half) intuitions for KL divergence

CallumMcDougall12 Oct 2022 21:07 UTC

162 points

25 comments10 min readLW link 1 review

(www.perfectlynormal.co.uk)

Searching for Bayes-Structure

Eliezer Yudkowsky28 Feb 2008 22:01 UTC

63 points

49 comments5 min readLW link

Message Length

Zack_M_Davis20 Oct 2020 5:52 UTC

134 points

25 comments12 min readLW link

Introduction to abstract entropy

Alex_Altair20 Oct 2022 21:03 UTC

234 points

78 comments18 min readLW link 1 review

[Question] Where can one learn deep intuitions about information theory?

Valentine16 Dec 2021 15:47 UTC

67 points

27 comments2 min readLW link

The Telephone Theorem: Information At A Distance Is Mediated By Deterministic Constraints

johnswentworth31 Aug 2021 16:50 UTC

96 points

26 comments9 min readLW link 2 reviews

Utility Maximization = Description Length Minimization

johnswentworth18 Feb 2021 18:04 UTC

213 points

44 comments5 min readLW link

Using Information Theory to tackle AI Alignment: A Practical Approach

Daniel Salami17 Dec 2022 1:37 UTC

10 points

4 comments7 min readLW link

Generalized Heat Engine

johnswentworth5 Nov 2020 19:01 UTC

186 points

11 comments7 min readLW link

[Retracted] Newton’s law of cooling from first principles

Nisan16 Jan 2024 4:21 UTC

9 points

15 comments2 min readLW link

You Can’t Predict a Game of Pinball

Jeffrey Heninger30 Mar 2023 0:40 UTC

61 points

12 comments6 min readLW link

(aiimpacts.org)

Msg Len

Zack_M_Davis12 Oct 2020 3:35 UTC

65 points

4 comments1 min readLW link

Information Assets

ozziegooen24 Aug 2021 4:32 UTC

42 points

3 comments9 min readLW link

Understanding the Merging of Opinions with Increasing Information theorem

ViktoriaMalyasova21 Apr 2022 14:13 UTC

16 points

1 comment5 min readLW link

Eavesdropping on Aliens: A Data Decoding Challenge

anonymousaisafety24 Jul 2022 4:35 UTC

44 points

9 comments4 min readLW link

On some first principles of intelligence

Macheng_Shen17 Mar 2023 17:42 UTC

−14 points

0 comments4 min readLW link

Combining causality with algorithmic information theory

[deleted]9 Jun 2012 1:31 UTC

21 points

3 comments1 min readLW link

Speculations on information under logical uncertainty

TsviBT24 Feb 2016 21:58 UTC

1 point

0 comments12 min readLW link

My (Mis)Adventures With Algorithmic Machine Learning

AHartNtkn20 Sep 2020 5:31 UTC

16 points

4 comments41 min readLW link

Information theory and the symmetry of updating beliefs

Academian20 Mar 2010 0:34 UTC

65 points

29 comments8 min readLW link

Kolmogorov’s theory of Algorithmic Probability

Aidan Rocke3 Aug 2023 0:58 UTC

5 points

2 comments2 min readLW link

(keplerlounge.com)

Blind artifacts

So8res7 Apr 2015 23:23 UTC

36 points

2 comments2 min readLW link

Speculations on Sleep in an Age of Machines

cleanwhiteroom29 Aug 2024 13:42 UTC

2 points

0 comments6 min readLW link

Towards Measures of Optimisation

mattmacdermott and Alexander Gietelink Oldenziel

12 May 2023 15:29 UTC

53 points

37 comments4 min readLW link

Knowledge Base 8: The truth as an attractor in the information space

iwis25 Apr 2024 15:28 UTC

−8 points

0 comments2 min readLW link

Erdős Problems in Algorithmic Probability

Aidan Rocke11 Sep 2023 16:44 UTC

13 points

4 comments2 min readLW link

Revisiting the Manifold Hypothesis

Aidan Rocke1 Oct 2023 23:55 UTC

13 points

19 comments4 min readLW link

Information-Theoretic Boxing of Superintelligences

JustinShovelain and Elliot Mckernon

30 Nov 2023 14:31 UTC

30 points

0 comments7 min readLW link

[Question] Are the Born probabilities really that mysterious?

Adele Lopez2 Mar 2021 3:08 UTC

45 points

14 comments1 min readLW link

Our compressed perception

pchvykov6 Apr 2021 11:01 UTC

12 points

0 comments2 min readLW link

Mindfulness as debugging

pchvykov30 Apr 2021 16:59 UTC

21 points

4 comments2 min readLW link

Does butterfly affect?

pchvykov14 May 2021 4:20 UTC

14 points

21 comments2 min readLW link

The entropy maxim for binary questions

dkl911 Feb 2024 17:17 UTC

2 points

1 comment1 min readLW link

(dkl9.net)

The Blackwell order as a formalization of knowledge

Alex Flint10 Sep 2021 2:51 UTC

41 points

10 comments11 min readLW link

Attachment THEORY AND THE EFFECTS OF SECURE ATTACHMENT ON CHILD DEVELOPMENT

Mihriban Temel8 Sep 2024 16:09 UTC

−8 points

0 comments9 min readLW link

Entropy isn’t sufficient to measure password strength

benwr17 Jan 2022 6:41 UTC

36 points

30 comments2 min readLW link

(www.benwr.net)

Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs

Kola Ayonrinde, Michael Pearce and Lee Sharkey

23 Aug 2024 18:52 UTC

39 points

5 comments16 min readLW link

Deconfusing Landauer’s Principle

EuanMcLean27 May 2022 17:58 UTC

58 points

13 comments15 min readLW link

A physicist’s approach to Origins of Life

pchvykov28 Jun 2022 15:23 UTC

12 points

6 comments16 min readLW link

What Is The True Name of Modularity?

CallumMcDougall, Lucius Bushnaq and Avery

1 Jul 2022 14:55 UTC

39 points

10 comments12 min readLW link

Information theoretic model analysis may not lend much insight, but we may have been doing them wrong!

Garrett Baker24 Jul 2022 0:42 UTC

7 points

0 comments10 min readLW link

LMSR subsidy parameter is the price of information

Abhimanyu Pallavi Sudhir25 May 2024 18:05 UTC

5 points

0 comments1 min readLW link

Steganography in Chain of Thought Reasoning

A Ray8 Aug 2022 3:47 UTC

61 points

13 comments6 min readLW link

[Incomplete] What is Computation Anyway?

DragonGod14 Dec 2022 16:17 UTC

16 points

1 comment13 min readLW link

(arxiv.org)

Word-Distance vs Idea-Distance: The Case for Lanoitaring

Sable6 Nov 2022 5:25 UTC

7 points

7 comments7 min readLW link

(affablyevil.substack.com)

The harnessing of complexity

geduardo10 Nov 2022 18:44 UTC

6 points

2 comments3 min readLW link

Knowledge Base 7: Long-tail knowledge and collective intelligence

iwis18 Apr 2024 14:21 UTC

−6 points

0 comments1 min readLW link

A hundredth of a bit of extra entropy

Adam Scherlis24 Dec 2022 21:12 UTC

83 points

4 comments3 min readLW link

“Inftoxicity” and other new words to describe malicious information and communication thereof

Jáchym Fibír23 Dec 2023 18:15 UTC

−1 points

6 comments3 min readLW link

How to Throw Away Information

johnswentworth5 Sep 2019 21:10 UTC

18 points

5 comments5 min readLW link

Law without law: from observer states to physics via algorithmic information theory

avturchin28 Sep 2018 10:07 UTC

16 points

6 comments2 min readLW link

(arxiv.org)

A quick sketch on how the Curry-Howard Isomorphism kinda appears to connect Algorithmic Information Theory with ordinal logics

[deleted]19 Apr 2015 19:35 UTC

26 points

27 comments3 min readLW link

An Undergraduate Reading Of: Semantic information, autonomous agency and non-equilibrium statistical physics

ryan_b30 Oct 2018 18:36 UTC

29 points

2 comments5 min readLW link

[LINK] The Bayesian Second Law of Thermodynamics

Shmi12 Aug 2015 16:52 UTC

13 points

16 comments1 min readLW link

No comments.

In­for­ma­tion Theory

Information Theory