The 27 papers

[WitheringWeights]

List of 27 papers (supposedly) given to John Carmack by Ilya Sutskever: “If you really learn all of these, you’ll know 90% of what matters today.”
The list has been floating around for a few weeks on Twitter/​LinkedIn. I figure some might have missed it so here you go.
Regardless of the veracity of the tale I am still finding it valuable.

https://​​punkx.org/​​jackdoe/​​30.html

1. The Annotated Transformer (nlp.seas.harvard.edu)

2. The First Law of Complexodynamics (scottaaronson.blog)

3. The Unreasonable Effectiveness of RNNs (karpathy.github.io)

4. Understanding LSTM Networks (colah.github.io)

5. Recurrent Neural Network Regularization (arxiv.org)

6. Keeping Neural Networks Simple by Minimizing the Description Length of the Weights (cs.toronto.edu)

7. Pointer Networks (arxiv.org)

8. ImageNet Classification with Deep CNNs (proceedings.neurips.cc)

9. Order Matters: Sequence to sequence for sets (arxiv.org)

10. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (arxiv.org)

11. Deep Residual Learning for Image Recognition (arxiv.org)

12. Multi-Scale Context Aggregation by Dilated Convolutions (arxiv.org)

13. Neural Quantum Chemistry (arxiv.org)

14. Attention Is All You Need (arxiv.org)

15. Neural Machine Translation by Jointly Learning to Align and Translate (arxiv.org)

16. Identity Mappings in Deep Residual Networks (arxiv.org)

17. A Simple NN Module for Relational Reasoning (arxiv.org)

18. Variational Lossy Autoencoder (arxiv.org)

19. Relational RNNs (arxiv.org)

20. Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton (arxiv.org)

21. Neural Turing Machines (arxiv.org)

22. Deep Speech 2: End-to-End Speech Recognition in English and Mandarin (arxiv.org)

23. Scaling Laws for Neural LMs (arxiv.org)

24. A Tutorial Introduction to the Minimum Description Length Principle (arxiv.org)

25. Machine Super Intelligence Dissertation (vetta.org)

26. PAGE 434 onwards: Komogrov Complexity (lirmm.fr)

27. CS231n Convolutional Neural Networks for Visual Recognition (cs231n.github.io)

[Amalthea]

Might be good to estimate the date of the recommendation—as the interview where Carmack mentioned this was in 2023, a rough guess might be 2021/​22?

[metachirality]

I like this format and framing of “90% of what matters” and someone should try doing it with other subjects.