This post was funded by a grant from Ben Pace.

Ben Pace asks:

To what extent have ideas and scientific discoveries gotten harder to find?
Related: Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.
Some theories have been put forward by Scott Alexander and Holden Karnofsky. Maybe they’re right, maybe not.

Scott Alexander treats scientific ideas as nonrenewable.

Imagine scientists venturing off in some research direction. At the dawn of history, they don’t need to venture very far before discovering a new truth. As time goes on, they need to go further and further.

Holden Karnofsky believes the pattern applies to both art and science.

The broad theme is that across a variety of areas in both art and science, we see a form of “innovation stagnation”: the best-regarded figures are disproportionately from long ago, and our era seems to “punch below its weight” when considering the rise in population, education, etc. Since the patterns look fairly similar for art and science, and both are forms of innovation, I think it’s worth thinking about potential common factors.

I have a different perspective.

Science

Physics

The most important scientific discoveries are those which are the most general and the most useful. Physics is the most general of sciences. Physics is basically solved.

There are unsolved problems in physics. Dark matter remains a mystery. Quantum mechanics has yet to be unified with general relativity. But the holes in physics don’t matter. You don’t need a Grand Unified Theory of quantum relativity to build a Mars base or a fusion reactor. All you need is today’s physics plus a whole lot of engineering.

The recent discoveries in physics like quantum computing and the photograph of a black hole aren’t really discoveries about the fundamental Laws of Physics. They’re technological achievements.

All of the rest of science is just applied physics too. One could argue that biology, chemistry and so on are just footnotes to Einstein. The fundamental laws of the universe are (for all practical purposes) known. The remaining questions are:

Astrophysics i.e. the study of places that don’t matter because we lack the technology to go there.
What has biology built out of matter?
What can we build out of matter?

No physicist will ever again make a discovery (in physics) as impactful as the great 20th century physicists. All the important fruit has been picked.

But that doesn’t mean science has been exhausted. It just means physics has been exhausted. Biology is advancing fast. I’ll never get tired of this graph about how, since 2007, biotechnology has advanced faster than computer technology ever did.

As recently as 2015, Nick Lane published a book that might have solved the origin of life. And biology isn’t even the most exciting frontier.

Machine Learning

In Contra Hoel, I talked about machine learning as feeling different from some other scientific fields: there are frequent exciting new discoveries. This shouldn’t be surprising. Physics is stagnant because Newton and Einstein already got all the cool results. But Newton and Einstein didn’t have TPUs so they couldn’t discover things about machine learning.
―The Low-Hanging Fruit Argument: Models And Predictions by Scott Alexander

Do discoveries in machine learning count as science or technology? If we use the strictest definition of “science” then machine learning counts as “technology”. But machine learning is also informing our understanding of the human mind. Psychology definitely counts as “science”. “Untangling how intelligence works” is the most important scientific problem of our age.

Ambitious people go to where the low-hanging fruit is. I think it would be a mistake for an ambitious young person to study physics when machine learning has so many more new developments. (I say this as someone who did study physics.)

Solving the problem of AGI would eclipse everything else humanity has ever achieved. We are making faster progress toward AGI than at any point in history. So why does it feel like the low-hanging fruit has already been picked?

Because the fruit that has been picked is public knowledge but the fruit that hasn’t been picked is not public knowledge. Unless you are an expert on the vanguard of a pioneering field, you cannot see the unpicked fruit.

I think the natural universe is like a game of chess. We have figured out the rules of our universe (like $\to F = m \to a$ and $E = m c^{2}$ ). But that was just the beginning. Now our challenge is figuring out how to play the game.

It is as if our civilization has just invented writing. (Which, on a cosmological scale, it has.) That was the boring part. Now is the time to write.

Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.

The greatest pure scientific discoveries have been exhausted. Many technologies are still invented by people doing things for fun. Steve Wozniak built his first computer for fun. Linus Torvolds, the inventor of Linux, literally titled his autobiography “Just for Fun”.

Creative Ideas are Plentiful

I have lots of ideas. How many ideas? I recently posted a list of some of my ideas for blog posts. The number of ideas I have vastly outnumbers the number of ideas I will ever find the time to write about. This isn’t because I don’t have enough time to write. It’s because writing generates ideas faster than I can write about them. I could write for a hundred lifetimes and never run out of things to say.

The phenomenon of “doing things gives you ideas for things to do” isn’t limited to writing. It happens in business too. Starting companies takes many years. Few people can start more than a couple companies in a lifetime. I got three ideas for companies to start while running my first company. Starting companies gives you ideas for new companies (and variations on existing companies).

Brandon Sanderson has more ideas for novels then he can write in a lifetime. I would be astonished of artists like Vanripper or Telepurte didn’t generate ideas faster than they can draw.

If you don’t draw then you probably have few ideas for things to draw. If you don’t write novels then you probably have few ideas for novels to write.

Steal Like an Artist

My favorite book about getting ideas is Steal Like an Artist: 10 Things Nobody Told You About Being Creative by Austin Kleon. Kleon’s core premise is that to truly make an apple pie from scratch you must first create the universe. Nobody is original. Everybody copies. “Originality” is copying without getting caught.

Steal from one person and you’re plagiarising.
Steal from two people and you’re writing fanfiction.
Steal from a hundred people and it’s all yours.

Everything good I have ever written is a mix of other peoples’ art. Bayeswatch (a dystopic science fiction series about AI alignment) is a cross between I, Robot by Isaac Asimov and Men in Black (1997). Predictive Coding has been Unified with Backpropagation is a summary of someone else’s research paper. Russia has Invaded Ukraine was the curated opinions of the experts on Foreign Affairs.

The more ideas other people publish the more there is available for you to remix and adapt. Thanks to the Internet, there is more material for you to steal than ever before.

The Founder Effect

Holden Karnofsky asks:

Say that Beethoven was the greatest musician of all time (at least in some particular significant sense—see below for some caveats). Why has there been no one better in the last ~200 years—despite a vastly larger world population, highly democratized technology for writing and producing music, and a higher share of the population with education, basic nutrition, and other preconditions for becoming a great musician? In brief, where’s today’s Beethoven?

Beethoven is impressive, sure. But being impressive and being good are different things.

It can be hard to separate the things you like from the things you’re impressed with. One trick is to ignore presentation. Whenever I see a painting impressively hung in a museum, I ask myself: how much would I pay for this if I found it at a garage sale, dirty and frameless, and with no idea who painted it? If you walk around a museum trying this experiment, you’ll find you get some truly startling results. Don’t ignore this data point just because it’s an outlier.
Another way to figure out what you like is to look at what you enjoy as guilty pleasures. Many things people like, especially if they’re young and ambitious, they like largely for the feeling of virtue in liking them. 99% of people reading Ulysses are thinking “I’m reading Ulysses” as they do it. A guilty pleasure is at least a pure one. What do you read when you don’t feel up to being virtuous? What kind of book do you read and feel sad that there’s only half of it left, instead of being impressed that you’re half way through? That’s what you really like.
―Copy What You Like by Paul Graham

Beethoven wasn’t the greatest musician of all time. If he was then modern people would be listening to Beethoven all the time. Instead, here is what I have actually been listening to. These are the last ten songs in my YouTube history.

Four of them are covers. Two are others remixes. None of them are Beethoven. None of them were written in the age of Beethoven. None of them are even remixes of work written in the age of Beethoven.

Master marketer Tim Ferris believes it’s easier to create a category than to conquer someone else’s category. Beethoven’s work is prestigious because Beethoven was early. People listen to Beethoven because Beethoven is prestigious. But Beethoven’s music is not actually better than the music getting published today.

Early work tends to be worse because artisans tend to improve over time due to competition and theft. Prestige, meanwhile, tends to increase over time (if you are remembered). Thus, we live in a world where old stuff (or, at least, the old stuff good enough to be preserved) is both highly-prestigious and yet not as good as the new stuff.

Execution

An aspiring author once got into an argument with epic fantasy author Jim Butcher. The aspiring author complained that he didn’t have any good ideas. Jim Butcher claimed ideas don’t matter. Jim Butcher could write about anything. Jim Butcher asked the aspiring author for the aspiring author’s two worst ideas. The aspiring author replied “Pokémon” and “the Lost Roman Legion”.

So Jim Butcher wrote an epic fantasy novel about Pokémon and the Lost Roman Legion.

It’s called Codex Alera.

It was supposed to be a trilogy.

The fourth novel in the six-book series ranked #17 on the New York Times Best Seller List.