Trying to Try

Eliezer Yudkowsky1 Oct 2008 8:58 UTC
217 points
58 comments3 min readLW linkArchive

“No! Try not! Do, or do not. There is no try.”
—Yoda

Years ago, I thought this was yet another example of Deep Wisdom that is actually quite stupid. SUCCEED is not a primitive action. You can’t just decide to win by choosing hard enough. There is never a plan that works with probability 1.

But Yoda was wiser than I first realized.

The first elementary technique of epistemology—it’s not deep, but it’s cheap—is to distinguish the quotation from the referent. Talking about snow is not the same as talking about “snow”. When I use the word “snow”, without quotes, I mean to talk about snow; and when I use the word “”snow”″, with quotes, I mean to talk about the word “snow”. You have to enter a special mode, the quotation mode, to talk about your beliefs. By default, we just talk about reality.

If someone says, “I’m going to flip that switch”, then by default, they mean they’re going to try to flip the switch. They’re going to build a plan that promises to lead, by the consequences of its actions, to the goal-state of a flipped switch; and then execute that plan.

No plan succeeds with infinite certainty. So by default, when you talk about setting out to achieve a goal, you do not imply that your plan exactly and perfectly leads to only that possibility. But when you say, “I’m going to flip that switch”, you are trying only to flip the switch—not trying to achieve a 97.2% probability of flipping the switch.

So what does it mean when someone says, “I’m going to try to flip that switch?”

Well, colloquially, “I’m going to flip the switch” and “I’m going to try to flip the switch” mean more or less the same thing, except that the latter expresses the possibility of failure. This is why I originally took offense at Yoda for seeming to deny the possibility. But bear with me here.

Much of life’s challenge consists of holding ourselves to a high enough standard. I may speak more on this principle later, because it’s a lens through which you can view many-but-not-all personal dilemmas—”What standard am I holding myself to? Is it high enough?”

So if much of life’s failure consists in holding yourself to too low a standard, you should be wary of demanding too little from yourself—setting goals that are too easy to fulfill.

Often where succeeding to do a thing, is very hard, trying to do it is much easier.

Which is easier—to build a successful startup, or to try to build a successful startup? To make a million dollars, or to try to make a million dollars?

So if “I’m going to flip the switch” means by default that you’re going to try to flip the switch—that is, you’re going to set up a plan that promises to lead to switch-flipped state, maybe not with probability 1, but with the highest probability you can manage—

—then “I’m going to ‘try to flip’ the switch” means that you’re going to try to “try to flip the switch”, that is, you’re going to try to achieve the goal-state of “having a plan that might flip the switch”.

Now, if this were a self-modifying AI we were talking about, the transformation we just performed ought to end up at a reflective equilibrium—the AI planning its planning operations.

But when we deal with humans, being satisfied with having a plan is not at all like being satisfied with success. The part where the plan has to maximize your probability of succeeding, gets lost along the way. It’s far easier to convince ourselves that we are “maximizing our probability of succeeding”, than it is to convince ourselves that we will succeed.

Almost any effort will serve to convince us that we have “tried our hardest”, if trying our hardest is all we are trying to do.

“You have been asking what you could do in the great events that are now stirring, and have found that you could do nothing. But that is because your suffering has caused you to phrase the question in the wrong way… Instead of asking what you could do, you ought to have been asking what needs to be done.”
—Steven Brust, The Paths of the Dead

When you ask, “What can I do?”, you’re trying to do your best. What is your best? It is whatever you can do without the slightest inconvenience. It is whatever you can do with the money in your pocket, minus whatever you need for your accustomed lunch. What you can do with those resources, may not give you very good odds of winning. But it’s the “best you can do”, and so you’ve acted defensibly, right?

But what needs to be done? Maybe what needs to be done requires three times your life savings, and you must produce it or fail.

So trying to have “maximized your probability of success”—as opposed to trying to succeed—is a far lesser barrier. You can have “maximized your probability of success” using only the money in your pocket, so long as you don’t demand actually winning.

Want to try to make a million dollars? Buy a lottery ticket. Your odds of winning may not be very good, but you did try, and trying was what you wanted. In fact, you tried your best, since you only had one dollar left after buying lunch. Maximizing the odds of goal achievement using available resources: is this not intelligence?

It’s only when you want, above all else, to actually flip the switch—without quotation and without consolation prizes just for trying—that you will actually put in the effort to actually maximize the probability.

But if all you want is to “maximize the probability of success using available resources”, then that’s the easiest thing in the world to convince yourself you’ve done. The very first plan you hit upon, will serve quite well as “maximizing”—if necessary, you can generate an inferior alternative to prove its optimality. And any tiny resource that you care to put in, will be what is “available”. Remember to congratulate yourself on putting in 100% of it!

Don’t try your best. Win, or fail. There is no best.

Eliezer Yudkowsky1 Oct 2008 8:58 UTC
217 points
58 comments3 min readLW linkArchive