Is there a better choice than “Stop After Win” (or does double-checking lead to failure loops)?
Here we go.
To recap:
Last week I wrote about what happened when I applied metrics to my piano practice, in which I outlined the “secret to learning” in eight steps:
Define win condition.
Define action you are going to take to achieve win condition.
Take defined action.
Evaluate action both against its original definition (that is, did you do what you said you were going to do or did you do something else) and against the win condition.
Ask yourself what is keeping you from achieving your win condition. Describe it as specifically as possible.
Define action you are going to take to solve/address/eliminate obstacle preventing win condition.
Repeat 2-7 until win condition is achieved.
STOP.
Step #8, “Stop after win,” has prompted significant discussion both in various subsets of the internet and in various rooms of our home. It goes against the standard practice (pun intended) of repeating a learned skill multiple times after the first accurate recall, otherwise known as “playing it five times correctly.”
Is it better to stop after an accurate recall, or to immediately repeat?
If I’ve got it wrong, that’s good news. I’ll say “WOO-HOO-HOO, I LEARNED SOMETHING NEW” and update my practice strategy accordingly.
But let’s do some more investigating — and share the initial results of a randomized experiment.
There are two arguments in favor of immediate repetition:
Immediate repetition allows you to confirm that you know something.
I have no counter-argument for this. I agree.
Immediate repetition increases the win/fail ratio. If you fail at a passage four times and win once, what is your brain likely to remember the next day — the four fails or the one win? If you can generate four fails followed by five wins, your brain should be more likely to remember the win condition.
My first counter-argument is that this only works if you can actually generate five consecutive wins in a row. What happens if it looks more like FFFFWFWWFW? What’s your brain going to remember then?
I’m also going to counter-argue — and my entire post was going to be on this particular topic, when I originally set out to write it — that accurately describing why you failed and what you need to do differently next time does as much to strengthen your neural network as an accurate pass. Conversely, you can weaken your neural network by playing the passage over and over without stopping after each pass to evaluate what happened.
There are also a few arguments in favor of stopping after an accurate recall:
As I mentioned in my last post, STOP AFTER WIN means that you are always returning to a passage from a WIN CONDITION. You got this last time, which means you can get it this time.
The counter-argument is, obviously, but what if you don’t get it right the first time you play it during the next practice session? This appears to be the most relevant counter-argument and the biggest potential issue with the practice.
Double-checking leads to fail loops. You either look at the stove once, observe that it is off, and go about your day — or you look at the stove, observe that it is off, take a few steps away, ask yourself if you really paid attention, turn around, look at the stove again, observe that it is off, walk out of the kitchen, ask yourself how you can trust your observation this time if you didn’t trust it the last time, turn around, stare at the stove for a while, point at every burner and say “off,” leave the kitchen, get halfway up the stairs, and turn around again.
The counter-argument could be “but if you do this for piano then you will be really really certain that you know something,” to which my counter-counter-argument is “no no no you don’t understand, every time you double-check you are reconfirming a mental state in which you are not sure whether you know something.”
Then you’ll say “what about checking your work in school, your math worksheet and your spelling test and so on” and I’ll say “yes yes yes checking once is good, very good, find your errors and fix them, but repeatedly checking over and over leads to mental discreditation.”
One commenter suggested that I run an experiment in which I flip a coin to determine which path to take — STOP AFTER WIN or REPEAT AFTER WIN.
Here are the results from this morning’s practice session:
I know that is probably hard to read, but in this case I played Hanon exercises 1-11 in full, identified two technical errors I wanted to correct (please note that these were not missed notes, these were fourth-finger-being-slower-than-other-fingers errors), isolated the measures in which the errors were present, played the first isolated section, WON, flipped the coin, got heads, WON again (but noted that it was “much harder to retain focus”), played the second isolated section, WON, flipped the coin, got heads, WON again.
All very good, right?
Sure — for a warmup.
Here’s what happened when I ran the same experiment on Ravel’s La Valse (four-hand duet, prima part):
At the end of my last independent practice session, measures 1 through 210 were all at WIN CONDITION (defined in this case as memorized, note-accurate, and even). When I worked on the piece as a duet with L last night, we identified a more efficient fingering that could be used in measure 162 — so I began today’s practice session by solidifying that change and winning measures 146-178.
Flip coin, tails, STOP AFTER WIN.
Then I played measures 179-210. I won these during my last session, and I won them this time around. This is how the system is intended to work.
I flipped the coin.
Heads.
I spent the next thirty minutes trying to pull off another win. Practice notes include “losing focus” and “thinking about experiment” and “complete memory lapse.”
This could be my fault — I mean, it is obviously my fault — in the sense that someone else might be able to pull off repeated wins in a similar experiment.
But my data is so far indicating that second-guessing something you just won only leads to confusion.
You’ll notice, if you pay close attention to the spreadsheet, that I found a potential loophole in all of this.
For short sections of music, the kind of passages that may only take a minute or two to play, you can define WIN as “achieve win condition five times consecutively,” or “WIN 5xCx.” This could potentially give us the best of both worlds (and/or I could have just reinvented the wheel).
That won’t work for larger sections of music, because five consecutive wins could take you a full hour to play even if you get five winning passes the first time you try.
And I’m still uncertain about whether that’s a good idea — as I was telling L the other night, playing a 5-minute section of a difficult piece (or a 15-minute piece in full) might be the kind of thing you can only do once a day.
“If you do it right,” I argued, “you have to give it everything you have, which doesn’t leave anything left over for a repeat performance.”
“Why would that be the case?” he argued right back. “If you really know something, you should be able to repeat it accurately as often as you want to.”
I’ll end by sharing the video I took of my last pass through measures 1-210 of Ravel’s La Valse (before identifying and working two errors):
If you were me, how would you manage your next practice session? Stop after win? Update all win conditions (that take fewer than two minutes to execute) to 5xCx? Keep flipping that coin? ❤️
Fatigue is real. Even machines can’t repeat actions indefinitely without needing a rest or maintenance. A more relaxed version of L’s statement is true, in that the ability to repeat something accurately more than once is one way to measure how well you know something. I’d expect this measure to correlate with, say, the ability to reproduce the piece in performance—but imperfectly. At a certain point, other constraints become binding. Have you practiced in front of an audience? On a piano other than your own, in a room other than where you’re used to practicing, in a different emotional and physical state than you’re used to?
I’d also caution that defining your “win” condition as “played the passage X times perfectly” has an Achilles’ heel, which is that it incorporates a lot of randomness. If you think of each play-through as having a P% probability of success, then you can achieve the win condition at an arbitrarily low value of P by playing a sufficient number of times. This can lead to frustration (if it goes on too long without success) or to illusory success (if you get lucky).
The way I’ve always suggested my students cope with this is by selecting the dimensions of their passage intelligently. They should pick a passage length that they feel confident they can play correctly on their first or second try. if that fails, they should shorten the passage significantly and try again, until they’ve found a portion of the passage—even a single note or chord—that they can play on their first or second try. Then they can start trying to combine these fragments into a longer whole.
This tends to be uncomfortable at first, because a good composer writes their music in such a way that you always want to hear one more note—kind of like how a good video game makes you want to play just one more turn. It also forces students to break out of their ruts. But, especially with adult students, they’ve tended to like it once they get used to it and find that it’s helpful to them.
But the probability of success increases as you accumulate previous successes, right?
And there’s a difference between “play 5 times perfectly” and “play 5 times perfectly consecutively.” Much more randomness (and potential regression to the mean) if you are allowed to have imperfect runs between your perfect ones.
For sure! I think people are trying to get a couple different things out of practice quantification:
Self-evaluation: evidence of their skill level
Decision-making: a criterion for deciding when marginal practice time is best spent elsewhere
Motivation: appreciating their successes, having a sense of progress, building consistency over time
It’s both tricky to know which one you’re doing, and also to do it effectively.
For self-evaluation, I think it’s most helpful to make your evaluation as similar to what you’re actually aiming for as possible. If you want to be able to perform in public without memory slips, then arrange some public performances to test whether you’re able to do this or not. A nursing home is great for this—or was before the pandemic, anyway. If you want to be able to accurately play long passages with no warm-up, then start by trying to play short passages with no warm-up. Being very clear about exactly what you’re evaluating, and having a sense of some alternatives, in order to select the most relevant test, is helpful here.
What I really think is important is to focus on “converging lines of evidence.” Playing in front of a crowd on a piano that’s not your own, being able to play accurately on your first time, being able to start in the middle of the piece, and being able to play at a variety of tempi give you a lot more information about your skill level than being able to play a particular passage accurately even 100x in a row.
For decision-making, the reason I think it’s best to move on after one success is that going too deep on one area tends to mean people don’t practice their entire piece, or get hyper-focused on one piece at a time. Also, memory research shows that small bits of practice on particular chunks of information spread out over time are more effective than overlearning the chunks all at once. So “STOP AFTER WIN” tends to spread efforts out more, which I think leads to more effective practice.
For motivation, “STOP AFTER WIN” produces a string of happy-feeling successes.
I agree with you on ALL OF THIS. Make your evaluation as similar to what you’re actually aiming for as possible, make sure you don’t neglect any sections of music and/or allow previously learned material to degrade, spread effort over time aka spaced repetition.
BTW, in our house we’re building a “piano performance ladder” (house concert, smaller venue, bigger venue, duets with other musicians, etc.). My mom used to teach this kind of thing to kids—play for parents first, then grandparents, then church or nursing home, etc. It holds up for adults too...
I like the idea of a piano performance ladder! Gives some built-in social validation to the work of learning piano music.
When making my comment on your last post, I made a wrong assumption: that the best strategy would be independent of the task. Because of this, I generalized from learning a much lower level of piano and from learning to juggle. For these things, each attempt would take from 2 seconds to 1 minute, and for such tasks I think it makes the most sense to continue after one win. But I can see the situation is very different if each attempt takes several minutes or more.
I have more discrimination between types of thing.
Play short or tricky points in isolation, get it right, then repeat the correct thing to ingrain it.
Then play the whole piece once, regardless of if it’s a W or L. Then if time remains, go back to #1.
Yep, that sounds like a reasonable strategy. Repeat the parts to perfect the whole...
In my experience with piano practicing, the consecutive win method produced much stronger results when I came back the next day.
My experience was with practicing relatively short segments (up to ~1 minute per repeat) and unrandomized, but I found that if I practiced until one win (LLLLLW) I would invariably get it wrong the next day while if I practiced until three consecutive wins (LLLLLWLLWWLWLWWW), I would usually (3/4 of the time) get it right the next day.
Obviously this takes more time, but I found that if I tried to STOP after one win, I would repeat the exact same process day after day on the same segment while if I repeated until three consecutive wins then I could usually move on the next day. That is to say, STOPping after one win meant that the entire practice section was useless.
If your segment is too long to repeat until multiple wins, it seems like breaking the segment into smaller pieces, perfecting them, and then treating combining as a final goal could result in less total time.
What I’m seeing in the comments, btw, is something I’ve been curious about testing next—whether the ideal scenario is something like “identify single problem, solve single problem, repeat solution five times perfectly consecutively (if you can’t do that, you haven’t solved the problem yet)” and then move on to the next problem.
At the end of your practice session you can test your work by playing all of the measures you’ve solved in current or previous sessions (the whole piece, if possible, or just the parts you’ve been able to address so far). If you win that attempt, STOP. If not, add failed measures to the next session’s problem-work-set...
This is a really good post – thanks!
I think I side more with you – for ‘performances’ like what you’re describing.
For certain ‘atomic’ or ‘small-length’ “something[s]” that one might know, it can or might be the case that a sufficiently knowledgable/capable person would be able to repeat the ‘something’ many many times accurately, nearly “as often as [they] want to”. (But I’m not sure “L” was thinking of repeating something over several hours.)
But even the best people at any particular kind of not-small-length ‘performance’ will find that their very best performances are fairly rare.
This seems very similar to how speedrunning video/computer games works. The best speedrunners absolutely are capable of repeating all kinds of ‘small’ actions or techniques nearly perfectly, and for hours at a time too.
But a ‘speedrunning performance’, e.g. a ‘complete’ run of some game, is so difficult, as ‘a unit’, that it’s not usually possible to repeat a subsequent ‘win’. (There are streaks of new records by the same person, but that’s rare – as fraction of all the games that same person plays.)
Then again, a speedrunner in one sense only ‘wins’ if they achieve a (new) ‘personal best’ run and I don’t think your own ‘performance win’ condition is similar in that way.
But – in some sense – you might be able to practice enough that you could repeat these performances, at least a few times, and still feel you like ‘win’. I imagine you wouldn’t also feel like “you have to give it everything you have” tho.
I will say that L clarified this morning that “repetition begins when something is learned,” which is to say that once you know something, the next step is to repeat what you know.
More on all of this as I continue to collect data...