Both Peter and Damien think that the further SuperMemo algorithms provide no benefit.
As far as I know they make they don’t make that judgement because of data but because they have a feeling the the algorithm isn’t better.
Piotr Wozniak who actually did run the data claims:
Below you will find a general outline of the seventh major formulation of the repetition spacing algorithm used in SuperMemo. It is referred to as Algorithm SM-11 since it was first implemented in SuperMemo 11.0 (SuperMemo 2002). Although the increase in complexity of Algorithm SM-11 as compared with its predecessors (e.g. Algorithm SM-6) is incomparably greater than the expected benefit for the user, there is a substantial theoretical and practical evidence that the increase in the speed of learning resulting from the upgrade may fall into the range from 30 to 50%.
I don’t think that’s it’s certain that Piotr is right. On the other hand if he’s right that’s on a scale that matters a great deal.
If you are better at estimating when a card will be forgotten you are also nearer at the point where you do deliberate practice that might make you better at learning.
The second issue is daily memory performance variation. I’m not sure but I think there might be days when the brain doesn’t work well at storing memories. If you answer 200 cards on such a day and they get sheduled into the future and you get 20 of the first 30 cards wrong when they get tested again it would make sense to reshedule the rest of the 170 cards to a time closer to the present.
We do have practical issues that the present algorithm doesn’t handle well. You can’t tell the present algorithm that you want to really know all the facts in a deck at a particular date when you write an exam.
Having a stable mathematical theory that can predict when a card would be forgotten can help towards that end.
You might also think about the kind of tools that psychologists use to measure a trait like unconscious racism in the present. Words or images get flashed for short time durations. You might measure unconscious racism the same way through testing people long-term memory for the ability to remember related information.
If you both have the tool of flashing images and the tool that measures the effect of unconscious racism on long term memory you can start asking questions such as: “Which unconscious racism metric changes first and which lags behind?”
The Mnemonsyth data doesn’t allow us to answer that question but it can provide a foundation on which the mathematical theories for long-term memory can build that help you to run that experiment.
It can be the basis for learning stuff about the way the human mind works that you can’t get by gathering 50 participants and putting them into an fMRI while you ask questions.
Scientific progress often comes from progress in underlying tools and frameworks.
Piotr Wozniak who actually did run the data claims:
I’m not sure what data he has run; skimming that page doesn’t help much. I know he has no dataset comparable to the Mnemosyne dataset because I sent him my initial results a few months ago and he told me so, so it can’t be based on that.
At the present time he has Supermemo Online and that should provide an interesting data set. But I don’t think he had that dataset at the time he wrote those lines.
I think Piotr worked a lot with his own data. But he also writes:
The increase in the speed of the convergence was achieved by employing actual approximation data obtained from students who used SuperMemo 6 and/or SuperMemo 7
Algorithm SM-8 is constantly being perfected in successive releases of SuperMemo, esp. to account for newly collected repetition data, convergence data, input parameters, etc.
He also described it in his thesis in a bit of detail.
32 test subjects does not compare to the Mnemosyne dataset but it does provide plenty of data for testing algorithms and the might be enough data to decide that SM-8 is significantly better than SM-2.
Both Peter and Damien think that the further SuperMemo algorithms provide no benefit.
As far as I know they make they don’t make that judgement because of data but because they have a feeling the the algorithm isn’t better.
Piotr Wozniak who actually did run the data claims:
I don’t think that’s it’s certain that Piotr is right. On the other hand if he’s right that’s on a scale that matters a great deal.
If you are better at estimating when a card will be forgotten you are also nearer at the point where you do deliberate practice that might make you better at learning.
The second issue is daily memory performance variation. I’m not sure but I think there might be days when the brain doesn’t work well at storing memories. If you answer 200 cards on such a day and they get sheduled into the future and you get 20 of the first 30 cards wrong when they get tested again it would make sense to reshedule the rest of the 170 cards to a time closer to the present.
We do have practical issues that the present algorithm doesn’t handle well. You can’t tell the present algorithm that you want to really know all the facts in a deck at a particular date when you write an exam. Having a stable mathematical theory that can predict when a card would be forgotten can help towards that end.
You might also think about the kind of tools that psychologists use to measure a trait like unconscious racism in the present. Words or images get flashed for short time durations. You might measure unconscious racism the same way through testing people long-term memory for the ability to remember related information.
If you both have the tool of flashing images and the tool that measures the effect of unconscious racism on long term memory you can start asking questions such as: “Which unconscious racism metric changes first and which lags behind?”
The Mnemonsyth data doesn’t allow us to answer that question but it can provide a foundation on which the mathematical theories for long-term memory can build that help you to run that experiment.
It can be the basis for learning stuff about the way the human mind works that you can’t get by gathering 50 participants and putting them into an fMRI while you ask questions.
Scientific progress often comes from progress in underlying tools and frameworks.
I’m not sure what data he has run; skimming that page doesn’t help much. I know he has no dataset comparable to the Mnemosyne dataset because I sent him my initial results a few months ago and he told me so, so it can’t be based on that.
At the present time he has Supermemo Online and that should provide an interesting data set. But I don’t think he had that dataset at the time he wrote those lines.
I think Piotr worked a lot with his own data. But he also writes:
He also described it in his thesis in a bit of detail. 32 test subjects does not compare to the Mnemosyne dataset but it does provide plenty of data for testing algorithms and the might be enough data to decide that SM-8 is significantly better than SM-2.