Admittedly, the success of AlphaZero relied on it being essentially able to generate very, very large amounts of very high-quality data, so this is a domain where synthetic data was very successful.
So a weaker version of the post is “you need either a lot of data, and high quality ones, or high amounts of compute, and there’s little going around it.”
Admittedly, the success of AlphaZero relied on it being essentially able to generate very, very large amounts of very high-quality data, so this is a domain where synthetic data was very successful.
So a weaker version of the post is “you need either a lot of data, and high quality ones, or high amounts of compute, and there’s little going around it.”