And note that we’ll be able to tell whether this is working, so in practice this is probably something that we can validate empirically—not something where we are going up against adversarial optimization pressure and so need a provable bound.
This is kind of surprising. (I had assumed that you need a provable bound since you talk about guarantees and cite a paper that talks about provable bounds.)
If you have some ML algorithm that only has an exponential provable bound but works well in practice, aren’t you worried that you might hit a hard instance of some task in the future that it would perform badly on, or there’s a context shift that causes a whole bunch of tasks to become harder to learn? Is the idea to detect that at run time and either pay the increased training cost or switch to another approach if that happens?
If you want to understand these intuitions in detail it likely requires doing the equivalent of a course in learning theory and reading a bunch of papers in the area (which doesn’t sound worth it to me, as a use of your time).
Ok, that’s good to know. I think the explanations you gave so far is good enough for my purposes at this point. (You might want to consider posting them somewhere easier to find with a warning similar to this one, so people don’t try to figure out what your intuitions are from the OL survey paper like I did.)
This is kind of surprising. (I had assumed that you need a provable bound since you talk about guarantees and cite a paper that talks about provable bounds.)
If you have some ML algorithm that only has an exponential provable bound but works well in practice, aren’t you worried that you might hit a hard instance of some task in the future that it would perform badly on, or there’s a context shift that causes a whole bunch of tasks to become harder to learn? Is the idea to detect that at run time and either pay the increased training cost or switch to another approach if that happens?
Ok, that’s good to know. I think the explanations you gave so far is good enough for my purposes at this point. (You might want to consider posting them somewhere easier to find with a warning similar to this one, so people don’t try to figure out what your intuitions are from the OL survey paper like I did.)