Can we actually speed up the “thinking” of fully trained ML models by K times during inference if we run it on processors that are K times faster?
Yes, definitely.
a. Yes b. Yes
This is all with the caveat that doing things faster doesn’t mean that it can solve bigger, more difficult problems or that it’s solutions will be of a higher quality.
Thanks for the reply. Do you have any position or intuitions on question 1 or 2?
Does more inference compute speed up inference time?
Can we actually speed up the “thinking” of fully trained ML models by K times during inference if we run it on processors that are K times faster?
Yes, definitely.
a. Yes b. Yes
This is all with the caveat that doing things faster doesn’t mean that it can solve bigger, more difficult problems or that it’s solutions will be of a higher quality.