You can reparametrize any monotonous function to make it linear.
This can be used to predict the function
Are wildly different claims. The point is that it’s always easy to do 1. in retrospect and this has no bearing whatsoever on 2.
I think we would agree that (Log-) Flops or parameters or some mild combination of those would count as a reasonable metric?
I’m not a statistician, but from what I know it should be extremely hard to predict S-curves before their inflection point, in particular if there’s no guarantee that what you’re predicting is literally a logistic function.
That being said, trying to create benchmarks for all kinds of tasks seems like a reasonable thing to do in an case.
You can reparametrize any monotonous function to make it linear.
This can be used to predict the function
Are wildly different claims. The point is that it’s always easy to do 1. in retrospect and this has no bearing whatsoever on 2.
I think we would agree that (Log-) Flops or parameters or some mild combination of those would count as a reasonable metric?
I’m not a statistician, but from what I know it should be extremely hard to predict S-curves before their inflection point, in particular if there’s no guarantee that what you’re predicting is literally a logistic function.
That being said, trying to create benchmarks for all kinds of tasks seems like a reasonable thing to do in an case.