Yes, this is an important point, of which I am well aware. This is why I expect unbounded-ADAM to only be a toy model. A more realistic ADAM would use a complexity measure that takes computational complexity into account instead of K. For example, you can look at the measure C I defined here. More realistically, this measure should be based on the frugal universal prior.
Yes, this is an important point, of which I am well aware. This is why I expect unbounded-ADAM to only be a toy model. A more realistic ADAM would use a complexity measure that takes computational complexity into account instead of K. For example, you can look at the measure C I defined here. More realistically, this measure should be based on the frugal universal prior.