This seems analogous to the situation where there’s some quant at Jane Street, and they’re about to run code that will make thousands of dollars trading stocks, and someone comes up to them and says “Wait! You should add checks to your code to make sure that no subset of your trades will lose you money!”
I would be quite surprised if Wall Street quant firms didn’t use unit tests of this sort before letting a new algorithm play with real money. And in fact, I can imagine a promising-sounding algorithm flaming out by making a cycle of Dutch-booked trades...
I think the point is, it doesn’t matter if the parts of your model that you’re not thinking about are wrong.
A fine quant algorithm could have an incomplete, inconsistent model that is Dutch-bookable only in regions outside of its main focus. Before it would go out and make those Dutch-booking trades, though, it would focus on those areas and, get itself consistent there, and make good (or no) trades instead.
I would be quite surprised if Wall Street quant firms didn’t use unit tests of this sort before letting a new algorithm play with real money. And in fact, I can imagine a promising-sounding algorithm flaming out by making a cycle of Dutch-booked trades...
I think the point is, it doesn’t matter if the parts of your model that you’re not thinking about are wrong.
A fine quant algorithm could have an incomplete, inconsistent model that is Dutch-bookable only in regions outside of its main focus. Before it would go out and make those Dutch-booking trades, though, it would focus on those areas and, get itself consistent there, and make good (or no) trades instead.