What’s the corresponding story here for trading bots? Are they designed in a sufficiently high-assurance way that new tail problems don’t come up, or do they not operate in the tails?
Ten years ago, Knight Capital was the largest high-frequency trader in US equities. On August 1 2012, somebody deployed a bug. Knight’s testing platform included a component which generated random orders and sent them to a simulated market; somebody accidentally hooked that up to the real market. It’s exactly the sort of error testing won’t catch, because it was a change outside of the things-which-are-tested; it was partly an error in deployment, and partly code which did not handle partial deployment. The problem was fixed about 45 minutes later. That was the end of Knight Capital.
So yes, trading bots definitely operate in the tails.
When the Knight bug happened, I was interning at the largest high-frequency trading company in US options. Even before that, the company was more religious about thorough testing than any other I’ve worked at. Everybody knew that one bug could end us, Knight was just a reminder (specifically a reminder to handle partial deployment properly).
What’s the corresponding story here for trading bots? Are they designed in a sufficiently high-assurance way that new tail problems don’t come up, or do they not operate in the tails?
Great question. Let’s talk about Knight Capital.
Ten years ago, Knight Capital was the largest high-frequency trader in US equities. On August 1 2012, somebody deployed a bug. Knight’s testing platform included a component which generated random orders and sent them to a simulated market; somebody accidentally hooked that up to the real market. It’s exactly the sort of error testing won’t catch, because it was a change outside of the things-which-are-tested; it was partly an error in deployment, and partly code which did not handle partial deployment. The problem was fixed about 45 minutes later. That was the end of Knight Capital.
So yes, trading bots definitely operate in the tails.
When the Knight bug happened, I was interning at the largest high-frequency trading company in US options. Even before that, the company was more religious about thorough testing than any other I’ve worked at. Everybody knew that one bug could end us, Knight was just a reminder (specifically a reminder to handle partial deployment properly).