Machine learning metrics are tricky; if you don’t know what they mean, they tend to sound impressive, when they really aren’t. 94% accuracy is actually terrible, to the point where I would call this a scam if it looked like it was being marketed B2B. Consider: If some company has a database with a million people in it, and this technology rules out 94% of possibilities, then this puts you in a group of 60,000 people. This is about the same accuracy is they’d get if they just measured your height, and ruled out everyone more than an inch shorter or taller than you. (In fact, I’d put pretty high odds on this being exactly what the “gait recognition” neural network is actually doing.) So it might work as a cross-check in combination with some other tracking technology (eg your phone’s MAC address), but if that happens, it’s the other tracking technology you should be focusing on.
As an ML practitioner, that’s not what I’d mean if I said “94% accurate”. I would mean that the label was correct 94% of the time. This is very much affected by the size of the db—that is probably why the use a weaselly phrase “can reach”—“The average recognition rate can reach 94.1%” says the Watrix link.
This is a good point concerning current gait recognition technology. However, I don’t doubt it will improve. On longer timescales, this should happen naturally as compute gets cheaper and more data gets collected. On shorter timescales, this can be accelerated using techniques such as synthetic data generation.
Perhaps there is a natural limit to gait recognition, if it turns out that people can’t be uniquely identified from their gait, even in the limit of perfect data. But if there isn’t, then in 10 years, “94%” will turn into “99.999%”, or whatever is needed for gait recognition to be worth thinking about.
In this situation (and in the situation where I leave my phone at home), this question becomes relevant again.
Machine learning metrics are tricky; if you don’t know what they mean, they tend to sound impressive, when they really aren’t. 94% accuracy is actually terrible, to the point where I would call this a scam if it looked like it was being marketed B2B. Consider: If some company has a database with a million people in it, and this technology rules out 94% of possibilities, then this puts you in a group of 60,000 people. This is about the same accuracy is they’d get if they just measured your height, and ruled out everyone more than an inch shorter or taller than you. (In fact, I’d put pretty high odds on this being exactly what the “gait recognition” neural network is actually doing.) So it might work as a cross-check in combination with some other tracking technology (eg your phone’s MAC address), but if that happens, it’s the other tracking technology you should be focusing on.
As an ML practitioner, that’s not what I’d mean if I said “94% accurate”. I would mean that the label was correct 94% of the time. This is very much affected by the size of the db—that is probably why the use a weaselly phrase “can reach”—“The average recognition rate can reach 94.1%” says the Watrix link.
This is a good point concerning current gait recognition technology. However, I don’t doubt it will improve. On longer timescales, this should happen naturally as compute gets cheaper and more data gets collected. On shorter timescales, this can be accelerated using techniques such as synthetic data generation.
Perhaps there is a natural limit to gait recognition, if it turns out that people can’t be uniquely identified from their gait, even in the limit of perfect data. But if there isn’t, then in 10 years, “94%” will turn into “99.999%”, or whatever is needed for gait recognition to be worth thinking about.
In this situation (and in the situation where I leave my phone at home), this question becomes relevant again.