I was surprised that it is possible to apply simple(?) signal processing techniques to extract subtle signals from a video, e.g. somebody’s heartbeat.
Surprise levels:
1) I never thought of that (that there could be useful hidden signals in standard video). Their paper references a few other attempts at this. 2) If I had thought of it, or someone had mentioned the idea, I would have guessed that those signals are not strong enough to
be extracted by any method. 3) And, even if there were a signal, I would have thought it would take very powerful techniques and many assumptions (like manually annotating where you expect to see the heartbeat, etc.) to make it work.
This is required less than I’d expect. From the paper:
we auto- matically select, and then amplify, a band of temporal frequencies that includes plausible human heart rates.
The same way we could obtain details from an astronomy video. One hour of a video of a distant planet might be worth of a big telescope, The long exposition time was the first step in this direction, long ago.
The current exo planet detection is another, bigger.
We simply don’t yet use every information we have.
Astronomy is an interesting connection to think about wrt to this work. In astronomy, we’re integrating the light received. In some sense this is dynamic, because there are small variations due to atmosphere. But the underlying signal is assumed to be static? I guess there are pulsars where we don’t expect that. Maybe then people have to apply similar techniques (filtering out dynamics, e.g. from atmosphere, at frequencies far from that expected from pulsars?)
You, an astronomer, should always ask yourself: Giving this light pattern in time, what is the most probable source which would give me this pattern. Be it static or dynamic, whichever fits the best.
The standard approach is to simulate multiple possible sources and use Bayesian techniques, such as maximum likelihood, to evaluate which ones match the data best and whether the best is a good enough fit. The waveforms matching in LIGO is one of the extremes, given how weak the potential signal is.
I was surprised that it is possible to apply simple(?) signal processing techniques to extract subtle signals from a video, e.g. somebody’s heartbeat.
Surprise levels:
1) I never thought of that (that there could be useful hidden signals in standard video). Their paper references a few other attempts at this.
2) If I had thought of it, or someone had mentioned the idea, I would have guessed that those signals are not strong enough to be extracted by any method.
3) And, even if there were a signal, I would have thought it would take very powerful techniques and many assumptions (like manually annotating where you expect to see the heartbeat, etc.) to make it work. This is required less than I’d expect. From the paper:
The same way we could obtain details from an astronomy video. One hour of a video of a distant planet might be worth of a big telescope, The long exposition time was the first step in this direction, long ago.
The current exo planet detection is another, bigger.
We simply don’t yet use every information we have.
Astronomy is an interesting connection to think about wrt to this work. In astronomy, we’re integrating the light received. In some sense this is dynamic, because there are small variations due to atmosphere. But the underlying signal is assumed to be static? I guess there are pulsars where we don’t expect that. Maybe then people have to apply similar techniques (filtering out dynamics, e.g. from atmosphere, at frequencies far from that expected from pulsars?)
You, an astronomer, should always ask yourself: Giving this light pattern in time, what is the most probable source which would give me this pattern. Be it static or dynamic, whichever fits the best.
The standard approach is to simulate multiple possible sources and use Bayesian techniques, such as maximum likelihood, to evaluate which ones match the data best and whether the best is a good enough fit. The waveforms matching in LIGO is one of the extremes, given how weak the potential signal is.