Someone launched a truly minimum-viable-product attack, without doing any of their homework, and quickly got caught, showing us what is coming.
They didn’t get caught; they got detected. They’re still out there, free to iterate on the strategy until they get good at it. They incurred almost no cost with this initial probe.
Like other forms of spam and social engineering, this is not going to be difficult for people ‘on the ball’ to defend against any time soon, but we should worry about the vulnerable, especially the elderly, and ensure they are prepared.
I’ve gotten phishes that I wasn’t sure about until I investigated them using tools and strategies not easily available to most “on the ball” people. And they weren’t even spear phishes. You can fool almost anybody if you have a reasonable amount of information about them and tailor the attack to them.
And “immunity” is not without cost. If it gets to the point where a large class of legitimate messages have to be ignored because they can’t be distinguished from false ones, that in itself does real damage.
Voices and faces used to be very convenient, easy, relatively reliable authentication tools, and it hurts to lose something like that. Also, voices and faces are kind of an emotional “root password”. Humans may be hardwired to find it hard to ignore them. At the very least, even if they are ignored, it’s going to be actually painful to do it.
I mean, I’m not saying it’s the apocalypse, and there are plenty of ways to scam without AI, but this stuff is not good AT ALL.
I mean, I’m not saying it’s the apocalypse, and there are plenty of ways to scam without AI, but this stuff is not good AT ALL.
It will however be a very strong impetus for establishing a verified identity phone system, which would also get rid of current human and simple machine generated spam calls.
I guess maybe. A system like that isn’t easy to set up, and it’s not like there aren’t plenty of scams out there already to provide whatever incentives.
To have helped with the publicized incident, the verification would have had to be both mandatory and very strong, because the scammer was claiming to be calling from the kidnapper’s phone, and could easily have made a totally credible claim that the victim’s phone was unavailable. That means no anonymous phone calls, anywhere, ever. A system where it’s impossible to communicate anonymously is very far from an unalloyed good, so it may or may not be a “positive consequence” at all on the whole.
Also, for the niche that voices were filling, anything that demands that you carry a device around with you is just plain not as good.
It’s pretty rare to get so banged up that your face and voice are unrecognizable, especially if you can still communicate at all. Devices, on the other hand, get lost or broken quite a bit, including in cases where you might be trying to ask somebody you knew for money.
In the common “I got arrested” scam, the mark expects that the impersonated person’s phone won’t be available to them. The victim could of course notice that the person isn’t calling from a police station, assuming the extra constraint that the identification system delivers an identifier that’s unambiguously not a police station… but that just means the scammer switches to the equally common “I got mugged” or “car accident” scams. There are so many degrees of freedom that you can work around almost any technical measure.
Voices (used to) bind the content of a message directly to a person’s vocal tract, and faces on video came pretty close to binding the message to the face. Device-based authentication relies on a much longer chain of steps, probably person to ID card/database photo to phone company records to crypto certificate to key to device. And, off on the side, the ID card database has to bind that face to information that can actually physically locate a scammer. Any of those steps can be subverted, and it’s a LOT of work to secure all of them, especially because...
With no coordination at all, everybody on the planet automatically gets a face and a voice that’s “compatible with the system”, and directly available to important relying parties (namely the people who actually know you and who are likely to be scam victims).
Your device, on the other hand, may be certified by any number of different carriers, manufacturers, or governments, who have to cooperate in really complicated ways to get any kind of real verification. It takes forever and costs a lot to set up anything like that at the scale of a worldwide phone system.
It would be easier to set up intra-family “web of trust” device-based authentication… but of course that fails on the “mandatory” and “automatic” parts.
Device-based authentication can be stronger in many ways than vocal or visual authentication could ever be, and in some cases it’s obviously superior, but I don’t think it’s a satisfying substitute. And most of its advantages tend to show up in much smaller communities/namespaces than the total worldwide phone system.
They didn’t get caught; they got detected. They’re still out there, free to iterate on the strategy until they get good at it. They incurred almost no cost with this initial probe.
I’ve gotten phishes that I wasn’t sure about until I investigated them using tools and strategies not easily available to most “on the ball” people. And they weren’t even spear phishes. You can fool almost anybody if you have a reasonable amount of information about them and tailor the attack to them.
And “immunity” is not without cost. If it gets to the point where a large class of legitimate messages have to be ignored because they can’t be distinguished from false ones, that in itself does real damage.
Voices and faces used to be very convenient, easy, relatively reliable authentication tools, and it hurts to lose something like that. Also, voices and faces are kind of an emotional “root password”. Humans may be hardwired to find it hard to ignore them. At the very least, even if they are ignored, it’s going to be actually painful to do it.
I mean, I’m not saying it’s the apocalypse, and there are plenty of ways to scam without AI, but this stuff is not good AT ALL.
It will however be a very strong impetus for establishing a verified identity phone system, which would also get rid of current human and simple machine generated spam calls.
So it does have some positive consequences.
I guess maybe. A system like that isn’t easy to set up, and it’s not like there aren’t plenty of scams out there already to provide whatever incentives.
To have helped with the publicized incident, the verification would have had to be both mandatory and very strong, because the scammer was claiming to be calling from the kidnapper’s phone, and could easily have made a totally credible claim that the victim’s phone was unavailable. That means no anonymous phone calls, anywhere, ever. A system where it’s impossible to communicate anonymously is very far from an unalloyed good, so it may or may not be a “positive consequence” at all on the whole.
Also, for the niche that voices were filling, anything that demands that you carry a device around with you is just plain not as good.
It’s pretty rare to get so banged up that your face and voice are unrecognizable, especially if you can still communicate at all. Devices, on the other hand, get lost or broken quite a bit, including in cases where you might be trying to ask somebody you knew for money.
In the common “I got arrested” scam, the mark expects that the impersonated person’s phone won’t be available to them. The victim could of course notice that the person isn’t calling from a police station, assuming the extra constraint that the identification system delivers an identifier that’s unambiguously not a police station… but that just means the scammer switches to the equally common “I got mugged” or “car accident” scams. There are so many degrees of freedom that you can work around almost any technical measure.
Voices (used to) bind the content of a message directly to a person’s vocal tract, and faces on video came pretty close to binding the message to the face. Device-based authentication relies on a much longer chain of steps, probably person to ID card/database photo to phone company records to crypto certificate to key to device. And, off on the side, the ID card database has to bind that face to information that can actually physically locate a scammer. Any of those steps can be subverted, and it’s a LOT of work to secure all of them, especially because...
With no coordination at all, everybody on the planet automatically gets a face and a voice that’s “compatible with the system”, and directly available to important relying parties (namely the people who actually know you and who are likely to be scam victims).
Your device, on the other hand, may be certified by any number of different carriers, manufacturers, or governments, who have to cooperate in really complicated ways to get any kind of real verification. It takes forever and costs a lot to set up anything like that at the scale of a worldwide phone system.
It would be easier to set up intra-family “web of trust” device-based authentication… but of course that fails on the “mandatory” and “automatic” parts.
Device-based authentication can be stronger in many ways than vocal or visual authentication could ever be, and in some cases it’s obviously superior, but I don’t think it’s a satisfying substitute. And most of its advantages tend to show up in much smaller communities/namespaces than the total worldwide phone system.