Short version: Make an Eckman-style micro-expression reader in a wearable computer.
Fleshed out version: You have a wearable computer (perhaps something like Google glass) which sends video from its camera (or perhaps two cameras if one camera is not enough) over to a high-powered CPU which processes the images, locates the faces, and then identifies micro expressions by matching and comparing the current image (or 3D model) to previous frames to infer which bits of the face have moved in which directions. If a strong enough micro-expression happens, the user is informed by a tone or other notification. Alternatively, one could go the more pedagogical route by showing then a still frame of the person doing the micro-expression some milliseconds prior with the relevant bits of the face highlighted.
Feasibility: We already can make computers are good at finding faces in images and creating 3D models from multiple camera perspectives. I’m pretty sure small cameras are good enough by now. We need the beefy CPU and/or GPU as a separate device for now because it’s going to be a while before wearables are good enough to do this kind of heavy-duty processing on their own, but wifi is good enough to transmit very high resolution video. The foggiest bit in my model would be whether current image processing techniques are up to the challenge. Would anyone with expertise in machine vision care to comment on this?
Possible positive consequences: Group collaboration easily succumbs to politics and scheming unless a certain (large) level of trust and empathy has been established. (For example, I’ve seen plenty of hacker news comments confirm that having a strong friendship with one’s startup cofounder is important.) A technology such as this would allow for much more rapid (and justified) trust-building between potential collaborators. This might also allow for the creation of larger groups of smart people who all trust each other. (Which would be invaluable for any project which produces information which shouldn’t be leaked because it would allow such projects to be larger.) Relatedly, this might also allow one to train really excellent therapist-empaths.
Possible negative consequence: Police states where the police are now better at reading people’s minds.
Version 0.1 can be for Skype conversations.
Imagine the heightened ‘super power’ ability to discreetly (or not so discreetly) pick up on this during your personal and business Skype chats.
I wear a GoPro camera around my neck for a life-logging project, and have tried it with a wifi (EyeFi) card. If you want live video or pics, the battery lasts around 1 hr for 1picture per 5 seconds. If you want video at 30fps at 960p, the interchangeable batteries last about 1.5 hours and records about 5.5 hrs on a 32gb card (max size supported)
The files are huge, cumbersome, and do little for me.
I have been entertaining the idea of a version that recognizes your mood throughout the day with your webcam, and plots it over time based on what type of tasks you were performing. Over time, your laptop could suggest transitioning from certain tasks to others based on your expressions to optimize for personalized productivity and mood.
Another idea is to make lending out laptops free if the user agrees to having essentially no privacy—you’d sell the information and user expressions as they experience certain sites back to the companies that would pay for this program and reap a healthy profit along the way! (A part of which you’d totally send to me.)
This could be a sustainable way to get more internet enabled laptops into more hands and push people to become more contributing, creating members of society rather than the majority passive consumers that we experience today.
Version .1 of this laptop program could be lending out old/donated/extra laptops under the condition that the lendees who use the laptops create 1 thing a day of notable worth to themselves (or one project per every x hours). So everyone is held together by incrementally improving themselves and creating projects of value that are auto uploaded from a certain folder into a community of people who give each other feedback, etc.
Look forward to talking about this next time I see you in person Marcello. Also typing about this here for anyone else that happens to be reading, feel free to find me on Facebook.com/AltonSun to further the conversation.
I did an internship with these guys a couple years ago, and one of the teams was already working on the problem. Ekman’s ideas were specifically brought up as a basic idea of what they were doing, but expanded beyond just microexpressions and the like. Some other things they were looking at included pore-dilation, and thermal imaging.
It wasn’t my team, so I don’t remember too many details, but I remember a problem being that they had to have the subjects immobile in one place, and surrounded by an array of very expensive cameras and sensors.
If you could design a system where you could set up enough sensors to look at everyone in a room, despite the fact that they were moving around, etc, and be able to pick out warning signs for violence (one of their most desired use cases), you’d be in business.
They were sponsored by DARPA or AFRL, but the work was public, so might be able to find some info by browsing around. Also, if they were doing it, I would guess that other colleges were too.
Further, even if you assume Ekman’s research is entirely valid, I’m not sure that video analysis technology can be reliable or effective enough to be useful at the present stage, especially when we’re talking about stimuli that last for such a short period—typically 40-100 ms. Machine vision isn’t my forte either, so I’m not positive, but you would have to be really accurate for something like this to be fully useful, and quite fast as well if you want real-time conversational feedback. It’s also important to consider that having a program like this with a substantial error rate would likely be worse than not having a program at all.
I do think that—if possible—this would be a great idea (indeed, it would represent one of the “killer apps” for Google Glass and similar wearable computing projects if it were effective), but I think both the research behind the idea and the ability to actually implement this in an effective fashion are very shaky at this stage.
Short version: Make an Eckman-style micro-expression reader in a wearable computer.
Fleshed out version: You have a wearable computer (perhaps something like Google glass) which sends video from its camera (or perhaps two cameras if one camera is not enough) over to a high-powered CPU which processes the images, locates the faces, and then identifies micro expressions by matching and comparing the current image (or 3D model) to previous frames to infer which bits of the face have moved in which directions. If a strong enough micro-expression happens, the user is informed by a tone or other notification. Alternatively, one could go the more pedagogical route by showing then a still frame of the person doing the micro-expression some milliseconds prior with the relevant bits of the face highlighted.
Feasibility: We already can make computers are good at finding faces in images and creating 3D models from multiple camera perspectives. I’m pretty sure small cameras are good enough by now. We need the beefy CPU and/or GPU as a separate device for now because it’s going to be a while before wearables are good enough to do this kind of heavy-duty processing on their own, but wifi is good enough to transmit very high resolution video. The foggiest bit in my model would be whether current image processing techniques are up to the challenge. Would anyone with expertise in machine vision care to comment on this?
Possible positive consequences: Group collaboration easily succumbs to politics and scheming unless a certain (large) level of trust and empathy has been established. (For example, I’ve seen plenty of hacker news comments confirm that having a strong friendship with one’s startup cofounder is important.) A technology such as this would allow for much more rapid (and justified) trust-building between potential collaborators. This might also allow for the creation of larger groups of smart people who all trust each other. (Which would be invaluable for any project which produces information which shouldn’t be leaked because it would allow such projects to be larger.) Relatedly, this might also allow one to train really excellent therapist-empaths.
Possible negative consequence: Police states where the police are now better at reading people’s minds.
Version 0.1 can be for Skype conversations. Imagine the heightened ‘super power’ ability to discreetly (or not so discreetly) pick up on this during your personal and business Skype chats.
I wear a GoPro camera around my neck for a life-logging project, and have tried it with a wifi (EyeFi) card. If you want live video or pics, the battery lasts around 1 hr for 1picture per 5 seconds. If you want video at 30fps at 960p, the interchangeable batteries last about 1.5 hours and records about 5.5 hrs on a 32gb card (max size supported)
The files are huge, cumbersome, and do little for me.
I have been entertaining the idea of a version that recognizes your mood throughout the day with your webcam, and plots it over time based on what type of tasks you were performing. Over time, your laptop could suggest transitioning from certain tasks to others based on your expressions to optimize for personalized productivity and mood.
Affectiva’s Affdex is a company to look to for this, and has a great demo that plots your expressions over time while watching commercials: http://www.affectiva.com/affdex/#pane_overview
Another idea is to make lending out laptops free if the user agrees to having essentially no privacy—you’d sell the information and user expressions as they experience certain sites back to the companies that would pay for this program and reap a healthy profit along the way! (A part of which you’d totally send to me.)
This could be a sustainable way to get more internet enabled laptops into more hands and push people to become more contributing, creating members of society rather than the majority passive consumers that we experience today.
Version .1 of this laptop program could be lending out old/donated/extra laptops under the condition that the lendees who use the laptops create 1 thing a day of notable worth to themselves (or one project per every x hours). So everyone is held together by incrementally improving themselves and creating projects of value that are auto uploaded from a certain folder into a community of people who give each other feedback, etc.
Look forward to talking about this next time I see you in person Marcello. Also typing about this here for anyone else that happens to be reading, feel free to find me on Facebook.com/AltonSun to further the conversation.
I did an internship with these guys a couple years ago, and one of the teams was already working on the problem. Ekman’s ideas were specifically brought up as a basic idea of what they were doing, but expanded beyond just microexpressions and the like. Some other things they were looking at included pore-dilation, and thermal imaging.
It wasn’t my team, so I don’t remember too many details, but I remember a problem being that they had to have the subjects immobile in one place, and surrounded by an array of very expensive cameras and sensors.
If you could design a system where you could set up enough sensors to look at everyone in a room, despite the fact that they were moving around, etc, and be able to pick out warning signs for violence (one of their most desired use cases), you’d be in business.
They were sponsored by DARPA or AFRL, but the work was public, so might be able to find some info by browsing around. Also, if they were doing it, I would guess that other colleges were too.
I’m skeptical of this because of my general suspicion of the validity of Paul Ekman’s research program, especially since many of the potential applications of this technology seem related to the shakiest area of the research program—lie detection.
Further, even if you assume Ekman’s research is entirely valid, I’m not sure that video analysis technology can be reliable or effective enough to be useful at the present stage, especially when we’re talking about stimuli that last for such a short period—typically 40-100 ms. Machine vision isn’t my forte either, so I’m not positive, but you would have to be really accurate for something like this to be fully useful, and quite fast as well if you want real-time conversational feedback. It’s also important to consider that having a program like this with a substantial error rate would likely be worse than not having a program at all.
I do think that—if possible—this would be a great idea (indeed, it would represent one of the “killer apps” for Google Glass and similar wearable computing projects if it were effective), but I think both the research behind the idea and the ability to actually implement this in an effective fashion are very shaky at this stage.