Applying Behavioral Psychology on Myself
In which I attempt to apply findings from behavioral psychology to my own life.
Behavioral Psychology Finding #1: Habituation
The psychological process of “extinction” or “habituation” occurs when a stimulus is administered repeatedly to an animal, causing the animal’s response to gradually diminish. You can imagine that if you were to eat your favorite food for breakfast every morning, it wouldn’t be your favorite food after a while. Habituation tends to happen the fastest when the following three conditions are met:
The stimulus is delivered frequently
The stimulus is delivered in small doses
The stimulus is delivered at regular intervals
Source is here.
Applied Habituation
I had a project I was working on that was really important to me, but whenever I started working on it I would get demoralized. So I habituated myself to the project: I alternated 2 minutes of work with 2 minutes of sitting in the yard for about 20 minutes. This worked.
Interestingly enough, about halfway through this exercise I realized that what was really making it difficult for me to work on my project was the fact that it involved so many choices. So as my 20 minutes progressed, I started spending my 2 minutes trying to make as difficult decisions as possible. This habituation to decision demoralization seems to have had an immediate, fairly lasting impact on a wide variety of activities.
I’m really looking forward to hearing from someone who attempts to apply habituation to an ugh field.
Applied Habituation in Reverse
If you want to enjoy your favorite song until the day you die, dance to it infrequently at irregular intervals while it plays full blast. (Reversed conditions for habituation.)
Behavioral Psychology Finding #2: Intermittent Reinforcement
The reason why slot machines are so engaging is because they deliver rewards at random. If slot machines payed small rewards out on every round, playing them would be like work.
Applied Intermittent Reinforcement
For a while, there was a time-consuming chore that I was required to do every evening. I would often put it off until 2-3 AM and work while sleepy as a result.
To solve this problem, I started eating a gummy worm with 50% probability each time I did the chore at a pre-determined time early in the evening. (I gave myself the first two gummy worms with 100% probability to start things off.) My success rate with this method was very high.
Further Research
Another self-help technique I’ve had tremendous success with is using Linux’s cron utility to cause Firefox tabs to open periodically and tell me to switch activities if I’m wasting time. However, I’ve found that forcing myself to switch activities is highly stressful.
Perhaps it’s possible to habituate the negative response to activity switching by having practice sessions where you periodically switch between distraction and work? Or maybe you could use intermittent reinforcement and randomly decide to give yourself something nice if you’re successful in an upgrade to a higher-quality activity.
(I’m not experimenting with these at the moment because I’m currently fairly happy with my work/relaxation balance.)
Thanks to Psychohistorian for reminding me I wanted to write about this. I’m hoping he won’t get mad at me for writing on the same topic he did so soon after his post.
- The Power of Reinforcement by (21 Jun 2012 13:42 UTC; 172 points)
- 's comment on “Stupid” questions thread by (13 Jul 2013 5:36 UTC; 17 points)
- Two Anki plugins to reinforce reviewing (updated) by (3 Dec 2012 22:04 UTC; 17 points)
- [Altruist Support] The Plan by (6 May 2011 3:53 UTC; 13 points)
- 's comment on Reinforcement and Short-Term Rewards as Anti-Akratic by (14 Apr 2013 2:53 UTC; 12 points)
- 's comment on Open Thread: July 2010 by (2 Jul 2010 5:15 UTC; 8 points)
- 's comment on What have you recently tried, and failed at? by (6 Jul 2012 1:28 UTC; 6 points)
- Goals vs. Rewards by (4 Jan 2011 1:43 UTC; 6 points)
- Discuss: Have you experimented with Pavlovian conditioning? by (11 Oct 2010 6:15 UTC; 5 points)
- 's comment on Basics of Animal Reinforcement by (6 Jul 2011 16:21 UTC; 3 points)
- 's comment on Goals vs. Rewards by (4 Jan 2011 2:34 UTC; 3 points)
- 's comment on What have you recently tried, and failed at? by (6 Jul 2012 1:25 UTC; 2 points)
If you have many things to do and you are wasting time, then you should number those things from 1 to n and assign n+1 to wasting time and then use http://random.org to generate a random number between 1 and n+1 (1 and n+1 included) to decide what you should do. This adds some excitement and often works.
I thought I’d share my pick-thing-to-do-at-random app that helps somewhat. You just add things and then it shows you them at random. You can click to commit to do something for a while, or just flick to another thing if you can’t do that now. I’ve added hundreds of both timewasters and productive activities there and it’s quite cool to do this kind of lottery to determine what to do now.
Obviously it won’t work if you just keep flicking until you happen upon a favorite timewaster, nor when you have something that needs to be done now. It’s also essential to have clearly defined activities, even if it’s just “think really hard about what to do about and make that a new activity” or whatever. Tell me what you think.
http://things-be-done.appspot.com/ (google login needed for persistent storage, but you can play without logging in, data will be associated with a cookie left in your browser (and will be transferred once you do login))
i used it for a few hours then it broke, and now all it will do is give me an error dump with this last line: ’BadRequestError: cannot get more than 1000 keys in a single call”
It should work now, please test. Sorry about that problem.
I needed to change the method to pick a random entity—no easy feat in app engine, apparently. As a side effect, there might be some apparent nonuniformity in sampling when you have few todos. It will smooth out as you start/stop them and add more.
Thanks for the report! I’ll look into it.
Pretty neat. Thanks!
Applied intermittent reinforcement results (1 month trial):
Household chores: The only time in the past month that I failed to tidy the kitchen before going to bed was two days ago, when I had a fever of 102 deg F. All other chores have 100% success rate.
Get to bed earlier: 0% success rate, alas.
Success rate in the past month is a little less: 19⁄25 (I’m not counting the five days I was sick). The end of August was particularly bumpy and then I was sick at the start of September, but I’m back on track now.
Thanks for the report, that sounds like a remarkable achievement!
How much of your success do you attribute to placebo related influences?
It’s hard to say, since in this case any kind of blind experimentation would require memory erasure and a conspiracy to prevent me from finding out that variable reinforcement schedules produce the strongest effects. But that same info I’d need to be prevented from knowing makes my prior probability for a placebo effect rather low.
I’ll also add to my results that the technique doesn’t work where I have pre-established Ugh Fields. (In those cases, I couldn’t even get started, so I have no information about what the success rate would be over time.) It’s too late at night for me to try to work through whether this has implications for the plausibility of placebo effects.
I’m interested in what you rewarded for going to bed earlier (or given the 0% success rate, what you planned to reward if it ever happened) and how/when you rewarded it. Maybe rewarding subtasks would have helped.
Each item on my daily to-do list earned me a 50% chance of a small chocolate square. Getting to bed early was one item on the list. Awarding myself the award was the last thing I did each day.
I have mild ocd, habituation is used by me to overcome intrusive thoughts
How do you do it?
It is very helpful that you wrote on the same topic you did so soon after his post. He describes public research on the these topics, and previous results. You took those results, together with suggestions in the comments, and performed a self-experiment (so close in fact that I thought you started this in response to his post). I find your summary more clearly presents the results, while his focuses on the research and I only got some of the background and results from the comments. I particularly was interested in the idea of variable rewards, which I hadn’t thought to associate in a more positive (non-gambling) context.
I hope to use these in everyday life, as I have with many articles here. In fact, how would Less Wrong readers feel about a small self-help pamphlet/novella summarizing the various things posted here? I haven’t encountered a very useful self-help book, although I haven’t looked particularly hard either. Actually, the most results I’d gotten were from The Strategy of Conflict, which is a game theory book about deterrence. The Akrasia Tactics Review already went a decent way toward summarizing many of these.
In case anyone else wants to try the cron thing, here’s what worked on Ubuntu. (Naturally you could also use a repeating timer. Here are a couple vibrating ones.)
First you want to choose a terminal editor if you haven’t already. nano is supposed to be good for beginners. Unfortunately, I wasted the time necessary to learn Vim. (Click-and-type editors beat Vim handily in the editing-speed trials I’ve performed on myself using part of Vim’s own tutorial document.) From the command line:
Then edit your crontab:
And insert the following line:
This should open file.html in Firefox every minute. Be sure to include a newline at the end of the file; that’s supposed to be important. Logging in and out has sometimes jiggled things in to action for me.
Of course, every minute is only good for testing purposes. This should do every fifteen minutes from 9 to 5 Monday through Friday:
More docs here.
I recommend you choose a different font color and background for each distinct message you want to periodically send yourself. This helps the messages function as separate stimuli, and you can stop reading them after a while.
I managed to do the equivalent of this on Windows XP. First, go to
Control Panel > Scheduled Tasks > Add Scheduled Task
A wizard comes up. Click Next, and in the next window select your browser and click Next. Enter a name for the task, select Every Day, and click Next. On the next page make sure the task is set to be performed daily starting from today and the current time, and hit Next. Enter your admin password on the next window and hit Next. On the last window, check the box by “Open advanced properties for this task when I click Finish” and click Finish.
On the window that comes up, there’s a “Run” field that has something like
E:\PROGRA~1\MOZILL~1\firefox.exe
in it (depending on what your browser is and where it’s installed). You need to append “file:///C:/path/to/file.html” to the end of it so that it looks like this:
E:\PROGRA~1\MOZILL~1\firefox.exe file:///C:/path/to/file.html
Finally, go to the Schedule tab, click the “Advanced” button, and on the window that comes up check “Repeat” and have it repeat every 15 minutes (or however frequently you want it to repeat). Also set the Duration to 24 hours.
I use this page for the task, because it amuses me. It uses javascript to randomize the font and background colors, although it’s randomized presets instead of being totally random to ensure that the text is always readable.
If you don’t mind sharing, how has this been going for you?
(I’m really interested to hear how well my life hacks work for other people.)
Not much of an effect so far. The major flaw is that I disable the task whenever I want to do a non-procrastination task, which for me consists of studying math. The math textbooks I have are pdfs, and it’s irritating to have the procrastination page pop up when I’m not procrastinating. However, when my mind does wander while trying to study, I’ll open up a browser window and suddenly I’m procrastinating with no pop-ups to stop me.
A clear workaround would be to buy dead tree textbooks so that this doesn’t happen, but I don’t have the financial resources for that. Perhaps the solution is to resolve to re-enable the task whenever I open a window, but that’s a bit tedious. I may look into making a Firefox addon for this, so that it happens automatically whenever the browser is open. That’s likely the optimal solution (if it’s possible to do).
You could unplug your computer’s ethernet cable when you’re studying.
One thing to watch out for if you try this is getting in the habit of plugging it in real quick whenever you want to do something on the internet. You could mandate a one-minute delay before re-plugging or something like that.
It might not be the specific behavioral-psych-like things you do that are doing the work here.
It might instead be focused attention on “the process” + a ritual + a sense of confidence granted by “science.”
Under this theory, you could swap out the ritual (2 minutes on/off or gummies) with any other ritual, as long as you could make yourself feel confident that it would work, and spent time visualizing and explaining how the arbitrary ritual would be effective.
For example, maybe if I’m having trouble studying on a regular schedule, I could buy a funny hat that I think of as my thinking cap. I decide that the thinking cap will make me want to study. And I do whatever explanatory/visualizing tricks I need for about an hour or two to endow the thinking cap hypothesis with an aura of reality.
Under this hypothesis, the real reason why these specific behavioral techniques are so powerful is that it’s easy for you to construct that compelling narrative, because you are a believer in science, and this seems like that.
A scientific skeptic would have low success with these techniques, but high success with techniques that have been blessed by their preferred sources of wisdom.
If true, others wanting to emulate your experiment would actually not want to just copy you, unless they found your specific ritual to seem highly compelling. Even if they did, they’d want to spend about as much time as you did on visualizing/explaining and experimenting with it. And if they didn’t like your ritual, they’d want to first search for one they find more compelling.
This sort of conditioning works best when the reward is administered within about 500 ms of the response (sorry, don’t have a citation). Something to keep in mind.
How do you apply that to cases where the response is a task that can stretch out over an hour?
You don’t. You take the response to be ‘starting’ to work on that task instead.
I’m not sure. You may not be able to in any feasible or satisfactory way, which was sort of my point.
Thanks!
That sounds like an extremely interesting process. I’ll have to try that myself. Most appreciated.
Nice post, John. Please post more on those topics.
Question: not sure I understood exactly what you meant by “I started eating a gummy worm with 50% probability each time I did the chore at a pre-determined time early in the evening.”
cough technically....
If I am not mistaken… every time he finished a chore that he had scheduled he tossed a (metaphoric or literal) coin. If heads, he ate a gummy worm but if tails he did not eat a gummy worm.
Correct. I used this website.
You know when I saw that it allowed you to choose which kind of coin to flip I thought it was going to be cute and actually implement a trivial bias based on studies on how the various coins are weighted. (I checked, it doesn’t.)
Why did you choose 50% chance of reward of the gummy, and not 100%? Or half a gummy?
There’s a lot of evidence that randomized reward schemes are more addictive than non-randomized reward schemes. This is one major reason why gambling is so addictive.
In addition, a guaranteed reward could create an anchoring on the reward, devaluing it.
Agree and add reference -
This is something that will be covered in any foundational undergraduate (or high school) psychology subject.
Reinforcement, punishment and ‘extinction’ (which is not giving any response till the training goes away). When covered in depth there are all sorts of schedules for that can be used for reinforcement. What we are considering here is Variable Ratio, in this case 1⁄2. When considering the effectiveness of reinforcement schedules:
(To summarize my upcoming point in tl;dr form: if you don’t find yourself rationalizing “maybe I’m onto the pattern” while your stomach rumbles as you contemplate the upside of getting gummi bears marginally more often, you might be tickling a different variety-seeking mechanism than you think. Nothing wrong with that, but if you want to get really good at optimizing that tickle, detailed knowledge about which mechanism it is might be helpful.)
From time to time when reading technical articles related to effective strategies for artificial agents faced with “n-armed bandit” problems, I am reminded of observed animal behavior patterns like PREE, and wonder how close the correspondence might be. N-armed bandits have been studied for a long time, and it seems like an obvious conjecture, but I have never seen much analysis of this. I never encounter such analysis spontaneously when people talk about a particular psych observation, even at ML-friendly sites like LW. And hunting for it with e.g. Google “partial reinforcement n-armed bandit” suggests that it must be a pretty obscure topic, because in the articles I find, the analysis I am looking for is swamped by different topics like reinforcement learning, and obscure topics like how a web designer trying to optimize humans’ response to the website can usefully think of his website A/B testing as an n-armed bandit problem.
Can anyone recommend systematic attempts to explore how close this correspondence might be?
Of the usual pop psychology examples of overresponse to partial reinforcement, it looks to me as though gambling truly is narrowly tuned to the PREE phenomenon, and is working essentially by fooling an agent designed to solve a bandit problem. Other examples, however, tend to be sufficiently ambiguous or contradictory in various ways that I thinksomething unrelated could be going on. Humans can respond to variety in all sorts of positive ways. E.g., (dammit, I’m going blank on the name of) the classic confounding effect in industrial productivity studies where change itself, in either direction, can easily cause a positive effect independent of whether the new situation is objectively better in any useful sense.
Notice that successful gambling operations are contrived so that if you ever did discover even a small pattern (e.g., 53% success instead of 48%) it follows by perfectly correct analysis that the discovery would be enormously valuable. Under such extreme conditions, even a small nudge from a simple n-armed bandit heuristic (like a nagging intuition corresponding to a high prior probability that high variance implies a significant probability of discovering something that improves performance by a mere 10%) can get amplified to dramatically wrong behavior. Also notice that there is a strong observed pattern of compulsive gamblers fooling themselves into thinking they are finding small patterns. If gambling were a case of partial reinforcement directly tickling purely subconscious deep structures unrelated to n-armed banditry, then “I’m onto the pattern” might still sometimes be used to rationalize the irrational behavior, but it’s not clear why it would be a strongly favored rationalization (compared to, e.g., “risk taking makes me glamorous”).
Compare this to behavior patterns that aren’t observed. E.g., I’ve never heard of anyone making cigarettes qualitatively more addictive by making them unpredictable, e.g., by selling mixed packs of placebo and nicotinic cigarettes. Could this be because there’s no way for the robot to get rationally excited about the enormous upside of spotting a small pattern in such randomness? (And anything close to this which does succeed, e.g. toy prizes in cereal boxes, tends to be successful for only about as long as the robot’s inputs from the world model let it be strongly uncertain about the upside..)
People do claim to spot partial-reinforcement-related phenomena in other behavior patterns which can’t easily be explained as a bandit problem heuristic being tricked. E.g., people often accuse World of Warcraft and similar games of manipulating the intermittent reward mechanism to cause addictive behavior. WoW treasures are indeed randomized, and people do indeed become fascinated by the game, and I don’t see how the robot could be getting excited about a huge upside of spotting the pattern. But WoW is in the entertainment industry. WoW developers could have saved a substantial amount of money by hiring far fewer artists to create far fewer kinds of trees and other decorations, but that it would be a bad idea. Hollywood could save even more money by aggressively reusing sets and actors and props and scripts between movies. Even in extremes like soap operas where many customers are looking for repetitive essentially-predictable escape, a successful entertainment product benefits from many kinds of variety. It seems to me that the positive importance of randomizing treasures needn’t be explained by partial reinforcement any more than the positive importance, in a soap opera in which villains walk onto the stage in hundreds of different episodes, of avoiding a clear pattern of villains entering stage right every single time.
This is an interesting subject. Are there any other posts like this? My bigest flaw is procrastination :(
This is an interesting subject. Are there any other posts like this? My bigest flaw is procrastination :(
That was a damn good article!
It was short, to the point, and based on real data, and useful as well. So unlike the polite verbiage of karma whores. Even William of Ockham would have been proud of you.
Kim0+