they might still be more effective than nothing at all.
They might. They are likely less effective than nothing at all. Your advice about “Do not stand out” has the possibility of prevent people from starting blogs and take a public stance for their own values.
The are not good guidelines.
Let’s say a person in the EA/rationality space get’s publically attacked. What should be their first response?
That’s a trick question. Don’t do anything public while being in the emotion triggered by the attack.
Seek out to other people in the community that have media experience and contact them privately.
Do goal-factoring yourself about what’s important in the situation. If you have a specific response in mind check with other people. In addition to EA/rationality people ask outsiders for their opinion.
Respond publically only after you checked with other people and are in a state to think clearly about the issue. Do not think that you can handle the situation optimally yourself without outside input by reading a list with ten rules or even reading the full Art of War.
Use Goal-factoring and Hemming circle’s. Basic rationality techniques.
It’s a CFAR concept that’s explained at their workshops but which hasn’t been elaborated anywhere online, to my knowledge. (I take it you haven’t been to a CFAR workshop yet, by the way? I think that their stuff and yours would have amazing amounts of synergy potential.)
The idea is that you have a small group of people, about 3 or 4, and then you go around the circle and each person gets to discuss their “Hamming question”, named after Richard Hamming who’d go around asking other scientists, “what are the most important problems in your field, and why aren’t you working on them”. The Hamming question’s the application of that to your life—“what’s the biggest problem in your life, and what would it take to solve it”. The other people are there to help the person who’s trying to figure out their Hamming question, with e.g. probing questions and suggestions.
Hamming circles are a special case of “debugging circles”, where the idea (3-4 people in circle, discussing each person’s issue in turn) is the same, but you’re also allowed to use the circle to discuss something that’s not actually your Hamming question, but something smaller. I think Christian might have been thinking about debugging circles in general.
Here’s a set of guidelines for running debugging circles (though I prefer to call them something like “growth circles” or “development circles”) that I came up with after some experience with them:
Etiquette
Only bring up something if you actually want a solution for it. Everyone has times when they just need to vent and want sympathy rather than solutions, but development circles aren’t the place for that. This doesn’t mean that you would need to accept any suggestion that the others bring up, but it does mean that that you should be open to others offering suggestions in general. Once the session ends, you’re free to just ignore and forget anything that didn’t seem to make sense to you.
Be courteous of others and their time. Do your best to make sure that everyone, both you and the others, get a roughly equal share of the group’s attention. If you have lots of problems in your life, don’t dump all of them on the group at once, but rather focus on one or a small set of related ones. If it starts looking like the discussion has gotten stuck on one person’s issues for an extended time and the others might not have a chance to have their issues discussed, gently but firmly suggest moving on to the next person. Try to be considerate enough to pass on your own turn early enough that someone else doesn’t need to prompt you to do so.
If someone is undergoing a particularly difficult time or has a particularly important issue going on, it’s alright to sometimes spend a disproportionate time on them: but you should try to avoid being that person each time.
But have a fair respect of your own time, as well. The opposite also applies: if you genuinely feel that there’s nothing in your life that needs discussing, you’re free to cut your own turn short, but if you do it many occasions in a row, you’re probably not taking full advantage of the group. If nothing else, you can always use the group to get an opinion on any assumptions behind your current plans. While you don’t want to go overtime on your turn, you also don’t want to cut your own turn short. You have a right to get help from the group in return for helping others: stick to that right.
Don’t proselytize your view. Maybe you’re completely certain that the cause of the other person’s problems is that they don’t have cat ears as a part of their attire, which would totally fix everything if they just changed that. You’re free to think that, but if they disagree with your suggestion, don’t get stuck arguing but let it go.
Approaches
Start by trying to understand what problem the person is trying to solve. “I’ve been trying to sign up for dance lessons but can never seem to get around it.” One possible approach would be to immediately start offering ways for the person to sign up for dance lessons. Often a more fruitful one would be to first ask—why do you want to attend dance lessons? Maybe it turns out that the person doesn’t actually care about learning to dance, but is feeling bad because their friend, a great dancer, always gets all the attention at parties. Then the actual problem is not “how to learn to dance” but “how to get other people to notice me”. It’s quite possible that not knowing to dance isn’t actually the biggest issue there.
Test your understanding of the problem. When you’re formulating an understanding of the problem, it can be useful to frequently verbalize it to the other person to make sure that you’ve understood correctly. “So you seem to be feeling bad because your girlfriend just became the President of your country while you mostly spend time playing video games, is that right?”
A rule of thumb that I sometimes use is “do I feel like I understand this problem and its causes well enough that I could explain to a third person why this person wants to solve it and why they haven’t been able to solve it yet?” If the answer is no, try asking more questions first.
Even “obvious” problems may benefit from questions. Someone once mentioned that they tend to often jump to being critical of others, which tends to be harmful. Here the causal mechanism seemed to be pretty obvious, but asking “how does it tend to be harmful” was still useful in bringing out details of the exact nature of the typical criticism and how people tended to react to that.
Look for trigger-action patterns. “I always end up being on the computer and wasting time and then feeling bad.” What specific things on the computer act as time-wasters, and how exactly does the person end up doing those things? Maybe they often feel bored or anxious, which causes them to open Facebook, which causes them to get lost in a maze of discussions and links. Would there be a way to either remove the anxiety, or find a new action to carry out when anxious? Which one would be easier?
Look for positive and negative reinforcers in the environment. “I often post a link to Facebook, and then I keep returning to Facebook throughout the day because I want to see whether it’s accumulated new likes and comments.” Here, logging on to Facebook after posting a link keeps getting reinforced by the accumulation of comments and likes, which provide a reward each time that the page is opened and there’s a new one. Could something be done to eliminate those reinforcers? (“If anyone sees me responding to a comment or posting a link, don’t like it but do remind me that I was supposed to be working.”) Or maybe provide reinforcers for something else? (“For each ten minutes that passes without me logging onto Facebook, could you please come give me a hug?”)
Be specific about the causes of emotional reactions. “My boss is so full of himself, it drives me nuts.” Exactly how does the full-of-himself-ness manifest? If the exact behavior is “he often interrupts”, maybe something could be done about that thing in particular. Best case: the boss comes from a conversational culture where interrupting is normal, and hasn’t even realized that someone would consider it rude—but this would have been impossible for the others to suggest if the problem description would only have been on the level of “he’s so full of himself”.
This is also a useful technique for reducing your own annoyance at others, even if it was just something you did in your head. “I’m getting frustrated now because that person is talking really loudly and I would like to read.” Breaking down an atomic “AAAAAGH I’M SO FRUSTRATED” into a “I’m feeling [specific emotion] because [specific cause] and [that violates my desire/need to something]” is not only useful for debugging, it can also relieve the frustration by itself.
Assume that problems won’t fix themselves. In one session, someone says they intend to implement some change for next week’s meeting. In the next session, they say, “yeah, that plan didn’t really work out, but I was kinda busy and distracted this week. I’m going to try harder.”
Chances are, if they were busy and distracted this week, they’re likely to be busy and distracted the next week, too. “I’m going to try harder” often translates either as “I don’t actually care about solving this problem but want to give the impression that I do”, or alternatively, “I don’t actually know how to fix this but I’m going to try again the same way, in the hopes of magically getting a different result now”. Assuming that the person really does want to solve their problem, try to figure out exactly what went wrong and how it could be avoided in the future.
Ask, “is there a more general problem here?” Someone wants to cut down on the amount of money that they spend on fast food. One day when they’re coming home from work they walk past a hamburger place, are tempted by the advertisements, and go there to eat. This happens several times.
The specific problem in this case would be “I always end up eating at the Burger King on the 27th street on my way home”. The more general form of the problem might be something like “each time I walk past a fast food place when I’m hungry, I end up eating there”. General solutions might be “pick a route that allows you to avoid seeing fast food places when you’re hungry” and “make sure to carry something with you that allows you stave off the worst of the hunger until you’re home”.
Focusing. Someone is having difficulties deciding whether to try to solve a problem or whether to accept its consequences and let it be. One approach would be to have them verbalize all the reasons why the unsolved issue bothers them, and then say out loud, “having considered all of these consequences of the problem, I find that they’re acceptable and it’s better to just let this be”. Does saying that feel right to them, or does something about it feel wrong? What if they were to say, “having considered all of these consequences of the problem, I find that they’re unacceptable and I want to solve the problem”, instead? Would that feel right or wrong?
Quick Murphyjitsu. After you’ve come up with a plan, it may be useful to have the other person do a quick Murphyjitsu on it. How surprised would they be if this plan failed? If not particularly, is there any obvious failure mode that comes to mind and which could be fixed?
Check that the person remembers something actionable. Sometimes discussion may suggest some actionable things, then drift to e.g. more general discussion of the problem which doesn’t provide as many concrete suggestions. If this happens, make sure that the person whose problems are being debugged still remembers the actionable suggestions they got earlier on.
Look for positive and negative reinforcers in the environment. “I often post a link to Facebook, and then I keep returning to Facebook throughout the day because I want to see whether it’s accumulated new likes and comments.” Here, logging on to Facebook after posting a link keeps getting reinforced by the accumulation of comments and likes, which provide a reward each time that the page is opened and there’s a new one. Could something be done to eliminate those reinforcers? (“If anyone sees me responding to a comment or posting a link, don’t like it but do remind me that I was supposed to be working.”) Or maybe provide reinforcers for something else? (“For each ten minutes that passes without me logging onto Facebook, could you please come give me a hug?”)
Be specific about the causes of emotional reactions. “My boss is so full of himself, it drives me nuts.” Exactly how does the full-of-himself-ness manifest? If the exact behavior is “he often interrupts”, maybe something could be done about that thing in particular. Best case: the boss comes from a conversational culture where interrupting is normal, and hasn’t even realized that someone would consider it rude—but this would have been impossible for the others to suggest if the problem description would only have been on the level of “he’s so full of himself”.
This is also a useful technique for reducing your own annoyance at others, even if it was just something you did in your head. “I’m getting frustrated now because that person is talking really loudly and I would like to read.” Breaking down an atomic “AAAAAGH I’M SO FRUSTRATED” into a “I’m feeling [specific emotion] because [specific cause] and [that violates my desire/need to something]” is not only useful for debugging, it can also relieve the frustration by itself.
Assume that problems won’t fix themselves. In one session, someone says they intend to implement some change for next week’s meeting. In the next session, they say, “yeah, that plan didn’t really work out, but I was kinda busy and distracted this week. I’m going to try harder.”
Chances are, if they were busy and distracted this week, they’re likely to be busy and distracted the next week, too. “I’m going to try harder” often translates either as “I don’t actually care about solving this problem but want to give the impression that I do”, or alternatively, “I don’t actually know how to fix this but I’m going to try again the same way, in the hopes of magically getting a different result now”. Assuming that the person really does want to solve their problem, try to figure out exactly what went wrong and how it could be avoided in the future.
Ask, “is there a more general problem here?” Someone wants to cut down on the amount of money that they spend on fast food. One day when they’re coming home from work they walk past a hamburger place, are tempted by the advertisements, and go there to eat. This happens several times.
The specific problem in this case would be “I always end up eating at the Burger King on the 27th street on my way home”. The more general form of the problem might be something like “each time I walk past a fast food place when I’m hungry, I end up eating there”. General solutions might be “pick a route that allows you to avoid seeing fast food places when you’re hungry” and “make sure to carry something with you that allows you stave off the worst of the hunger until you’re home”.
Focusing. Someone is having difficulties deciding whether to try to solve a problem or whether to accept its consequences and let it be. One approach would be to have them verbalize all the reasons why the unsolved issue bothers them, and then say out loud, “having considered all of these consequences of the problem, I find that they’re acceptable and it’s better to just let this be”. Does saying that feel right to them, or does something about it feel wrong? What if they were to say, “having considered all of these consequences of the problem, I find that they’re unacceptable and I want to solve the problem”, instead? Would that feel right or wrong?
Quick Murphyjitsu. After you’ve come up with a plan, it may be useful to have the other person do a quick Murphyjitsu on it. How surprised would they be if this plan failed? If not particularly, is there any obvious failure mode that comes to mind and which could be fixed?
Check that the person remembers something actionable. Sometimes discussion may suggest some actionable things, then drift to e.g. more general discussion of the problem which doesn’t provide as many concrete suggestions. If this happens, make sure that the person whose problems are being debugged still remembers the actionable suggestions they got earlier on.
They might. They are likely less effective than nothing at all. Your advice about “Do not stand out” has the possibility of prevent people from starting blogs and take a public stance for their own values.
The are not good guidelines.
Let’s say a person in the EA/rationality space get’s publically attacked. What should be their first response?
That’s a trick question. Don’t do anything public while being in the emotion triggered by the attack.
Seek out to other people in the community that have media experience and contact them privately. Do goal-factoring yourself about what’s important in the situation. If you have a specific response in mind check with other people. In addition to EA/rationality people ask outsiders for their opinion.
Respond publically only after you checked with other people and are in a state to think clearly about the issue. Do not think that you can handle the situation optimally yourself without outside input by reading a list with ten rules or even reading the full Art of War.
Use Goal-factoring and Hemming circle’s. Basic rationality techniques.
I’ve at least heard of goal-factoring. What is/are “Hemming circle’s”? Google only turns up articles about sewing.
Sorry, it’s Hamming and not Hemming. (I often say the term and seldom write it)
It’s basically getting a few people together to speak through a problem that one person has.
Searching for “Hamming circle” yields only information about hamming codes. Do you have a link?
It’s a CFAR concept that’s explained at their workshops but which hasn’t been elaborated anywhere online, to my knowledge. (I take it you haven’t been to a CFAR workshop yet, by the way? I think that their stuff and yours would have amazing amounts of synergy potential.)
The idea is that you have a small group of people, about 3 or 4, and then you go around the circle and each person gets to discuss their “Hamming question”, named after Richard Hamming who’d go around asking other scientists, “what are the most important problems in your field, and why aren’t you working on them”. The Hamming question’s the application of that to your life—“what’s the biggest problem in your life, and what would it take to solve it”. The other people are there to help the person who’s trying to figure out their Hamming question, with e.g. probing questions and suggestions.
Hamming circles are a special case of “debugging circles”, where the idea (3-4 people in circle, discussing each person’s issue in turn) is the same, but you’re also allowed to use the circle to discuss something that’s not actually your Hamming question, but something smaller. I think Christian might have been thinking about debugging circles in general.
Here’s a set of guidelines for running debugging circles (though I prefer to call them something like “growth circles” or “development circles”) that I came up with after some experience with them:
Etiquette
Only bring up something if you actually want a solution for it. Everyone has times when they just need to vent and want sympathy rather than solutions, but development circles aren’t the place for that. This doesn’t mean that you would need to accept any suggestion that the others bring up, but it does mean that that you should be open to others offering suggestions in general. Once the session ends, you’re free to just ignore and forget anything that didn’t seem to make sense to you.
Be courteous of others and their time. Do your best to make sure that everyone, both you and the others, get a roughly equal share of the group’s attention. If you have lots of problems in your life, don’t dump all of them on the group at once, but rather focus on one or a small set of related ones. If it starts looking like the discussion has gotten stuck on one person’s issues for an extended time and the others might not have a chance to have their issues discussed, gently but firmly suggest moving on to the next person. Try to be considerate enough to pass on your own turn early enough that someone else doesn’t need to prompt you to do so.
If someone is undergoing a particularly difficult time or has a particularly important issue going on, it’s alright to sometimes spend a disproportionate time on them: but you should try to avoid being that person each time.
But have a fair respect of your own time, as well. The opposite also applies: if you genuinely feel that there’s nothing in your life that needs discussing, you’re free to cut your own turn short, but if you do it many occasions in a row, you’re probably not taking full advantage of the group. If nothing else, you can always use the group to get an opinion on any assumptions behind your current plans. While you don’t want to go overtime on your turn, you also don’t want to cut your own turn short. You have a right to get help from the group in return for helping others: stick to that right.
Don’t proselytize your view. Maybe you’re completely certain that the cause of the other person’s problems is that they don’t have cat ears as a part of their attire, which would totally fix everything if they just changed that. You’re free to think that, but if they disagree with your suggestion, don’t get stuck arguing but let it go.
Approaches
Start by trying to understand what problem the person is trying to solve. “I’ve been trying to sign up for dance lessons but can never seem to get around it.” One possible approach would be to immediately start offering ways for the person to sign up for dance lessons. Often a more fruitful one would be to first ask—why do you want to attend dance lessons? Maybe it turns out that the person doesn’t actually care about learning to dance, but is feeling bad because their friend, a great dancer, always gets all the attention at parties. Then the actual problem is not “how to learn to dance” but “how to get other people to notice me”. It’s quite possible that not knowing to dance isn’t actually the biggest issue there.
Test your understanding of the problem. When you’re formulating an understanding of the problem, it can be useful to frequently verbalize it to the other person to make sure that you’ve understood correctly. “So you seem to be feeling bad because your girlfriend just became the President of your country while you mostly spend time playing video games, is that right?”
A rule of thumb that I sometimes use is “do I feel like I understand this problem and its causes well enough that I could explain to a third person why this person wants to solve it and why they haven’t been able to solve it yet?” If the answer is no, try asking more questions first.
Even “obvious” problems may benefit from questions. Someone once mentioned that they tend to often jump to being critical of others, which tends to be harmful. Here the causal mechanism seemed to be pretty obvious, but asking “how does it tend to be harmful” was still useful in bringing out details of the exact nature of the typical criticism and how people tended to react to that.
Look for trigger-action patterns. “I always end up being on the computer and wasting time and then feeling bad.” What specific things on the computer act as time-wasters, and how exactly does the person end up doing those things? Maybe they often feel bored or anxious, which causes them to open Facebook, which causes them to get lost in a maze of discussions and links. Would there be a way to either remove the anxiety, or find a new action to carry out when anxious? Which one would be easier?
(comment length limit hit, continued in next comment)
(comment continued)
Look for positive and negative reinforcers in the environment. “I often post a link to Facebook, and then I keep returning to Facebook throughout the day because I want to see whether it’s accumulated new likes and comments.” Here, logging on to Facebook after posting a link keeps getting reinforced by the accumulation of comments and likes, which provide a reward each time that the page is opened and there’s a new one. Could something be done to eliminate those reinforcers? (“If anyone sees me responding to a comment or posting a link, don’t like it but do remind me that I was supposed to be working.”) Or maybe provide reinforcers for something else? (“For each ten minutes that passes without me logging onto Facebook, could you please come give me a hug?”)
Be specific about the causes of emotional reactions. “My boss is so full of himself, it drives me nuts.” Exactly how does the full-of-himself-ness manifest? If the exact behavior is “he often interrupts”, maybe something could be done about that thing in particular. Best case: the boss comes from a conversational culture where interrupting is normal, and hasn’t even realized that someone would consider it rude—but this would have been impossible for the others to suggest if the problem description would only have been on the level of “he’s so full of himself”.
This is also a useful technique for reducing your own annoyance at others, even if it was just something you did in your head. “I’m getting frustrated now because that person is talking really loudly and I would like to read.” Breaking down an atomic “AAAAAGH I’M SO FRUSTRATED” into a “I’m feeling [specific emotion] because [specific cause] and [that violates my desire/need to something]” is not only useful for debugging, it can also relieve the frustration by itself.
Assume that problems won’t fix themselves. In one session, someone says they intend to implement some change for next week’s meeting. In the next session, they say, “yeah, that plan didn’t really work out, but I was kinda busy and distracted this week. I’m going to try harder.”
Chances are, if they were busy and distracted this week, they’re likely to be busy and distracted the next week, too. “I’m going to try harder” often translates either as “I don’t actually care about solving this problem but want to give the impression that I do”, or alternatively, “I don’t actually know how to fix this but I’m going to try again the same way, in the hopes of magically getting a different result now”. Assuming that the person really does want to solve their problem, try to figure out exactly what went wrong and how it could be avoided in the future.
Ask, “is there a more general problem here?” Someone wants to cut down on the amount of money that they spend on fast food. One day when they’re coming home from work they walk past a hamburger place, are tempted by the advertisements, and go there to eat. This happens several times.
The specific problem in this case would be “I always end up eating at the Burger King on the 27th street on my way home”. The more general form of the problem might be something like “each time I walk past a fast food place when I’m hungry, I end up eating there”. General solutions might be “pick a route that allows you to avoid seeing fast food places when you’re hungry” and “make sure to carry something with you that allows you stave off the worst of the hunger until you’re home”.
Focusing. Someone is having difficulties deciding whether to try to solve a problem or whether to accept its consequences and let it be. One approach would be to have them verbalize all the reasons why the unsolved issue bothers them, and then say out loud, “having considered all of these consequences of the problem, I find that they’re acceptable and it’s better to just let this be”. Does saying that feel right to them, or does something about it feel wrong? What if they were to say, “having considered all of these consequences of the problem, I find that they’re unacceptable and I want to solve the problem”, instead? Would that feel right or wrong?
Quick Murphyjitsu. After you’ve come up with a plan, it may be useful to have the other person do a quick Murphyjitsu on it. How surprised would they be if this plan failed? If not particularly, is there any obvious failure mode that comes to mind and which could be fixed?
Check that the person remembers something actionable. Sometimes discussion may suggest some actionable things, then drift to e.g. more general discussion of the problem which doesn’t provide as many concrete suggestions. If this happens, make sure that the person whose problems are being debugged still remembers the actionable suggestions they got earlier on.
This should be a top level post, probably in main.
(comment continued)
Look for positive and negative reinforcers in the environment. “I often post a link to Facebook, and then I keep returning to Facebook throughout the day because I want to see whether it’s accumulated new likes and comments.” Here, logging on to Facebook after posting a link keeps getting reinforced by the accumulation of comments and likes, which provide a reward each time that the page is opened and there’s a new one. Could something be done to eliminate those reinforcers? (“If anyone sees me responding to a comment or posting a link, don’t like it but do remind me that I was supposed to be working.”) Or maybe provide reinforcers for something else? (“For each ten minutes that passes without me logging onto Facebook, could you please come give me a hug?”)
Be specific about the causes of emotional reactions. “My boss is so full of himself, it drives me nuts.” Exactly how does the full-of-himself-ness manifest? If the exact behavior is “he often interrupts”, maybe something could be done about that thing in particular. Best case: the boss comes from a conversational culture where interrupting is normal, and hasn’t even realized that someone would consider it rude—but this would have been impossible for the others to suggest if the problem description would only have been on the level of “he’s so full of himself”.
This is also a useful technique for reducing your own annoyance at others, even if it was just something you did in your head. “I’m getting frustrated now because that person is talking really loudly and I would like to read.” Breaking down an atomic “AAAAAGH I’M SO FRUSTRATED” into a “I’m feeling [specific emotion] because [specific cause] and [that violates my desire/need to something]” is not only useful for debugging, it can also relieve the frustration by itself.
Assume that problems won’t fix themselves. In one session, someone says they intend to implement some change for next week’s meeting. In the next session, they say, “yeah, that plan didn’t really work out, but I was kinda busy and distracted this week. I’m going to try harder.”
Chances are, if they were busy and distracted this week, they’re likely to be busy and distracted the next week, too. “I’m going to try harder” often translates either as “I don’t actually care about solving this problem but want to give the impression that I do”, or alternatively, “I don’t actually know how to fix this but I’m going to try again the same way, in the hopes of magically getting a different result now”. Assuming that the person really does want to solve their problem, try to figure out exactly what went wrong and how it could be avoided in the future.
Ask, “is there a more general problem here?” Someone wants to cut down on the amount of money that they spend on fast food. One day when they’re coming home from work they walk past a hamburger place, are tempted by the advertisements, and go there to eat. This happens several times.
The specific problem in this case would be “I always end up eating at the Burger King on the 27th street on my way home”. The more general form of the problem might be something like “each time I walk past a fast food place when I’m hungry, I end up eating there”. General solutions might be “pick a route that allows you to avoid seeing fast food places when you’re hungry” and “make sure to carry something with you that allows you stave off the worst of the hunger until you’re home”.
Focusing. Someone is having difficulties deciding whether to try to solve a problem or whether to accept its consequences and let it be. One approach would be to have them verbalize all the reasons why the unsolved issue bothers them, and then say out loud, “having considered all of these consequences of the problem, I find that they’re acceptable and it’s better to just let this be”. Does saying that feel right to them, or does something about it feel wrong? What if they were to say, “having considered all of these consequences of the problem, I find that they’re unacceptable and I want to solve the problem”, instead? Would that feel right or wrong?
Quick Murphyjitsu. After you’ve come up with a plan, it may be useful to have the other person do a quick Murphyjitsu on it. How surprised would they be if this plan failed? If not particularly, is there any obvious failure mode that comes to mind and which could be fixed?
Check that the person remembers something actionable. Sometimes discussion may suggest some actionable things, then drift to e.g. more general discussion of the problem which doesn’t provide as many concrete suggestions. If this happens, make sure that the person whose problems are being debugged still remembers the actionable suggestions they got earlier on.