Take the sentence “Moreover, research on poly relationships show that people with consensual non-monogamous relationships are happier, especially with their sex lives, than those who are in traditional monogamous relationships, and communicate more openly.”
“Research” links to no peer reviewed paper. “Happy” links to another mainstream media article. “Communicate more openly” a published paper but that published paper isn’t a decent study that compared poly people with nonpoly people but it’s about a focus group discussion among poly women.
If you want to get people engaged with research than cite a bunch of papers directly and discuss them.
We think it’s quite appropriate to direct readers to an article that cites many research-based papers, as opposed to citing the papers themselves—it’s a pretty efficient goal factoring approach.
Moreover, the large majority of our target audience for the blog posts—people who are early onward in the process of gaining more rational thinking—would be unlikely to read research studies in-depth, and would be much more likely to read articles informed by research studies. So please keep that in mind as you read the Intentional Insights website. Our goal is to raise the sanity waterline by translating complex content for broad audiences.
In the Intentional Insights article about polyamory, the word “research” links to the following article in Psychology Today that cites a wide variety of research papers. The word “happy” also links to a similar article
Part of rationality is not trusting a mainstream media article to accurately represent the state of scientific knowledge. To the extend that you implicitly try to teach that a post titled “Open Relationships Reduce Jealousy? 12 Surprising Facts About Non-Monogamy” is equivalent to peer reviewed research because it includes citations, you aren’t raising the sanity waterline.
You are not discussing the arguments for whether or not polyamory raises happiness.
The argument in that article is also pretty stupid. Just because a poll that only targeted Swingers shows that the Swingers in that study were more happy than average census takers doesn’t mean you can generalize.
That’s no controlled study setup.
If you actually want to teach rationality than you should teach people not to trust claims made based on non-controlled observational studies. At least I would guess that’s what most people on LW would expect from a project that tries to teach rationality.
The other paper towards which you link directly contains the nice phrase: “The research aimed to explore issues around gender in non-monogamous relationships and to explore the potentials of participant-led methods to conduct research into this aspect of women’s sexuality with a qualitative, feminist framework.”
If you are serious about that, write your next article about why someone who wants to be a rationalist should take research like that seriously.
Our goal is to raise the sanity waterline by translating complex content for broad audiences.
So your goal isn’t to teach critical thinking but do the critical thinking yourself an then give it’s results to your audience?
I’m curious, do you believe that the goal of translating complex content for broad audiences is a value-less endeavor? If so, then I accept we have a difference of opinion.
As part of doing so, we believe that raising the sanity waterline requires creating cognitive ease for audiences who do not yet have advanced rational analysis skills. If we disagree, then I accept we have a difference of opinion.
I don’t have any issue with someone trying to change beliefs of society. That are many cases where belief change is useful.
On the other hand that’s not the same thing as teaching people critical thinking or raising the sanity waterline.
Eliezer counts ”...what constitutes evidence and why;” as one of the things of raising the sanity waterline. By teaching people that they should treat uncontrolled surveys and research done in a participant-led, qualitative, feminist framework as good evidence you are part of the problem and not part of the solution as far as raising the sanity waterline goes.
Deciding that you are rational and therefore can see that polyamory should be better respected in society and then using whatever way to convince people whether or not that way has anything to do with rational argument isn’t what I consider raising the sanity waterline.
research done in a participant-led, qualitative, feminist framework as good evidence you are part of the problem and not part of the solution as far as raising the sanity waterline goes
Can you clarify why you believe that such research is unacceptable as a form of Level III evidence? After all, some evidence is better than no evidence for actually changing one’s mind. Polyamory research is a very new field, and descriptive studies are pretty much the best there is at this stage. But if you find other relevant research on polyamory that we missed, I would be happy to update my beliefs.
Can you clarify why you believe that such research is unacceptable as a form of Level III evidence?
I don’t see a good reason to believe that it’s representative of all polyamorous people. There’s a certain scene that very much values openness but there are other people who live polyamorously and who aren’t very open. Just because the feminist who runs the study is herself very open in her communication and interacts with others who are also very open doesn’t mean that everybody is.
You also did present it as fact in your article and especially if you are writing for a naive audience telling them about the uncertainty is useful if you want to train critical thinking. Skepticism is a default for critical thinking. If there no evidence either way, than there no evidence.
Feynman held his cargo-cult science speech about reasoning with better quality.
If you also sincerely believe that we should put more weight on science done with an explicit feminist framework that’s a position worthy writing an article about because I guess that you are holding a minority position under self-labeled rationalists.
A core question of rationalism is: “Why do we believe what we believe? In this case you don’t tell the reader why you think the particular material you reference should make him believe that poly people are happy and open.
I don’t see a good reason to believe that it’s representative of all polyamorous people.
I’m curious why you attribute that position to our blog post—that’s the opposite of a steel man move. In fact, the Intentional Insights blog post did not make the claim you seem to attribute to it. What we claimed was the following, which I am quoting from the blog post: “research on poly relationships show that people with consensual non-monogamous relationships are happier, especially with their sex lives, than those who are in traditional monogamous relationships, and communicate more openly.”
If you have research—any sort of research—demonstrating otherwise, I would be happy to update my beliefs. I think we can agree that the quality of evidence should be the deciding factor :-)
I think the common meaning of the phrase “research shows X” is “research shows X is true”.
If that’s not what you want to argue you can use phrases like “research suggests X”.
If the thing you want to teach is evidence based reasoning it would be still useful to explain a naive audience the strength of the evidence.
A post either has mistakes or it doesn’t. The point of steelmanning is to change someone’s argument to make it better. I’m not denying that it’s possible to steelman the claim that polyamory increases happiness or openness but that’s not what I care about here. The thing that matter is whether or not this post encourages critical thinking. Using “show” when you mean “suggest” doesn’t help for that goal.
I agree that the common meaning of the phrase “research shows X” is “research shows X is true.” However, can you clarify to me where you believe the post makes mistakes in its use of “show” vs. “suggest”?
The statement I made in the salient blog post on the Intentional Insights website was as follows “research on poly relationships show that people with consensual non-monogamous relationships are happier.” I believe that the evidence I cited there supports the notion that “research shows X is true” when X = “on poly relationships shows that people with consensual non-monogamous relationships are happier.” For examples of such evidence, see this article, or this book or this article.
If I did make a mistake, as you suggest, I’d be glad to update my beliefs—I love to be convinced to do so :-) Indeed, I acknowledged in an earlier comment that the post went a little too strongly into using cognitive ease strategies to make its points. So we’ll work to tone down the cognitive ease strategies in the future, and thank you for being one of the people to help update my beliefs incrementally.
My core issue isn’t about the availability of research but about the research that you cited. If you want to teach evidence based reasoning than you have to give people good sources. You don’t want to teach people to treat media articles titled 12 surprising facts about X as real evidence. It would raise the sanity waterline to teach people that such articles don’t constitute evidence.
You cited a link citing the first article. From Ioannidis we know that in general observational studies are pretty unreliable.
This article isn’t even a good observational studies. The gave their questionnaire only to Swingers and then compare those Swingers with external values from the literature. There no reason to believe that’s a good idea for a value like self reported happiness.
As far as the second article goes. First as far as I see you didn’t cite it. If you think that it’s an important article that someone should use for forming an evidence based view on the subject, you should cite it.
Secondly it says: “However, inconsistent with our predictions, anxiety was unrelated to current relationship
status”. I don’t see how that translates into them being more happy.
As far as the book goes, it doesn’t seem to be a peer reviewed study. Even when it might reference them.
Can you clarify why you believe that such research is unacceptable as a form of Level III evidence? After all, some evidence is better than no evidence for actually changing one’s mind. Polyamory research is a very new field, and descriptive studies are pretty much the best there is at this stage. But if you find other relevant research on polyamory that we missed, I would be happy to update my beliefs.
As that comment illuminates, I used the kind of evidence available currently.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.” CNM is the common acronym for consensual non-monogamous relationships. To me, “high levels of satisfaction” equates to being “happy.” However, I accept that we might interpret the term “high levels of satisfaction” differently—to me, it equates to being “happy,” but it might not to you. I think that’s primarily a semantic issue, though, and would rather not pursue it.
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
The link you give for Level III evidence means that it’s “controlled trial without randomization”. Controlling against a literature value isn’t a controlled trial.
Just given your questionaire to swingers that want to take a questionaire about swingers and presumably give the world a good impression about what it means to be a swinger to some average census value or literature value doesn’t work.
The feminist paper says that it uses a qualitative approach. In the hierarchy that you linked that’s Level VI.
As that comment illuminates, I used the kind of evidence available currently.
To the extend that currently there isn’t strong evidence available, you shouldn’t use definitive language like “show” but suggest to illustrate that the evidence isn’t strong.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.”
When reading that your first reaction should be “How do they know?” In this case this is part of their discussion of previous work and a result that comes out of the data they gathered and that they discuss in the methods section.
The first paper I looked at from that pile contains lines like “A social constructionist discourse analytic approach was taken to the data.”, so again qualitative. The second is also again qualitative. If it’s multiple qualitative studies that’s in your linked scheme level V.
“Sizable minority” is also a term that doesn’t tell you whether the average member of the population is better or worth then average.
I am confused by your claims that the book is not a peer reviewed study.
It’s a book and no study.Books are generally not primary sources.
Regarding the book, I want to draw your attention to the content of my comment above:
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
Can you please point out to me where we disagree regarding the book being a peer-reviewed study? That was the issue you raised, and that was what I responded to. So please let me know what your thoughts are about this matter.
Books often help to understand a subject matter in more depth. On the other hand they are usually no primary sources and therefore not that good for demonstrating the evidence for particular claims.
Has primary sources and is a primary source are two different things. When determining a specific claim such as is there research that proves that polyamorous people are happy than I care about the specific set up of a study to evaluate the evidence.
In general in academia everybody who runs a decent study wants to have the study in a citable paper and that paper is usually the best source for understanding what was done. In addition to his book Kahneman published a variety of papers.
I don’t have an issue with the sentence: “A major 15-year ethnographic research project showed the richness and diversity of poly families, within which individuals form relationships with a wide variety of partners and enjoy emotional and sexual freedom. ”
Showing richness and diversity is something that a qualitative ethnographic research project can do.
I have expressed confusion above over your claim regarding books, where you stated that they are “not that good for demonstrating the evidence for particular claims.” Can you please respond to that specific request for clarification?
You made an argument that books are not helpful for demonstrating claims. First, you stated that the book I cited was not peer reviewed. I pointed out that the book was actually peer reviewed. Would you please respond to my statement and avoid making further claims until you actually respond to my statement?
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources. There is no difference, in fact, between a peer-reviewed academic article and a book—they are both peer-reviewed pieces, and are both based on primary source evidence. Do you disagree? If so, please explain your disagreement. After all, if you do not take books to be good evidence, that is a basic difference in our worldviews, and I have no interest in further engaging in this matter.
First, you stated that the book I cited was not peer reviewed.
I said “Peer reviewed study”. That phrase not only contains “peer reviewed” but also study. In general high quality studies get published in papers. A good scientific textbook refers to a bunch of peer reviewed studies but it usually isn’t the primary source.
In this case it might be that some gender study folks have different idea of how to do science. After all they want to do science in a feminist framework. That might result in a study really being published in a book.
I think most people on LW do have alarm bells that ring if a gender study folks say they don’t like the way science is done traditionally and they rather want to follow a feminist framework.
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources.
A book that is based on primary sources is a secondary source, it’s not itself a primary source. A much better secondary source than a click-bait mainstream media article but still a secondary source.
That depends, in particular on whether “some evidence” is a representative sample. If the only evidence you see is a selected subset, it might well be worse than no evidence at all.
Good point about a selected subset, I agree. We have tried, in that post, to be representative of the current research on polyamory. If you happen to find contradictory research, I would be happy to update my beliefs.
The belief you ought to update (and reject), based on the evidence that is the almost unanimous comments to your post, is that, regardless of whether polyamory advocacy is rational, it is useful to keep it prominently visible on your site.
That’s a fair point, which I accept, and we are working on additional posts. In fact, we just posted one about dual process theory. Do you think that post is better suited for promoting rationality?
In general yes, but you started the OP with arguing that LW has too much jargon. In that post you invent new jargon with “autopilot system” and “intentional system”. If I google those terms there are only 4 hits with both and most of them aren’t even relevant hits.
Hm, we intended the “autopilot vs. intentional” to communicate things more clearly to a broad audience, but I see the point you’re making, something to think about for the future.
When it comes to the specific terms, “intentional” isn’t what I would use to label System II. When meditating I can be focus my intention without being in analytic mode.
Wikipedia suggests that the terms explicit system, the rule-based system, the rational system, or the analytic system.
All of them are probably less misleading than “intentional”.
But that isn’t the biggest issue here. Jargon is often invented because the speaker thinks it’s useful. At the same time inventing new jargon makes it harder for outsiders to follow you. If you reuse the terms in another article an outsider doesn’t exactly know what they are referring to.
It also makes it harder for someone who read your introductory material to afterwards read the existing academic literature because he has to learn new concepts.
You make superficial arguments against jargon instead of addressing the existing arguments for inventing jargon to justify your project and then you go and invent new jargon that suggests you didn’t deeply engage into thinking about the issue.
The real problem here isn’t that inventing new jargon is bad. The problem is that you aren’t committing to any principles and follow them. You lack strategic commitment.
That’s a problem for the kind of organisation that you want to build that’s bigger than any specific mistake to be found in the few posts that you wrote till now.
In this case I’m not particularly happy with the terms System I and System II either. I think there a case for using more descriptive words. But if you want to do so, the proper thing would be to review the different terms that are in use in the literature and the arguments people make for using one term over another.
That takes research and is hard work but it would probably lead to a better blog post than going the lazy route and simply inventing your own terms. Going the lazy route for inventing jargon is why things got the way they are on LW.
Discussing the semantics by looking at advantages of different terminology also helps a lot to understand the underlying subject matter in more depth.
I’m curious what makes you believe we did not review the literature? System 1 and 2 are used by Kahneman and Stanovich and West, while Thaler and Sunstein use automatic and reflective, Goleman uses lower and higher, and others use different terms for dual process theory. A good summary of the literature and the wide variety of terms used is available here.
On our Board of Directors, we have a cognitive neuroscientist whose input on using the terms “intentional” and “autopilot” we used to inform our choice, as well as a licensed therapist who has a great sense of how to communicate complex ideas from psychology to broad audiences. For another example of how therapists use terms such as “intentional,” check out this blog post.
Of course, we can have a further discussion about the benefits of dual process theory and using metaphors such as elephant and rider. Your thoughts?
However, again, I appreciate the feedback, and we will consider our use of these terms in the future.
You don’t provide reasons in your article of why you prefer the term intentional over the other terms that are used.
I can see the motivation to avoid System I/System II. Why use “intentional” when Kahneman used “reflective”. Why do you think using “intentional” is more clear to a new audience than “reflective”?
Why doesn’t your article include those reasons, so that the reader knows why you choose your terms?
Carlos Cabrera, whose input on using the terms “intentional” and “automatic” we used to inform our choice.
Then where does “autopilot system” come from? If you used a different term that your board member advised you to use, you screwed up even more.
I believe that I did provide the reasons we preferred the term “intentional”—based on the feedback of the cognitive neuroscientist and licensed therapist on our Board of Directors, with the latter especially helpful for the perspective of someone who has wide experience with how complex psychological terms are explained to broad audiences. As I pointed out above, the term “intentional” is widely used by therapists to explain how our minds work, especially the System 2 part, to broad audiences not well educated in psychology. Since our goal is to explain complex ideas drawn from cognitive neuroscience and psychology to broad audiences, that is why we made the choice to go with “intentional.” We have an experimental attitude, of course, and will see how much this term is helpful for broad audiences, and revise our use of it if it seems to be suboptimal.
Thanks for pointing out the spelling error, I meant to write “autopilot” in the comment I made above. I edited the comment based on your noticing the error.
I believe that I did provide the reasons we preferred the term “intentional”—based on the feedback of the cognitive neuroscientist and licensed therapist on our Board of Directors, with the latter especially helpful for the perspective of someone who has wide experience with how complex psychological terms are explained to broad audiences.
That’s not a description of why you think the term is better. Especially sufficiently better that it warrants inventing new jargon.
If you simply use the term because other board members prefer that term, then why do those board members prefer it over the Kaheman term reflective?
To me reflective also seems like a word used by therapists.
If you (including your board) think some people would be confused by reflective and have a better idea upon the concept when they hear intentional that’s an interesting opinion. Explaining why that might be the case in your article that introduces the terminology would be useful because then your audience would understand a way to misunderstand the concept and more likely understand the concept in the right way.
If your arguments are good you even have the change to motivate other people to copy your newly invented jargon.
That’s not a description of why you think the term is better.
I notice I’m confused. In the comment I made above, I believe I provided clear reasons for why we preferred the term “intentional.” Just in case it was not clear, our use of the term is based on the feedback of the cognitive neuroscientist and licensed therapist on our Board of Directors, with the latter especially helpful for the perspective of someone who has wide experience with how complex psychological terms are explained to broad audiences. As I pointed out above, the term “intentional” is widely used by therapists to explain how our minds work, especially the System 2 part, to broad audiences not well educated in psychology. Since our goal is to explain complex ideas drawn from cognitive neuroscience and psychology to broad audiences, that is why we made the choice to go with “intentional.” Moreover, scholars use the term “intentional system” to refer to agency and agents, and also see this one. Since one of our goals is to promote greater agency, we decided to use the term intentional, and orient toward tying the two concepts together in our content targeted at broad audiences.
We have an experimental attitude, of course, and we will see how much this term is helpful for broad audiences. We will update out beliefs and revise our use of this notion if it seems to be suboptimal for the target audience we’re going for. Appreciate your helpful comments on this, it’s always great to get such thoughtful engagement and helpful consideration on the benefits of using different terms.
Thanks for providing an actual motivation this time around.
Moreover, scholars use the term “intentional system” to refer to agency and agents,
I think that’s the core problem with the term. You suggest that doing things via system I isn’t agency. I think that’s a Straw Vulcan position. There’s no reason why intuition can’t be agency.
If I set the intention to meditate for 20 minutes and then meditate for 20 +-1 minutes without an external timekeeping device I have agency. There no reflective logical reasoning involved to get that timing. My understanding of Kahneman would be that’s using a System I process. Do you think that’s a System II process?
I am curious why you ascribe to me the position that “doing things via system I isn’t agency.” Can you please clarify to me where you believe I made that claim? I think I was pretty careful to avoid saying silly things like that.
You say that scholars use the term intentional system to refer to agency. You want to use the term intentional system to describe System II. That associates system II with agency.
If I set a “I meditate for 20 minutes intention” I would not call that a reflective, logical process. Maybe it’s reflective if I go through a process of thinking whether 15 or 20 minutes are better. I don’t think it’s reflective if I do have a habit of meditating for 20 minutes and just set up that intention at the beginning of meditating.
I think there are very intuitive processes that do have intentions and agency but my understanding of the terms System I and System II is that those intuitive processes are System I. System II suggests to me that I analyze what I have to do to make my intention become real. If I just trust my intuition to guide me and tell me when 20 minutes are over, I don’t understand that as System II.
Do you think that having an intention to meditate for 20 minutes and then trusting that everything will work by intuition is partly a system II process?
As I stated in the comment above, I do associate System 2 with agency, but in a very specific way. Namely, I stated that we can use our System 2 to train our System 1 to have more rational thinking and feeling patterns, ones better suited to achieving our long-term goals, and thus gaining agency.
In other words, I did not say that using System 1 is not being an agent. Being intentional about how one uses System 1, and training it, to be better suited to match one’s goals that we believe would actually fulfill our desires, is what I referred to as gaining greater agency.
Yup, I think that having an intention to meditate for 20 minutes and then trusting that everything will work by intuition is partly a System 2 process. It’s about framing oneself well, and training one’s intuition.
Take the sentence “Moreover, research on poly relationships show that people with consensual non-monogamous relationships are happier, especially with their sex lives, than those who are in traditional monogamous relationships, and communicate more openly.”
“Research” links to no peer reviewed paper. “Happy” links to another mainstream media article. “Communicate more openly” a published paper but that published paper isn’t a decent study that compared poly people with nonpoly people but it’s about a focus group discussion among poly women.
If you want to get people engaged with research than cite a bunch of papers directly and discuss them.
I notice I’m confused.
In the Intentional Insights article about polyamory, the word “research” links to the following article in Psychology Today that cites a wide variety of research papers. The word “happy” also links to a similar article
We think it’s quite appropriate to direct readers to an article that cites many research-based papers, as opposed to citing the papers themselves—it’s a pretty efficient goal factoring approach.
Moreover, the large majority of our target audience for the blog posts—people who are early onward in the process of gaining more rational thinking—would be unlikely to read research studies in-depth, and would be much more likely to read articles informed by research studies. So please keep that in mind as you read the Intentional Insights website. Our goal is to raise the sanity waterline by translating complex content for broad audiences.
Part of rationality is not trusting a mainstream media article to accurately represent the state of scientific knowledge. To the extend that you implicitly try to teach that a post titled “Open Relationships Reduce Jealousy? 12 Surprising Facts About Non-Monogamy” is equivalent to peer reviewed research because it includes citations, you aren’t raising the sanity waterline.
You are not discussing the arguments for whether or not polyamory raises happiness. The argument in that article is also pretty stupid. Just because a poll that only targeted Swingers shows that the Swingers in that study were more happy than average census takers doesn’t mean you can generalize. That’s no controlled study setup.
If you actually want to teach rationality than you should teach people not to trust claims made based on non-controlled observational studies. At least I would guess that’s what most people on LW would expect from a project that tries to teach rationality.
The other paper towards which you link directly contains the nice phrase: “The research aimed to explore issues around gender in non-monogamous relationships and to explore the potentials of participant-led methods to conduct research into this aspect of women’s sexuality with a qualitative, feminist framework.”
If you are serious about that, write your next article about why someone who wants to be a rationalist should take research like that seriously.
So your goal isn’t to teach critical thinking but do the critical thinking yourself an then give it’s results to your audience?
I’m curious, do you believe that the goal of translating complex content for broad audiences is a value-less endeavor? If so, then I accept we have a difference of opinion.
As part of doing so, we believe that raising the sanity waterline requires creating cognitive ease for audiences who do not yet have advanced rational analysis skills. If we disagree, then I accept we have a difference of opinion.
I don’t have any issue with someone trying to change beliefs of society. That are many cases where belief change is useful.
On the other hand that’s not the same thing as teaching people critical thinking or raising the sanity waterline.
Eliezer counts ”...what constitutes evidence and why;” as one of the things of raising the sanity waterline. By teaching people that they should treat uncontrolled surveys and research done in a participant-led, qualitative, feminist framework as good evidence you are part of the problem and not part of the solution as far as raising the sanity waterline goes.
Deciding that you are rational and therefore can see that polyamory should be better respected in society and then using whatever way to convince people whether or not that way has anything to do with rational argument isn’t what I consider raising the sanity waterline.
Can you clarify why you believe that such research is unacceptable as a form of Level III evidence? After all, some evidence is better than no evidence for actually changing one’s mind. Polyamory research is a very new field, and descriptive studies are pretty much the best there is at this stage. But if you find other relevant research on polyamory that we missed, I would be happy to update my beliefs.
I don’t see a good reason to believe that it’s representative of all polyamorous people. There’s a certain scene that very much values openness but there are other people who live polyamorously and who aren’t very open. Just because the feminist who runs the study is herself very open in her communication and interacts with others who are also very open doesn’t mean that everybody is.
You also did present it as fact in your article and especially if you are writing for a naive audience telling them about the uncertainty is useful if you want to train critical thinking. Skepticism is a default for critical thinking. If there no evidence either way, than there no evidence.
Feynman held his cargo-cult science speech about reasoning with better quality.
If you also sincerely believe that we should put more weight on science done with an explicit feminist framework that’s a position worthy writing an article about because I guess that you are holding a minority position under self-labeled rationalists.
A core question of rationalism is: “Why do we believe what we believe? In this case you don’t tell the reader why you think the particular material you reference should make him believe that poly people are happy and open.
I’m curious why you attribute that position to our blog post—that’s the opposite of a steel man move. In fact, the Intentional Insights blog post did not make the claim you seem to attribute to it. What we claimed was the following, which I am quoting from the blog post: “research on poly relationships show that people with consensual non-monogamous relationships are happier, especially with their sex lives, than those who are in traditional monogamous relationships, and communicate more openly.”
If you have research—any sort of research—demonstrating otherwise, I would be happy to update my beliefs. I think we can agree that the quality of evidence should be the deciding factor :-)
I think the common meaning of the phrase “research shows X” is “research shows X is true”. If that’s not what you want to argue you can use phrases like “research suggests X”.
If the thing you want to teach is evidence based reasoning it would be still useful to explain a naive audience the strength of the evidence.
A post either has mistakes or it doesn’t. The point of steelmanning is to change someone’s argument to make it better. I’m not denying that it’s possible to steelman the claim that polyamory increases happiness or openness but that’s not what I care about here. The thing that matter is whether or not this post encourages critical thinking. Using “show” when you mean “suggest” doesn’t help for that goal.
I agree that the common meaning of the phrase “research shows X” is “research shows X is true.” However, can you clarify to me where you believe the post makes mistakes in its use of “show” vs. “suggest”?
The statement I made in the salient blog post on the Intentional Insights website was as follows “research on poly relationships show that people with consensual non-monogamous relationships are happier.” I believe that the evidence I cited there supports the notion that “research shows X is true” when X = “on poly relationships shows that people with consensual non-monogamous relationships are happier.” For examples of such evidence, see this article, or this book or this article.
If I did make a mistake, as you suggest, I’d be glad to update my beliefs—I love to be convinced to do so :-) Indeed, I acknowledged in an earlier comment that the post went a little too strongly into using cognitive ease strategies to make its points. So we’ll work to tone down the cognitive ease strategies in the future, and thank you for being one of the people to help update my beliefs incrementally.
My core issue isn’t about the availability of research but about the research that you cited. If you want to teach evidence based reasoning than you have to give people good sources. You don’t want to teach people to treat media articles titled 12 surprising facts about X as real evidence. It would raise the sanity waterline to teach people that such articles don’t constitute evidence.
You cited a link citing the first article. From Ioannidis we know that in general observational studies are pretty unreliable. This article isn’t even a good observational studies. The gave their questionnaire only to Swingers and then compare those Swingers with external values from the literature. There no reason to believe that’s a good idea for a value like self reported happiness.
As far as the second article goes. First as far as I see you didn’t cite it. If you think that it’s an important article that someone should use for forming an evidence based view on the subject, you should cite it.
Secondly it says: “However, inconsistent with our predictions, anxiety was unrelated to current relationship status”. I don’t see how that translates into them being more happy.
As far as the book goes, it doesn’t seem to be a peer reviewed study. Even when it might reference them.
It’s great to see we are both committed to using evidence, and the debate is now focusing on the nature of the evidence.
I believe I had earlier stated the following in my comment above
As that comment illuminates, I used the kind of evidence available currently.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.” CNM is the common acronym for consensual non-monogamous relationships. To me, “high levels of satisfaction” equates to being “happy.” However, I accept that we might interpret the term “high levels of satisfaction” differently—to me, it equates to being “happy,” but it might not to you. I think that’s primarily a semantic issue, though, and would rather not pursue it.
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
The link you give for Level III evidence means that it’s “controlled trial without randomization”. Controlling against a literature value isn’t a controlled trial.
Just given your questionaire to swingers that want to take a questionaire about swingers and presumably give the world a good impression about what it means to be a swinger to some average census value or literature value doesn’t work.
The feminist paper says that it uses a qualitative approach. In the hierarchy that you linked that’s Level VI.
To the extend that currently there isn’t strong evidence available, you shouldn’t use definitive language like “show” but suggest to illustrate that the evidence isn’t strong.
When reading that your first reaction should be “How do they know?” In this case this is part of their discussion of previous work and a result that comes out of the data they gathered and that they discuss in the methods section. The first paper I looked at from that pile contains lines like “A social constructionist discourse analytic approach was taken to the data.”, so again qualitative. The second is also again qualitative. If it’s multiple qualitative studies that’s in your linked scheme level V.
“Sizable minority” is also a term that doesn’t tell you whether the average member of the population is better or worth then average.
It’s a book and no study.Books are generally not primary sources.
Regarding the book, I want to draw your attention to the content of my comment above:
Can you please point out to me where we disagree regarding the book being a peer-reviewed study? That was the issue you raised, and that was what I responded to. So please let me know what your thoughts are about this matter.
Books often help to understand a subject matter in more depth. On the other hand they are usually no primary sources and therefore not that good for demonstrating the evidence for particular claims.
I notice I’m confused over your claim about books and primary sources.
My brief monograph, for example, is flush with primary sources. The specific book I cited earlier, the one from the Intentional Insights blog post on polyamory, is a 15-year ethnographic study, and is thus quite full of primary sources. Daniel Kahneman’s Thinking, Fast and Slow is also a book, and has plenty of primary sources.
So can you please clarify to me your comments about books? Appreciate it :-)
Has primary sources and is a primary source are two different things. When determining a specific claim such as is there research that proves that polyamorous people are happy than I care about the specific set up of a study to evaluate the evidence.
In general in academia everybody who runs a decent study wants to have the study in a citable paper and that paper is usually the best source for understanding what was done. In addition to his book Kahneman published a variety of papers.
I don’t have an issue with the sentence: “A major 15-year ethnographic research project showed the richness and diversity of poly families, within which individuals form relationships with a wide variety of partners and enjoy emotional and sexual freedom. ”
Showing richness and diversity is something that a qualitative ethnographic research project can do.
I think especially the claim about polyamory people being happier is not established. I opened a question on skeptics stackexchange about that claim. http://skeptics.stackexchange.com/questions/23807/are-polyamorous-people-happier
I have expressed confusion above over your claim regarding books, where you stated that they are “not that good for demonstrating the evidence for particular claims.” Can you please respond to that specific request for clarification?
You made an argument that books are not helpful for demonstrating claims. First, you stated that the book I cited was not peer reviewed. I pointed out that the book was actually peer reviewed. Would you please respond to my statement and avoid making further claims until you actually respond to my statement?
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources. There is no difference, in fact, between a peer-reviewed academic article and a book—they are both peer-reviewed pieces, and are both based on primary source evidence. Do you disagree? If so, please explain your disagreement. After all, if you do not take books to be good evidence, that is a basic difference in our worldviews, and I have no interest in further engaging in this matter.
I said “Peer reviewed study”. That phrase not only contains “peer reviewed” but also study. In general high quality studies get published in papers. A good scientific textbook refers to a bunch of peer reviewed studies but it usually isn’t the primary source.
In this case it might be that some gender study folks have different idea of how to do science. After all they want to do science in a feminist framework. That might result in a study really being published in a book.
I think most people on LW do have alarm bells that ring if a gender study folks say they don’t like the way science is done traditionally and they rather want to follow a feminist framework.
A book that is based on primary sources is a secondary source, it’s not itself a primary source. A much better secondary source than a click-bait mainstream media article but still a secondary source.
That depends, in particular on whether “some evidence” is a representative sample. If the only evidence you see is a selected subset, it might well be worse than no evidence at all.
Good point about a selected subset, I agree. We have tried, in that post, to be representative of the current research on polyamory. If you happen to find contradictory research, I would be happy to update my beliefs.
The belief you ought to update (and reject), based on the evidence that is the almost unanimous comments to your post, is that, regardless of whether polyamory advocacy is rational, it is useful to keep it prominently visible on your site.
That’s a fair point, which I accept, and we are working on additional posts. In fact, we just posted one about dual process theory. Do you think that post is better suited for promoting rationality?
In general yes, but you started the OP with arguing that LW has too much jargon. In that post you invent new jargon with “autopilot system” and “intentional system”. If I google those terms there are only 4 hits with both and most of them aren’t even relevant hits.
Hm, we intended the “autopilot vs. intentional” to communicate things more clearly to a broad audience, but I see the point you’re making, something to think about for the future.
When it comes to the specific terms, “intentional” isn’t what I would use to label System II. When meditating I can be focus my intention without being in analytic mode. Wikipedia suggests that the terms explicit system, the rule-based system, the rational system, or the analytic system. All of them are probably less misleading than “intentional”.
But that isn’t the biggest issue here. Jargon is often invented because the speaker thinks it’s useful. At the same time inventing new jargon makes it harder for outsiders to follow you. If you reuse the terms in another article an outsider doesn’t exactly know what they are referring to.
It also makes it harder for someone who read your introductory material to afterwards read the existing academic literature because he has to learn new concepts.
You make superficial arguments against jargon instead of addressing the existing arguments for inventing jargon to justify your project and then you go and invent new jargon that suggests you didn’t deeply engage into thinking about the issue.
The real problem here isn’t that inventing new jargon is bad. The problem is that you aren’t committing to any principles and follow them. You lack strategic commitment. That’s a problem for the kind of organisation that you want to build that’s bigger than any specific mistake to be found in the few posts that you wrote till now.
In this case I’m not particularly happy with the terms System I and System II either. I think there a case for using more descriptive words. But if you want to do so, the proper thing would be to review the different terms that are in use in the literature and the arguments people make for using one term over another. That takes research and is hard work but it would probably lead to a better blog post than going the lazy route and simply inventing your own terms. Going the lazy route for inventing jargon is why things got the way they are on LW.
Discussing the semantics by looking at advantages of different terminology also helps a lot to understand the underlying subject matter in more depth.
Thanks for the feedback.
I’m curious what makes you believe we did not review the literature? System 1 and 2 are used by Kahneman and Stanovich and West, while Thaler and Sunstein use automatic and reflective, Goleman uses lower and higher, and others use different terms for dual process theory. A good summary of the literature and the wide variety of terms used is available here.
On our Board of Directors, we have a cognitive neuroscientist whose input on using the terms “intentional” and “autopilot” we used to inform our choice, as well as a licensed therapist who has a great sense of how to communicate complex ideas from psychology to broad audiences. For another example of how therapists use terms such as “intentional,” check out this blog post.
Of course, we can have a further discussion about the benefits of dual process theory and using metaphors such as elephant and rider. Your thoughts?
However, again, I appreciate the feedback, and we will consider our use of these terms in the future.
*Edited term “automatic” to “autopilot”
You don’t provide reasons in your article of why you prefer the term intentional over the other terms that are used.
I can see the motivation to avoid System I/System II. Why use “intentional” when Kahneman used “reflective”. Why do you think using “intentional” is more clear to a new audience than “reflective”?
Why doesn’t your article include those reasons, so that the reader knows why you choose your terms?
Then where does “autopilot system” come from? If you used a different term that your board member advised you to use, you screwed up even more.
I believe that I did provide the reasons we preferred the term “intentional”—based on the feedback of the cognitive neuroscientist and licensed therapist on our Board of Directors, with the latter especially helpful for the perspective of someone who has wide experience with how complex psychological terms are explained to broad audiences. As I pointed out above, the term “intentional” is widely used by therapists to explain how our minds work, especially the System 2 part, to broad audiences not well educated in psychology. Since our goal is to explain complex ideas drawn from cognitive neuroscience and psychology to broad audiences, that is why we made the choice to go with “intentional.” We have an experimental attitude, of course, and will see how much this term is helpful for broad audiences, and revise our use of it if it seems to be suboptimal.
Thanks for pointing out the spelling error, I meant to write “autopilot” in the comment I made above. I edited the comment based on your noticing the error.
That’s not a description of why you think the term is better. Especially sufficiently better that it warrants inventing new jargon.
If you simply use the term because other board members prefer that term, then why do those board members prefer it over the Kaheman term reflective? To me reflective also seems like a word used by therapists.
If you (including your board) think some people would be confused by reflective and have a better idea upon the concept when they hear intentional that’s an interesting opinion. Explaining why that might be the case in your article that introduces the terminology would be useful because then your audience would understand a way to misunderstand the concept and more likely understand the concept in the right way.
If your arguments are good you even have the change to motivate other people to copy your newly invented jargon.
I notice I’m confused. In the comment I made above, I believe I provided clear reasons for why we preferred the term “intentional.” Just in case it was not clear, our use of the term is based on the feedback of the cognitive neuroscientist and licensed therapist on our Board of Directors, with the latter especially helpful for the perspective of someone who has wide experience with how complex psychological terms are explained to broad audiences. As I pointed out above, the term “intentional” is widely used by therapists to explain how our minds work, especially the System 2 part, to broad audiences not well educated in psychology. Since our goal is to explain complex ideas drawn from cognitive neuroscience and psychology to broad audiences, that is why we made the choice to go with “intentional.” Moreover, scholars use the term “intentional system” to refer to agency and agents, and also see this one. Since one of our goals is to promote greater agency, we decided to use the term intentional, and orient toward tying the two concepts together in our content targeted at broad audiences.
We have an experimental attitude, of course, and we will see how much this term is helpful for broad audiences. We will update out beliefs and revise our use of this notion if it seems to be suboptimal for the target audience we’re going for. Appreciate your helpful comments on this, it’s always great to get such thoughtful engagement and helpful consideration on the benefits of using different terms.
Thanks for providing an actual motivation this time around.
I think that’s the core problem with the term. You suggest that doing things via system I isn’t agency. I think that’s a Straw Vulcan position. There’s no reason why intuition can’t be agency.
If I set the intention to meditate for 20 minutes and then meditate for 20 +-1 minutes without an external timekeeping device I have agency. There no reflective logical reasoning involved to get that timing. My understanding of Kahneman would be that’s using a System I process. Do you think that’s a System II process?
I am curious why you ascribe to me the position that “doing things via system I isn’t agency.” Can you please clarify to me where you believe I made that claim? I think I was pretty careful to avoid saying silly things like that.
The claim I did make is that we can use our System 2 to train our System 1 to have more rational thinking and feeling patterns, ones better suited to achieving our long-term goals, and thus gaining agency. After all, agency is about achieving the goals that we believe would actually fulfill our desires. Certainly, doing things via System 1 can be used very effectively as an agentive move, if one trains one’s elephant well.
You say that scholars use the term intentional system to refer to agency. You want to use the term intentional system to describe System II. That associates system II with agency.
If I set a “I meditate for 20 minutes intention” I would not call that a reflective, logical process. Maybe it’s reflective if I go through a process of thinking whether 15 or 20 minutes are better. I don’t think it’s reflective if I do have a habit of meditating for 20 minutes and just set up that intention at the beginning of meditating.
I think there are very intuitive processes that do have intentions and agency but my understanding of the terms System I and System II is that those intuitive processes are System I. System II suggests to me that I analyze what I have to do to make my intention become real. If I just trust my intuition to guide me and tell me when 20 minutes are over, I don’t understand that as System II.
Do you think that having an intention to meditate for 20 minutes and then trusting that everything will work by intuition is partly a system II process?
As I stated in the comment above, I do associate System 2 with agency, but in a very specific way. Namely, I stated that we can use our System 2 to train our System 1 to have more rational thinking and feeling patterns, ones better suited to achieving our long-term goals, and thus gaining agency.
In other words, I did not say that using System 1 is not being an agent. Being intentional about how one uses System 1, and training it, to be better suited to match one’s goals that we believe would actually fulfill our desires, is what I referred to as gaining greater agency.
Yup, I think that having an intention to meditate for 20 minutes and then trusting that everything will work by intuition is partly a System 2 process. It’s about framing oneself well, and training one’s intuition.
Hope that clarifies things :-)