Can you clarify why you believe that such research is unacceptable as a form of Level III evidence? After all, some evidence is better than no evidence for actually changing one’s mind. Polyamory research is a very new field, and descriptive studies are pretty much the best there is at this stage. But if you find other relevant research on polyamory that we missed, I would be happy to update my beliefs.
As that comment illuminates, I used the kind of evidence available currently.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.” CNM is the common acronym for consensual non-monogamous relationships. To me, “high levels of satisfaction” equates to being “happy.” However, I accept that we might interpret the term “high levels of satisfaction” differently—to me, it equates to being “happy,” but it might not to you. I think that’s primarily a semantic issue, though, and would rather not pursue it.
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
The link you give for Level III evidence means that it’s “controlled trial without randomization”. Controlling against a literature value isn’t a controlled trial.
Just given your questionaire to swingers that want to take a questionaire about swingers and presumably give the world a good impression about what it means to be a swinger to some average census value or literature value doesn’t work.
The feminist paper says that it uses a qualitative approach. In the hierarchy that you linked that’s Level VI.
As that comment illuminates, I used the kind of evidence available currently.
To the extend that currently there isn’t strong evidence available, you shouldn’t use definitive language like “show” but suggest to illustrate that the evidence isn’t strong.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.”
When reading that your first reaction should be “How do they know?” In this case this is part of their discussion of previous work and a result that comes out of the data they gathered and that they discuss in the methods section.
The first paper I looked at from that pile contains lines like “A social constructionist discourse analytic approach was taken to the data.”, so again qualitative. The second is also again qualitative. If it’s multiple qualitative studies that’s in your linked scheme level V.
“Sizable minority” is also a term that doesn’t tell you whether the average member of the population is better or worth then average.
I am confused by your claims that the book is not a peer reviewed study.
It’s a book and no study.Books are generally not primary sources.
Regarding the book, I want to draw your attention to the content of my comment above:
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
Can you please point out to me where we disagree regarding the book being a peer-reviewed study? That was the issue you raised, and that was what I responded to. So please let me know what your thoughts are about this matter.
Books often help to understand a subject matter in more depth. On the other hand they are usually no primary sources and therefore not that good for demonstrating the evidence for particular claims.
Has primary sources and is a primary source are two different things. When determining a specific claim such as is there research that proves that polyamorous people are happy than I care about the specific set up of a study to evaluate the evidence.
In general in academia everybody who runs a decent study wants to have the study in a citable paper and that paper is usually the best source for understanding what was done. In addition to his book Kahneman published a variety of papers.
I don’t have an issue with the sentence: “A major 15-year ethnographic research project showed the richness and diversity of poly families, within which individuals form relationships with a wide variety of partners and enjoy emotional and sexual freedom. ”
Showing richness and diversity is something that a qualitative ethnographic research project can do.
I have expressed confusion above over your claim regarding books, where you stated that they are “not that good for demonstrating the evidence for particular claims.” Can you please respond to that specific request for clarification?
You made an argument that books are not helpful for demonstrating claims. First, you stated that the book I cited was not peer reviewed. I pointed out that the book was actually peer reviewed. Would you please respond to my statement and avoid making further claims until you actually respond to my statement?
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources. There is no difference, in fact, between a peer-reviewed academic article and a book—they are both peer-reviewed pieces, and are both based on primary source evidence. Do you disagree? If so, please explain your disagreement. After all, if you do not take books to be good evidence, that is a basic difference in our worldviews, and I have no interest in further engaging in this matter.
First, you stated that the book I cited was not peer reviewed.
I said “Peer reviewed study”. That phrase not only contains “peer reviewed” but also study. In general high quality studies get published in papers. A good scientific textbook refers to a bunch of peer reviewed studies but it usually isn’t the primary source.
In this case it might be that some gender study folks have different idea of how to do science. After all they want to do science in a feminist framework. That might result in a study really being published in a book.
I think most people on LW do have alarm bells that ring if a gender study folks say they don’t like the way science is done traditionally and they rather want to follow a feminist framework.
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources.
A book that is based on primary sources is a secondary source, it’s not itself a primary source. A much better secondary source than a click-bait mainstream media article but still a secondary source.
It’s great to see we are both committed to using evidence, and the debate is now focusing on the nature of the evidence.
I believe I had earlier stated the following in my comment above
As that comment illuminates, I used the kind of evidence available currently.
For the second article, it states “a sizeable minority of people engage in CNM and report high levels of satisfaction.” CNM is the common acronym for consensual non-monogamous relationships. To me, “high levels of satisfaction” equates to being “happy.” However, I accept that we might interpret the term “high levels of satisfaction” differently—to me, it equates to being “happy,” but it might not to you. I think that’s primarily a semantic issue, though, and would rather not pursue it.
I am confused by your claims that the book is not a peer reviewed study. It is published by Rowman & Littlefield, a well-known and well-respected publisher, which has a solid peer review process. I would appreciate your clarification on what you meant when you suggested it was not.
The link you give for Level III evidence means that it’s “controlled trial without randomization”. Controlling against a literature value isn’t a controlled trial.
Just given your questionaire to swingers that want to take a questionaire about swingers and presumably give the world a good impression about what it means to be a swinger to some average census value or literature value doesn’t work.
The feminist paper says that it uses a qualitative approach. In the hierarchy that you linked that’s Level VI.
To the extend that currently there isn’t strong evidence available, you shouldn’t use definitive language like “show” but suggest to illustrate that the evidence isn’t strong.
When reading that your first reaction should be “How do they know?” In this case this is part of their discussion of previous work and a result that comes out of the data they gathered and that they discuss in the methods section. The first paper I looked at from that pile contains lines like “A social constructionist discourse analytic approach was taken to the data.”, so again qualitative. The second is also again qualitative. If it’s multiple qualitative studies that’s in your linked scheme level V.
“Sizable minority” is also a term that doesn’t tell you whether the average member of the population is better or worth then average.
It’s a book and no study.Books are generally not primary sources.
Regarding the book, I want to draw your attention to the content of my comment above:
Can you please point out to me where we disagree regarding the book being a peer-reviewed study? That was the issue you raised, and that was what I responded to. So please let me know what your thoughts are about this matter.
Books often help to understand a subject matter in more depth. On the other hand they are usually no primary sources and therefore not that good for demonstrating the evidence for particular claims.
I notice I’m confused over your claim about books and primary sources.
My brief monograph, for example, is flush with primary sources. The specific book I cited earlier, the one from the Intentional Insights blog post on polyamory, is a 15-year ethnographic study, and is thus quite full of primary sources. Daniel Kahneman’s Thinking, Fast and Slow is also a book, and has plenty of primary sources.
So can you please clarify to me your comments about books? Appreciate it :-)
Has primary sources and is a primary source are two different things. When determining a specific claim such as is there research that proves that polyamorous people are happy than I care about the specific set up of a study to evaluate the evidence.
In general in academia everybody who runs a decent study wants to have the study in a citable paper and that paper is usually the best source for understanding what was done. In addition to his book Kahneman published a variety of papers.
I don’t have an issue with the sentence: “A major 15-year ethnographic research project showed the richness and diversity of poly families, within which individuals form relationships with a wide variety of partners and enjoy emotional and sexual freedom. ”
Showing richness and diversity is something that a qualitative ethnographic research project can do.
I think especially the claim about polyamory people being happier is not established. I opened a question on skeptics stackexchange about that claim. http://skeptics.stackexchange.com/questions/23807/are-polyamorous-people-happier
I have expressed confusion above over your claim regarding books, where you stated that they are “not that good for demonstrating the evidence for particular claims.” Can you please respond to that specific request for clarification?
You made an argument that books are not helpful for demonstrating claims. First, you stated that the book I cited was not peer reviewed. I pointed out that the book was actually peer reviewed. Would you please respond to my statement and avoid making further claims until you actually respond to my statement?
I pointed out a number of books, including my book and Daniel Kahneman’s book, that are based on primary sources. There is no difference, in fact, between a peer-reviewed academic article and a book—they are both peer-reviewed pieces, and are both based on primary source evidence. Do you disagree? If so, please explain your disagreement. After all, if you do not take books to be good evidence, that is a basic difference in our worldviews, and I have no interest in further engaging in this matter.
I said “Peer reviewed study”. That phrase not only contains “peer reviewed” but also study. In general high quality studies get published in papers. A good scientific textbook refers to a bunch of peer reviewed studies but it usually isn’t the primary source.
In this case it might be that some gender study folks have different idea of how to do science. After all they want to do science in a feminist framework. That might result in a study really being published in a book.
I think most people on LW do have alarm bells that ring if a gender study folks say they don’t like the way science is done traditionally and they rather want to follow a feminist framework.
A book that is based on primary sources is a secondary source, it’s not itself a primary source. A much better secondary source than a click-bait mainstream media article but still a secondary source.