I am not in charge of the wiki—my saying that I like a particular version (and I still like that version better) is not the same thing as my approving it.
Also, “caricature of rationality” strikes me as both harsh and false. I hadn’t gotten it into words before, but I’m concerned that you’re not teaching rationality at all, just conveying the idea that “rationality” is an applause light. Do you have methods for checking on what you’re actually teaching?
NancyLebovitz, this is a very important question! Yup, we do have methods of checking in on what we are teaching.
For example, let’s take our meaning and purpose content, which is essentially about teaching people to be oriented toward the long term and achieving their long-term goals. We have an app that has a psychometric test to measure people’s level of meaning and purpose prior to engaging with our content, and then after they engage with our content, including continuing follow-up going forward. The continuing follow-up is meant to address the issue of attention bias and Hawthorne effect, namely to test whether people just got an immediate boost or if there a long-term benefit to people engaging with our content.
We are developing similar apps for other types of content, such as an app for planning fallacy.
We also intend to run randomized control studies on our content’s impact, as described in our theory of change. We have written up proposals for those studies, and are currently trying to get funding for these studies.
That’s the link to the sign-up page for the app. As you can see from the footer, that information is gathered for the purpose of doing demographic analysis as part of our data analysis.
If you want to know the scientific literature on meaning and purpose, there are plenty of sources available. Here’s a good overview, and you can follow the footnotes to learn more.
As I stated earlier, this app uses a psychometric test. It uses a 1-10 Likert scale and a series of questions drawn from scientific literature measuring meaning. The test measures answers on questions, as do other psychometric tests of states of mind such as depression or happiness. In other words, there is no unit measure of “meaning” just as there is no unit measure of “depression” or “happiness.” I guess you could try to measure it in terms of Shannons if you really wanted, but that’s not what the scholarly literature uses for these types of exams.
If you assign a numeric value to something you call “meaning”, there must be a unit measure.
For example, consider IQ points. The unit measure (one IQ point) is 1⁄15 of the standard deviation of the distribution of IQ scores (which are basically normalised ranks) in white populations. We can argue what it corresponds to in the underlying reality, but at least it is well-defined.
The unit measure, however, may be entirely defined by the scale in question. Gleb could say “my unit measure is 1⁄9 of the full range of possible answers on my Meaning Measuring Scale”. That would be pretty useless, but so is “my unit measure is 1⁄15 of one standard deviation on my Cleverness Measuring Scale”.
Perhaps it’s better to reference scores to standard deviation rather than the full possible range—I guess it depends on the particular case. The other advantage IQ has is that there are now lots of IQ tests and they tend to get somewhat correlated results (suggesting that maybe they’re measuring something real) that also correlate with other interesting things (suggesting that maybe the real thing they’re measuring is useful) and they get used quite a lot (so that if you quote an IQ score there’s a good chance that the people you’re addressing will understand roughly what you mean).
Those are all genuine (or at least possibly-genuine) ways in which IQ scores are more useful than Gleb Meaningfulness Metric scores. But I don’t see that IQ is any better off than Gleb Meaningfulness Metric in terms of unit-measure-having. One IQ point doesn’t correspond to a fixed increment in thinking speed or memory capacity or ability to solve any particular kind of problem, or anything like that; it’s just a certain fraction of how much variation there is in one kind of brainpower score. One Gleb Meaningfulness Metric point, likewise, is just a certain fraction of how much variation is possible in one kind of feeling-like-your-life-has-meaning score.
Perhaps it’s better to reference scores to standard deviation rather than the full possible range—I guess it depends on the particular case.
Yep, that highly depends on the shape of the distribution.
One Gleb Meaningfulness Metric point, likewise, is just a certain fraction of how much variation is possible
Well, we (or at least I) haven’t seen Gleb’s Meaninfulness Metric, so I have no idea if it’s defined via population standard deviation like IQ. It may or it may be. I brought up IQ as an example of a unit which does not directly correspond to, say, thinking speed or working memory capacity—it’s entangled with the test itself, but it does make the numbers interpretable.
This is a nicety; while appreciated it looks like you are trying to suck up to Nancy.
(taking into account what Lumifier said about the app) in your description;
We also intend to run randomized control studies on our content’s impact, as described in our theory of change. We have written up proposals for those studies, and are currently trying to get funding for these studies.
Is the only thing you said are doing to be checking on what you’re actually teaching. And it’s something that has not happened yet. (granted these things take time). I read the entire post as; “nothing yet, but we want to—in these ways...”.
randomized control studies on our content’s impact
depending on the method (if done with web content) could be described as A/B split testing. Which is standard these days for internet behaviour of groups spreading clickbait, not an accountable test.
I would suggest not taking Lumifer’s description of his initial impression of the sign-up page as indicative of the app itself. I think there’s sufficient evidence of Lumifer being not unbiased in describing Intentional Insights content. So consider checking out the app itself.
Here are the study proposals themselves, which you can evaluate yourself for whether they are A/B split testing: 1, 2.
EDIT Forgot to mention, my comment on the nature of NancyLebovitz’s question had to do with my own desire to signal that this is a very important question to me, not give praise.
It could be you are actually asking why did anyone change the “approved” version. The answer is that there ain’t no such thing as “approved” versions—this is a wiki, open to edits. No one needs to ask for permission to change the page.
Nope, Elo’s motivations are not obvious to me. I don’t want to suffer from the typical mind fallacy here, so I used the first virtue of curiosity to get more information.
So let’s try the obvious default thing: Elo thinks that the old version does not reflect the character of InIn properly and changed the page to make it—from her point of view—better / more accurate. Any particular reason you find the obvious motivation unsatisfying?
I don’t want to suffer from the typical mind fallacy here, so I used the first virtue of curiosity to get more information.
Really really sounds like you are using these things to imply, “I am more rational than the others on this forum”, compliments to @Lumifier for not taking it that way.
“Elo’s motivations are not obvious to me. I am asking because I was curious about why he did this”
Would be a better way to say the same thing, without saying “I used the virtue and mentioned a fallacy (that’s all about what rationality is about right?), reward me with internet gold”.
Was the phrasing used, instead of (“you are...”). To try to point out what it sounds like he is saying… (probably could have said—“to me it sounds like...”, but that should be a given—I am posting the post...)
Ugliest or not; it’s an interpretation that I chose to share. I also tried to improve the statement to make it less “ugly”.
I don’t know—should I not do that at all?
Can you help with what it “sounds like” I am doing? (keen to learn and change)
I don’t oppose drawing interpretations, but the whole InIn discussion has been tainted by opponents attacking each other’s motivations for what they said instead of the content of what they said.
in light of that: “to me it sounds like...” is probably more appropriate.
My state is that Gleb is trying to do good things—i.e. raise the sanity waterline (as a goal).
As yet he has made a lot of noise and not shown success in the process; or produced content worthy of respect. It remains to be seen if he:
improves;
continues to further offend seasoned members (through various methods i.e. weak content); or
quits.
Given those seem like the options; I would rather 1 then 3 then 2.
I expect from glebs perspective it looks like:
invest more effort into the content to the point where it seems not worth it
fight the haters and win, then keep doing more of what has happened.
give up on a serious dream to improve the world.
where his preference order is 2, 1, 3.
I hope we can meet in the middle; and given that “fight the haters and win on the internet” is a statement of comedy itself, I am keen to see the content improve.
I am not sure there is agreement about the direction of improvement.
Gleb has posted how he finds it difficult to write sleazy scummy content, but overpowers his reluctance and through great personal sacrifice does write it. I would expect that “improvement” for him means more concentrated snake oil or, perhaps, less personal discomfort with producing it.
I don’t think this is what Elo would consider “improvement”.
Gleb has posted how he finds it difficult to write sleazy scummy content, but overpowers his reluctance and through great personal sacrifice does write it. I would expect that “improvement” for him means more concentrated snake oil or, perhaps, less personal discomfort with producing it.
I don’t think it’s okay to put those words (“sleazy scummy concentrated snake oil”) in someone else’s mouth, unless it is part of an actual quote.
I am not quoting Gleb, I’m rephrasing his comment in my own words and from my own point of view. I think this is his original comment, but he repeated this in other places as well.
Gleb described having had to overpower reluctance to write in the style that publications like Lifehacker want, expressed some reservations about that style in morally-neutral language, and gave reasons for using it anyways. Separately, you and others (but not Gleb) described that style as sleazy and scummy. Mix these two things together and discard the attributions, and you’ve created the impression that Gleb thinks of the style as sleazy and scummy, and writes in it anyways. That would reflect negatively on his character if it were true, but it isn’t. Having to use an actual quote would have made this mistake impossible.
I wasn’t trying for any gotchas, my point was—and remains—that in this case the word “improvement” is likely to be understood differently by different people.
I am not passing any moral judgments or making insinuations about anyone’s character.
By the way, if you’re aiming for accuracy, the verb Gleb used was “cringe”. You chose to render it as “reluctance” which, I think, also changes the flavour albeit in another direction. It’s up to you to decide whether cringing and still writing “would reflect negatively on his character” :-P
edited. I agree; it was uncalled for. This war appears to be sucking me into the fray.
Elo, I’m confused by why you chose to rewrite the wiki page after jimrandomh rewrote it and NancyLebovitz explicitly approved his rewrite. Can you please explain your motivations?
EDIT Edited for a typo
I am not in charge of the wiki—my saying that I like a particular version (and I still like that version better) is not the same thing as my approving it.
Also, “caricature of rationality” strikes me as both harsh and false. I hadn’t gotten it into words before, but I’m concerned that you’re not teaching rationality at all, just conveying the idea that “rationality” is an applause light. Do you have methods for checking on what you’re actually teaching?
NancyLebovitz, this is a very important question! Yup, we do have methods of checking in on what we are teaching.
For example, let’s take our meaning and purpose content, which is essentially about teaching people to be oriented toward the long term and achieving their long-term goals. We have an app that has a psychometric test to measure people’s level of meaning and purpose prior to engaging with our content, and then after they engage with our content, including continuing follow-up going forward. The continuing follow-up is meant to address the issue of attention bias and Hawthorne effect, namely to test whether people just got an immediate boost or if there a long-term benefit to people engaging with our content.
We are developing similar apps for other types of content, such as an app for planning fallacy.
We also intend to run randomized control studies on our content’s impact, as described in our theory of change. We have written up proposals for those studies, and are currently trying to get funding for these studies.
That’s not a link to an app, that’s a link to a sign-up sheet which wants to know my family income. Really?
Besides that, the idea of measuring the “level of meaning” implies that there is a unit of meaning. What might that be?
That’s the link to the sign-up page for the app. As you can see from the footer, that information is gathered for the purpose of doing demographic analysis as part of our data analysis.
If you want to know the scientific literature on meaning and purpose, there are plenty of sources available. Here’s a good overview, and you can follow the footnotes to learn more.
I am not asking for psychological literature on meaning. I am asking what your app is using as a unit of meaning.
As I stated earlier, this app uses a psychometric test. It uses a 1-10 Likert scale and a series of questions drawn from scientific literature measuring meaning. The test measures answers on questions, as do other psychometric tests of states of mind such as depression or happiness. In other words, there is no unit measure of “meaning” just as there is no unit measure of “depression” or “happiness.” I guess you could try to measure it in terms of Shannons if you really wanted, but that’s not what the scholarly literature uses for these types of exams.
If you assign a numeric value to something you call “meaning”, there must be a unit measure.
For example, consider IQ points. The unit measure (one IQ point) is 1⁄15 of the standard deviation of the distribution of IQ scores (which are basically normalised ranks) in white populations. We can argue what it corresponds to in the underlying reality, but at least it is well-defined.
The unit measure, however, may be entirely defined by the scale in question. Gleb could say “my unit measure is 1⁄9 of the full range of possible answers on my Meaning Measuring Scale”. That would be pretty useless, but so is “my unit measure is 1⁄15 of one standard deviation on my Cleverness Measuring Scale”.
Perhaps it’s better to reference scores to standard deviation rather than the full possible range—I guess it depends on the particular case. The other advantage IQ has is that there are now lots of IQ tests and they tend to get somewhat correlated results (suggesting that maybe they’re measuring something real) that also correlate with other interesting things (suggesting that maybe the real thing they’re measuring is useful) and they get used quite a lot (so that if you quote an IQ score there’s a good chance that the people you’re addressing will understand roughly what you mean).
Those are all genuine (or at least possibly-genuine) ways in which IQ scores are more useful than Gleb Meaningfulness Metric scores. But I don’t see that IQ is any better off than Gleb Meaningfulness Metric in terms of unit-measure-having. One IQ point doesn’t correspond to a fixed increment in thinking speed or memory capacity or ability to solve any particular kind of problem, or anything like that; it’s just a certain fraction of how much variation there is in one kind of brainpower score. One Gleb Meaningfulness Metric point, likewise, is just a certain fraction of how much variation is possible in one kind of feeling-like-your-life-has-meaning score.
Yep, that highly depends on the shape of the distribution.
Well, we (or at least I) haven’t seen Gleb’s Meaninfulness Metric, so I have no idea if it’s defined via population standard deviation like IQ. It may or it may be. I brought up IQ as an example of a unit which does not directly correspond to, say, thinking speed or working memory capacity—it’s entangled with the test itself, but it does make the numbers interpretable.
This is a nicety; while appreciated it looks like you are trying to suck up to Nancy.
(taking into account what Lumifier said about the app) in your description;
Is the only thing you said are doing to be checking on what you’re actually teaching. And it’s something that has not happened yet. (granted these things take time). I read the entire post as; “nothing yet, but we want to—in these ways...”.
depending on the method (if done with web content) could be described as A/B split testing. Which is standard these days for internet behaviour of groups spreading clickbait, not an accountable test.
I would suggest not taking Lumifer’s description of his initial impression of the sign-up page as indicative of the app itself. I think there’s sufficient evidence of Lumifer being not unbiased in describing Intentional Insights content. So consider checking out the app itself.
Here are the study proposals themselves, which you can evaluate yourself for whether they are A/B split testing: 1, 2.
EDIT Forgot to mention, my comment on the nature of NancyLebovitz’s question had to do with my own desire to signal that this is a very important question to me, not give praise.
Aren’t Elo’s motivations obvious?
It could be you are actually asking why did anyone change the “approved” version. The answer is that there ain’t no such thing as “approved” versions—this is a wiki, open to edits. No one needs to ask for permission to change the page.
Nope, Elo’s motivations are not obvious to me. I don’t want to suffer from the typical mind fallacy here, so I used the first virtue of curiosity to get more information.
So let’s try the obvious default thing: Elo thinks that the old version does not reflect the character of InIn properly and changed the page to make it—from her point of view—better / more accurate. Any particular reason you find the obvious motivation unsatisfying?
Really really sounds like you are using these things to imply, “I am more rational than the others on this forum”, compliments to @Lumifier for not taking it that way.
“Elo’s motivations are not obvious to me. I am asking because I was curious about why he did this”
Would be a better way to say the same thing, without saying “I used the virtue and mentioned a fallacy (that’s all about what rationality is about right?), reward me with internet gold”.
Why assume the ugliest possible interpretation of his meaning?
I would call this an example of tell culture.
Was the phrasing used, instead of (“you are...”). To try to point out what it sounds like he is saying… (probably could have said—“to me it sounds like...”, but that should be a given—I am posting the post...)
Ugliest or not; it’s an interpretation that I chose to share. I also tried to improve the statement to make it less “ugly”.
I don’t know—should I not do that at all?
Can you help with what it “sounds like” I am doing? (keen to learn and change)
I don’t oppose drawing interpretations, but the whole InIn discussion has been tainted by opponents attacking each other’s motivations for what they said instead of the content of what they said.
True.
in light of that: “to me it sounds like...” is probably more appropriate.
My state is that Gleb is trying to do good things—i.e. raise the sanity waterline (as a goal).
As yet he has made a lot of noise and not shown success in the process; or produced content worthy of respect. It remains to be seen if he:
improves;
continues to further offend seasoned members (through various methods i.e. weak content); or
quits.
Given those seem like the options; I would rather 1 then 3 then 2.
I expect from glebs perspective it looks like:
invest more effort into the content to the point where it seems not worth it
fight the haters and win, then keep doing more of what has happened.
give up on a serious dream to improve the world.
where his preference order is 2, 1, 3.
I hope we can meet in the middle; and given that “fight the haters and win on the internet” is a statement of comedy itself, I am keen to see the content improve.
I am not sure there is agreement about the direction of improvement.
Gleb has posted how he finds it difficult to write sleazy scummy content, but overpowers his reluctance and through great personal sacrifice does write it. I would expect that “improvement” for him means more concentrated snake oil or, perhaps, less personal discomfort with producing it.
I don’t think this is what Elo would consider “improvement”.
I don’t think it’s okay to put those words (“sleazy scummy concentrated snake oil”) in someone else’s mouth, unless it is part of an actual quote.
I am not quoting Gleb, I’m rephrasing his comment in my own words and from my own point of view. I think this is his original comment, but he repeated this in other places as well.
Gleb described having had to overpower reluctance to write in the style that publications like Lifehacker want, expressed some reservations about that style in morally-neutral language, and gave reasons for using it anyways. Separately, you and others (but not Gleb) described that style as sleazy and scummy. Mix these two things together and discard the attributions, and you’ve created the impression that Gleb thinks of the style as sleazy and scummy, and writes in it anyways. That would reflect negatively on his character if it were true, but it isn’t. Having to use an actual quote would have made this mistake impossible.
I wasn’t trying for any gotchas, my point was—and remains—that in this case the word “improvement” is likely to be understood differently by different people.
I am not passing any moral judgments or making insinuations about anyone’s character.
By the way, if you’re aiming for accuracy, the verb Gleb used was “cringe”. You chose to render it as “reluctance” which, I think, also changes the flavour albeit in another direction. It’s up to you to decide whether cringing and still writing “would reflect negatively on his character” :-P
that sounds like 2 not 1. yes. Now what (should be done)?
Information can be found in the talk page:
https://wiki.lesswrong.com/wiki/Talk:Intentional_Insights
And thank you @Lumifier for being reasonable.