After first read-through of your post the main thing that stuck with me was this:
But the thing is… rereading part of the Sequences, I feel Yudkowsky was making points about deep knowledge all along? Even the quote I just used, which I interpreted in my rereading a couple of weeks ago as being about making predictions, now sounds like it’s about the sort of negative form of knowledge that forbids “perpetual motion machines”.
This gives me an icky feeling.
(low confidence in the following parts of this comment)
It makes me think of the Bible. The “specifications” laid out in the bible are loosey-goosey enough that believers can always re-interpret such-and-such verse to actually mean whatever newer evidence permits. (I want to stress that I’m not drawing a parallel between unthinking Christian believers and anyone changing their belief based upon new evidence! I’m drawing a parallel between the difficult task of writing text designed to change future behavior.)
If it’s so loosey-goosey than what’s it good for?
That’s most definitely not to say that anything that you can re-interpret in the light of new evidence is full of shit. However, you’ve got to have a good and solid explanation for the discrepancy between your earlier and later interpretations. The importance of, and difficulty of producing, this explanation is probably based upon if we’re talking about a quantitative physics experiment or a complicated tome of reasoning, philosophy, rhetoric. The complicated tome case is important and hard because it’s so very hard to convey our most complicated thoughts in ways that are so explicit that we can’t interpret them in a multitude of ways.
I think producing the explanation of the discrepancy between earlier and later interpretations is likely full of cognitive booby traps.
I find myself confused by this comment. I’m going to try voicing this confusion as precisely as possible, so you can hopefully clarify it for me.
I am confused that you get an icky feeling from basically the most uncontroversial part of my post and Yudkowsky’s point. The part you’re quoting is just saying that Yudkowsky cares more about anticipation-constraining than predictions. Of course, predictions are a particular type of very strong anticipation-constraining, but saying “this is impossible” is not wishy-washy fake specification: if the impossible thing is done, that invalidates your hypothesis. So “no perpetual motion machines” is definitely anticipation-constraining in that sense, and can readily falsified.
I am confused because this whole anticipation constraing, especially saying what can’t be done, is very accepted in traditional Science. Yudkowsky says that Science Isn’t Strict Enough because he says that it allows any type of anticipation-constraining hypothesis to the rank of “acceptable hypothesis”: if it’s wrong, it will evenutally be falsified.
I am confused because you keep comparing deep knowledge with the sort of conclusions that can always be reinterpreted from new evidence, when my posts goes into a lot of details about how Yudkowsky writes about the anticipation-constraining aspect and how to be stricter with your hypothesis, not just allowing any non-disproved hypothesis the same level of credibility.
Also I feel that I should link to this post, where Yudkowsky argues that the whole “Religion is non-falsifiable” is actually a modern invention that it doesn’t make sense to retrofit into the past.
I’ll say a few different things and see if it helps:
I’m making a meta point about the particular form of what happened in the paragraph quoted, nothing specific about what Yudkowsky or you wrote.
Specifically, the form follows something like this pattern: Entity A writes stuff. Entity B thinks it means X. Entity A, Entity B, and many others discuss it for a long time to suss out what Entity A really means. Years later Entity A tells us (or we discover in some other way) what they really meant.
There’s nothing about that pattern that says that any entity was wrong or not useful or not Good, but it’s a pattern that causes an icky reaction from me.
An icky feeling doesn’t always mean Thing X is wrong or bad, just that Thing X pattern matches against enough things the person feeling ickyness has previously found to be wrong or bad. Imagine feeling ickyness about root canals. Things that hurt are generally bad, and feeling icky about them isn’t surprising, but sometimes things that hurt are good! (but just because something is icky doesn’t necessarily mean there’s good versions of the thing)
Whether or not the part I felt icky about was un-controversial seems mostly tangential to the point I was trying to make.
The actual content of what Yudkowsky or you wrote isn’t exactly what I’m talking about.
I’m not saying that what Yudkowsky or you wrote is wrong or right. (In fact, I think Yudkowsky and you seem correct!)
I’m not “comparing deep knowledge with the sort of conclusions that can always be reinterpreted from new evidence”. I’m talking about the pattern formed between Yudkowsky’s writing and your/our understanding of it regardless of the content/accuracy/virtuousness of the writing. In other words, the comparison is between something like general relativity(not committing to this being a good example, but hopefully it gestures in the correct direction) and insert-any-writing-that-you’ve-later-understood-to-mean-something-else.
It’s possible (even likely) that there is no solution to the problem of some human ideas not being conducive to transfer in human modes of communication from one mind to the other without also being subject to re-interpretation. In other words it’s possible that Yudkowsky conveyed his thoughts as well as is humanly possible. He’s certainly better at doing that than me.
Back and forth conversation in the wake of a post or posts will often clear up what the author really meant as part of the general process of conveying ideas. However, it’s surprising to me that Yudkowsky clarified what he really meant years later in the time, manner, and location that he did and that contributes to the icky feeling.
I often get the sense that Yudkowsky is also frustrated by the general idea I’m gesturing at here. The difficulty of conveying ideas of a certain type. Not just that they’re difficult to convey, but that they’re difficult to convey in a manner that makes people confident in the accuracy while at the same time making them confident in the accuracy for the right reasons.
At this point, I’m hoping it makes sense to you when I say I don’t think Yudkowsky’s post about religion’s falsifiable-ness is exactly on-point.
I find myself unsatisfied with the content of this comment, but as of right now I’m not sure how to better convey my thoughts. On the other hand I don’t want to ignore your comment, so here’s hoping this helps rather than hinders.
Joke aside, I feel less confused after your clarifications. I think the issue is that it wasn’t clear at all to me that you were talking about the whole “interpreting Yudkowsky” schtick as the icky feeling.
Now it makes sense, and I definitely agree with you that there are enormous parallel with Biblical analysis. Yudkowsky’s writing is very biblical in ways IMO (the parables and the dialogues), and in general is far more literary than 99% of the rat writing out there. I’m not surprised he found HPMOR easy to write, his approach to almost everything seem like a mix of literary fiction and science-fiction tropes/ideas.
Which is IMO why this whole interpretation is so important. More and more, I think I’m understanding why so many people get frustrated with Yudkowsky’s writing and points: because they come expecting essays with arguments and a central point, and instead they get a literary text that requires strong interpretation before revealing what it means. I expect your icky feeling to come from the same place.
(Note that I think Yudkowsky is not doing that to be obscure, but for a mix of “it’s easier for him” and “he believes that you only learn and internalize the sort of knowledge he’s trying to convey through this interpretative labor, if not on the world itself, at least on his text”.)
Also, as a clarifier: I’m not comparing the content of literary fiction or the Bible to Yudkowsky’s writing. Generally with analysis of the former, you either get mysterious answers or platitudes; more and more with Yudkwosky I’m getting what I feel are deep insights (and his feedback on this post make me think that I’m not off the mark by much for some of those).
After first read-through of your post the main thing that stuck with me was this:
This gives me an icky feeling.
(low confidence in the following parts of this comment)
It makes me think of the Bible. The “specifications” laid out in the bible are loosey-goosey enough that believers can always re-interpret such-and-such verse to actually mean whatever newer evidence permits. (I want to stress that I’m not drawing a parallel between unthinking Christian believers and anyone changing their belief based upon new evidence! I’m drawing a parallel between the difficult task of writing text designed to change future behavior.)
If it’s so loosey-goosey than what’s it good for?
That’s most definitely not to say that anything that you can re-interpret in the light of new evidence is full of shit. However, you’ve got to have a good and solid explanation for the discrepancy between your earlier and later interpretations. The importance of, and difficulty of producing, this explanation is probably based upon if we’re talking about a quantitative physics experiment or a complicated tome of reasoning, philosophy, rhetoric. The complicated tome case is important and hard because it’s so very hard to convey our most complicated thoughts in ways that are so explicit that we can’t interpret them in a multitude of ways.
I think producing the explanation of the discrepancy between earlier and later interpretations is likely full of cognitive booby traps.
I find myself confused by this comment. I’m going to try voicing this confusion as precisely as possible, so you can hopefully clarify it for me.
I am confused that you get an icky feeling from basically the most uncontroversial part of my post and Yudkowsky’s point. The part you’re quoting is just saying that Yudkowsky cares more about anticipation-constraining than predictions. Of course, predictions are a particular type of very strong anticipation-constraining, but saying “this is impossible” is not wishy-washy fake specification: if the impossible thing is done, that invalidates your hypothesis. So “no perpetual motion machines” is definitely anticipation-constraining in that sense, and can readily falsified.
I am confused because this whole anticipation constraing, especially saying what can’t be done, is very accepted in traditional Science. Yudkowsky says that Science Isn’t Strict Enough because he says that it allows any type of anticipation-constraining hypothesis to the rank of “acceptable hypothesis”: if it’s wrong, it will evenutally be falsified.
I am confused because you keep comparing deep knowledge with the sort of conclusions that can always be reinterpreted from new evidence, when my posts goes into a lot of details about how Yudkowsky writes about the anticipation-constraining aspect and how to be stricter with your hypothesis, not just allowing any non-disproved hypothesis the same level of credibility.
Also I feel that I should link to this post, where Yudkowsky argues that the whole “Religion is non-falsifiable” is actually a modern invention that it doesn’t make sense to retrofit into the past.
Now I’m confused about why you’re confused!
I’ll say a few different things and see if it helps:
I’m making a meta point about the particular form of what happened in the paragraph quoted, nothing specific about what Yudkowsky or you wrote.
Specifically, the form follows something like this pattern: Entity A writes stuff. Entity B thinks it means X. Entity A, Entity B, and many others discuss it for a long time to suss out what Entity A really means. Years later Entity A tells us (or we discover in some other way) what they really meant.
There’s nothing about that pattern that says that any entity was wrong or not useful or not Good, but it’s a pattern that causes an icky reaction from me.
An icky feeling doesn’t always mean Thing X is wrong or bad, just that Thing X pattern matches against enough things the person feeling ickyness has previously found to be wrong or bad. Imagine feeling ickyness about root canals. Things that hurt are generally bad, and feeling icky about them isn’t surprising, but sometimes things that hurt are good! (but just because something is icky doesn’t necessarily mean there’s good versions of the thing)
Whether or not the part I felt icky about was un-controversial seems mostly tangential to the point I was trying to make.
The actual content of what Yudkowsky or you wrote isn’t exactly what I’m talking about.
I’m not saying that what Yudkowsky or you wrote is wrong or right. (In fact, I think Yudkowsky and you seem correct!)
I’m not “comparing deep knowledge with the sort of conclusions that can always be reinterpreted from new evidence”. I’m talking about the pattern formed between Yudkowsky’s writing and your/our understanding of it regardless of the content/accuracy/virtuousness of the writing. In other words, the comparison is between something like general relativity(not committing to this being a good example, but hopefully it gestures in the correct direction) and insert-any-writing-that-you’ve-later-understood-to-mean-something-else.
It’s possible (even likely) that there is no solution to the problem of some human ideas not being conducive to transfer in human modes of communication from one mind to the other without also being subject to re-interpretation. In other words it’s possible that Yudkowsky conveyed his thoughts as well as is humanly possible. He’s certainly better at doing that than me.
Back and forth conversation in the wake of a post or posts will often clear up what the author really meant as part of the general process of conveying ideas. However, it’s surprising to me that Yudkowsky clarified what he really meant years later in the time, manner, and location that he did and that contributes to the icky feeling.
I often get the sense that Yudkowsky is also frustrated by the general idea I’m gesturing at here. The difficulty of conveying ideas of a certain type. Not just that they’re difficult to convey, but that they’re difficult to convey in a manner that makes people confident in the accuracy while at the same time making them confident in the accuracy for the right reasons.
At this point, I’m hoping it makes sense to you when I say I don’t think Yudkowsky’s post about religion’s falsifiable-ness is exactly on-point.
I find myself unsatisfied with the content of this comment, but as of right now I’m not sure how to better convey my thoughts. On the other hand I don’t want to ignore your comment, so here’s hoping this helps rather than hinders.
Oh no, confusion is going foom!
Joke aside, I feel less confused after your clarifications. I think the issue is that it wasn’t clear at all to me that you were talking about the whole “interpreting Yudkowsky” schtick as the icky feeling.
Now it makes sense, and I definitely agree with you that there are enormous parallel with Biblical analysis. Yudkowsky’s writing is very biblical in ways IMO (the parables and the dialogues), and in general is far more literary than 99% of the rat writing out there. I’m not surprised he found HPMOR easy to write, his approach to almost everything seem like a mix of literary fiction and science-fiction tropes/ideas.
Which is IMO why this whole interpretation is so important. More and more, I think I’m understanding why so many people get frustrated with Yudkowsky’s writing and points: because they come expecting essays with arguments and a central point, and instead they get a literary text that requires strong interpretation before revealing what it means. I expect your icky feeling to come from the same place.
(Note that I think Yudkowsky is not doing that to be obscure, but for a mix of “it’s easier for him” and “he believes that you only learn and internalize the sort of knowledge he’s trying to convey through this interpretative labor, if not on the world itself, at least on his text”.)
Also, as a clarifier: I’m not comparing the content of literary fiction or the Bible to Yudkowsky’s writing. Generally with analysis of the former, you either get mysterious answers or platitudes; more and more with Yudkwosky I’m getting what I feel are deep insights (and his feedback on this post make me think that I’m not off the mark by much for some of those).