I am a time-traveler who came back from the world where it (super duper predictably) turned out that a lot of early bright hopes didn’t pan out and various things went WRONG and alignment was HARD and it was NOT SOLVED IN ONE SUMMER BY TEN SMART RESEARCHERS
I think these kinds of comments update readers’ beliefs in a bad, invalid way. The bad event (AGI ruin) is argued for by… a request for me to condition on testimony of a survivor of that bad event. Yes, I know the whole thing is tongue-in-cheek. I know that EY is not literally claiming to be a time-traveller.
But in TurnTrout-culture, “I experienced X” is something to be said when X has actually been experienced. “The fact that X” is to be said when X is actually supported by a heap of accepted evidence.[1] “Have you met dath ilani?” is to be said when such entities actually exist and are not outputs of the model of intelligence which is being argued for. (Yes, that last one was flagged as a “bad argument”, but still.)
This paragraph of EY self-fic didn’t update me at all. But it almost did. When these statements are made, I am inclined to update my beliefs in the predictable way—to gullibly update on claims—unless I take special effort to not update on (checks dialogue) fictional evidence. Which effort I do take (as a matter of reflex, at this point), but that effort is a cost imposed on me.
This particular kind of misleading statement wasn’t made in this dialogue, but I’ve seen it made erroneously-according-to-me in private correspondence with smart researchers.
I think these kinds of comments update readers’ beliefs in a bad, invalid way. The bad event (AGI ruin) is argued for by… a request for me to condition on testimony of a survivor of that bad event. Yes, I know the whole thing is tongue-in-cheek. I know that EY is not literally claiming to be a time-traveller.
But in TurnTrout-culture, “I experienced X” is something to be said when X has actually been experienced. “The fact that X” is to be said when X is actually supported by a heap of accepted evidence.[1] “Have you met dath ilani?” is to be said when such entities actually exist and are not outputs of the model of intelligence which is being argued for. (Yes, that last one was flagged as a “bad argument”, but still.)
This paragraph of EY self-fic didn’t update me at all. But it almost did. When these statements are made, I am inclined to update my beliefs in the predictable way—to gullibly update on claims—unless I take special effort to not update on (checks dialogue) fictional evidence. Which effort I do take (as a matter of reflex, at this point), but that effort is a cost imposed on me.
This particular kind of misleading statement wasn’t made in this dialogue, but I’ve seen it made erroneously-according-to-me in private correspondence with smart researchers.