Someone downvoted the question above. What the hell? (My guess: it’s VoiceOfRa doing his downvote-the-enemy thing again.)
To the actual question: first of all, I think it’s entirely possible that we have additional layers (sys1 means “fast heuristic”, sys2 means “slow deliberate reasoning”; we surely have a big bag of heuristics, and I bet there are cases where we have extra-fast heuristics, fastish heuristics, and slow deliberate reasoning); and it seems like one could envisage an AI with (1) nothing like sys1 at all because its “proper” reasoning is cheap enough to be used all the time, (2) a human-like bag of heuristics that get used when circumstances allow, producing much the same distinction as we have, (3) smoothly varying how-much-approximation knobs that adjust according to how valuable quicker answers are, interpolating continuously between “system 1″ and “system 2”, and probably (4) all sorts of other things I haven’t thought of.
The sort of provably-safe AI that MIRI would like to see would presumably either be in category 1, or be designed so that in some sense sufficiently consequential decisions always get made “properly”. The latter seems like it would be hard to reason about. (Er, or it might be in category 4 in which case by definition I have nothing to say about it.)
Is retributive downvoting on other forums, or is it just a LW thing? Do we have more retributive downvoting than other sites? Can anyone think of some relationship between rationality and vindication? I feel like if anything we should be above that and have far less...
Is retributive downvoting on other forums, or is it just a LW thing?
Reddit also had it. I don’t frequent other forums that use voting, but a forum I used to be part of had a user that would delve into the history of people he disagreed with and report year-old comments to get those people banned.
Given that it’s an easy way to hinder “opponents” I very much doubt it’s LW exclusive.
Can anyone think of some relationship between rationality and vindication?
Apart from willingness to use tools others would think immoral, no. I also don’t think we need to go that far as an explanation. You only need one person doing it in a community as small as this one for it to become noticeable.
Ya know the funny thing is, I instinctively came here to upvote your reply. I suspect I would have done that even if you’re reply was of poor quality. Perhaps that could be construed as a form of retributive upvoting, in gratitude for the courtesy of replying. In that case, I would intuit that it is not of good practice, since it would equally skew the karma system (unless everyone is doing it, I suppose). Though, karma isn’t ahhh...can’t remember the economic term...replaceable by another unit of karma. There is a marginal value to karma and very different signals for negative/positive karma.
I think it’s not so much a LW thing as a “one particular person on LW” thing, in which case there needn’t be any particular connection with rationality or rationalism or anything else about LW apart from whatever circumstances led to us acquiring that one particular person. I expect (but have no evidence) that larger fora have occasional people doing pretty much any weird thing you might care to speculate about, including retributive downvoting. Maybe they get squashed by moderators or something. Maybe it’s usually a small enough fraction of voting activity that no one cares.
Hhmm, I wonder we as a community should volunteer a set of guidelines and norms about karmic behaviour as to aid the interpretation of karma. On the other hand, perhaps an intuitive system has its own charm.
There have been discussions of this before. I think the reason why we don’t have anything like official guidelines is that most of what one can say about how to vote is either obvious or controversial.
Here are my own principles:
Vote things up when they are particularly good (i.e., LW would be improved by having more things like them) and down when they are paricularly bad (i.e., LW would be improved by having fewer things like them).
Vote sparingly (think of my opinion as quality signal plus random noise, the noise depending on my own personality quirks, taste in writing style, etc.; vote only when the signal is significantly above the noise).
Vote to correct badly underrated/overrated things, but with even more circumspection (if this is done often, either the rating something ends up with depends strongly on the order in which different people see it or else you get multiple people revisiting each thing and changing their votes endlessly).
In cases of mere disagreement with a generally reasonable other party, prefer vocal disagreement to downvoting. Downvoting is better suited to cases where persuasion is unlikely to succeed.
Be especially reluctant to downvote things on account of political disagreement.
Vote according to the (de)merits of the thing being voted on, not its author.
For good or ill, many people find being downvoted personally hurtful (more so, I think, when their comment or post is rated negatively afterwards; going from 0 to −1 hurts more than going from +2 to +1). Accordingly, be just slightly more reluctant to downvote than to upvote at a given level of (positive or negative) quality.
I think almost everyone would agree with the first principle. The second and third seem obviously debatable, and in particular I know that some people think one should never take earlier votes into account when deciding what to do with a post or comment. The fourth and fifth seem more often agreed with than disagreed with in public, but it seems clear that not everyone acts that way and politically charged topics seem to attract more opinion-based voting. Almost everyone agrees with the sixth. I’m not even sure whether I agree with the seventh.
There’s no real way to enforce that. Even with those guidelines you’ll mostly end up with an intuitive system that’s maybe influenced by the guidelines.
Can anyone think of some relationship between rationality and vindication?
Sure, it’s straightforward humans-are-big-white-rats Skinnerian behaviourism. If you see behaviour you don’t want, apply pain until the behaviour stops. In a less crude form you’d call it “setting up an incentive system”.
Now that I think about it, we do have a range of speeds, including the occasional sudden revelation (as when an addict realizes that they really can and must stop).
I’d expect MIRI to build the “proper reasoning” layer, and then the AI would use its “proper reasoning” to design fast heuristic layers.
One example of a very-hard-to-avoid layering would be if the AI were distributed over large areas (global or perhaps interplanetary), there would probably have to be decisions made locally, but other decisions made globally. You could say the local processes are subagents and only the global process is the real AI, but it wouldn’t change the fact that the AI has designed trustworthy faster less-accurate decision processes.
Someone downvoted the question above. What the hell? (My guess: it’s VoiceOfRa doing his downvote-the-enemy thing again.)
To the actual question: first of all, I think it’s entirely possible that we have additional layers (sys1 means “fast heuristic”, sys2 means “slow deliberate reasoning”; we surely have a big bag of heuristics, and I bet there are cases where we have extra-fast heuristics, fastish heuristics, and slow deliberate reasoning); and it seems like one could envisage an AI with (1) nothing like sys1 at all because its “proper” reasoning is cheap enough to be used all the time, (2) a human-like bag of heuristics that get used when circumstances allow, producing much the same distinction as we have, (3) smoothly varying how-much-approximation knobs that adjust according to how valuable quicker answers are, interpolating continuously between “system 1″ and “system 2”, and probably (4) all sorts of other things I haven’t thought of.
The sort of provably-safe AI that MIRI would like to see would presumably either be in category 1, or be designed so that in some sense sufficiently consequential decisions always get made “properly”. The latter seems like it would be hard to reason about. (Er, or it might be in category 4 in which case by definition I have nothing to say about it.)
Is retributive downvoting on other forums, or is it just a LW thing? Do we have more retributive downvoting than other sites? Can anyone think of some relationship between rationality and vindication? I feel like if anything we should be above that and have far less...
Reddit also had it. I don’t frequent other forums that use voting, but a forum I used to be part of had a user that would delve into the history of people he disagreed with and report year-old comments to get those people banned.
Given that it’s an easy way to hinder “opponents” I very much doubt it’s LW exclusive.
Apart from willingness to use tools others would think immoral, no. I also don’t think we need to go that far as an explanation. You only need one person doing it in a community as small as this one for it to become noticeable.
Ya know the funny thing is, I instinctively came here to upvote your reply. I suspect I would have done that even if you’re reply was of poor quality. Perhaps that could be construed as a form of retributive upvoting, in gratitude for the courtesy of replying. In that case, I would intuit that it is not of good practice, since it would equally skew the karma system (unless everyone is doing it, I suppose). Though, karma isn’t ahhh...can’t remember the economic term...replaceable by another unit of karma. There is a marginal value to karma and very different signals for negative/positive karma.
Fungible?
I think it’s not so much a LW thing as a “one particular person on LW” thing, in which case there needn’t be any particular connection with rationality or rationalism or anything else about LW apart from whatever circumstances led to us acquiring that one particular person. I expect (but have no evidence) that larger fora have occasional people doing pretty much any weird thing you might care to speculate about, including retributive downvoting. Maybe they get squashed by moderators or something. Maybe it’s usually a small enough fraction of voting activity that no one cares.
Hhmm, I wonder we as a community should volunteer a set of guidelines and norms about karmic behaviour as to aid the interpretation of karma. On the other hand, perhaps an intuitive system has its own charm.
There have been discussions of this before. I think the reason why we don’t have anything like official guidelines is that most of what one can say about how to vote is either obvious or controversial.
Here are my own principles:
Vote things up when they are particularly good (i.e., LW would be improved by having more things like them) and down when they are paricularly bad (i.e., LW would be improved by having fewer things like them).
Vote sparingly (think of my opinion as quality signal plus random noise, the noise depending on my own personality quirks, taste in writing style, etc.; vote only when the signal is significantly above the noise).
Vote to correct badly underrated/overrated things, but with even more circumspection (if this is done often, either the rating something ends up with depends strongly on the order in which different people see it or else you get multiple people revisiting each thing and changing their votes endlessly).
In cases of mere disagreement with a generally reasonable other party, prefer vocal disagreement to downvoting. Downvoting is better suited to cases where persuasion is unlikely to succeed.
Be especially reluctant to downvote things on account of political disagreement.
Vote according to the (de)merits of the thing being voted on, not its author.
For good or ill, many people find being downvoted personally hurtful (more so, I think, when their comment or post is rated negatively afterwards; going from 0 to −1 hurts more than going from +2 to +1). Accordingly, be just slightly more reluctant to downvote than to upvote at a given level of (positive or negative) quality.
I think almost everyone would agree with the first principle. The second and third seem obviously debatable, and in particular I know that some people think one should never take earlier votes into account when deciding what to do with a post or comment. The fourth and fifth seem more often agreed with than disagreed with in public, but it seems clear that not everyone acts that way and politically charged topics seem to attract more opinion-based voting. Almost everyone agrees with the sixth. I’m not even sure whether I agree with the seventh.
There’s no real way to enforce that. Even with those guidelines you’ll mostly end up with an intuitive system that’s maybe influenced by the guidelines.
Sure, it’s straightforward humans-are-big-white-rats Skinnerian behaviourism. If you see behaviour you don’t want, apply pain until the behaviour stops. In a less crude form you’d call it “setting up an incentive system”.
I was wondering about VoR and that downvote.
Now that I think about it, we do have a range of speeds, including the occasional sudden revelation (as when an addict realizes that they really can and must stop).
I’d expect MIRI to build the “proper reasoning” layer, and then the AI would use its “proper reasoning” to design fast heuristic layers.
One example of a very-hard-to-avoid layering would be if the AI were distributed over large areas (global or perhaps interplanetary), there would probably have to be decisions made locally, but other decisions made globally. You could say the local processes are subagents and only the global process is the real AI, but it wouldn’t change the fact that the AI has designed trustworthy faster less-accurate decision processes.