Also known as Max Harms. (I post AI alignment content under my other account.)
Not the same person as MaxH!
Raelifin
Karma: 379
Improving on the Karma System
I also want to note that this proposal isn’t mutually exclusive with other ideas, including other karma systems. It seems fine to have there be an additional indicator of popularity that is distinct from quality. Or, more to my liking, would be a button that simply marks that you thought a post was interesting and/or express gratitude towards the writer, without making a statement about how bulletproof the reasoning was. (This might help capture the essence of Rule Thinkers In, Not Out and reward newbies for posting.)
One obvious flaw with this proposal is that the quality-indicator would only be a measure of expected rating by a moderator. But who says that our moderators are the best judges of quality? Like, the scheme is ripe for corruption, and simply pushing the popularity contest one level up to a small group of elites.
One answer is that if you don’t like the mods, you can go somewhere else. Vote with your feet, etc.
A more turtles-all-the-way-down answer is that the stakeholders of LW (the users, and possibly influential community members/investors?) agree on an aggregate set of metrics for how well the moderators are collectively capturing quality. Then, for each unit of time (eg year) and each potential moderator, set up a conditional prediction market with real dollars on whether that person being a moderator causes the metrics to go up/down compared to the previous time unit. Hire the ones that people predict will be best for the site.
To my mind the primary features of this system that bear on Duncan’s top-level post are:
High-reputation judges can confidently set the quality signal for a piece of writing, even if they’re in the minority. The truth is not a popularity contest, even when it comes to quality.
The emphasis on betting means that people who “upvote” low-quality posts or “downvote” high-quality ones are punished, making “this made me feel things, and so I’m going to bandwagon” a dangerous mental move. And people who make this sort of move would be efficiently sidelined.
In concert, I expect that it would be much easier to bring concentrated force down on low-quality bits of writing. Which would, in turn, I think make the quality price/signal a much more meaningful piece of information, instead of the current karma score which is as others noted, is overloaded as a measure.
First of all, thank you, Duncan, for this post. I feel like it captures important perspectives that I’ve had, and problems that I can see and puts them together in a pretty good way. (I also share your perspective that the post Could Be Better in several ways, but I respect you not letting the perfect be the enemy of the good.)
I find myself irritated right now (bothered, not angry) that our community’s primary method of highlighting quality writing is by karma-voting. It’s a similar kind of feeling to living in a democracy—yes, there are lots of systems that are worse, but really? Is this really the best we can do? (No particular shade on Ruby or the Lightcone team—making things is hard and I’m certainly glad LW exists and is as good as it is.)
Like, I think I have an idea that might make things substantially better that’s not terrible: make the standard signal for quality being a high price on a quality-arbitrated betting market. This is essentially applying the concept of Futarchy to internet forums (h/t ACX and Hanson). (If this is familiar to you, dear reader, feel free to skip to responses to this comment, where I talk about features of this proposal and other ideas.) Here’s how I could see it working:
When a user makes a post or comment or whatever, they also name a number between 0 and 100. This number is essentially a self-assessment of quality, where 0 means “I know this is flagrant trolling” and 100 means “This is obviously something that any interested party should read”. As an example, let’s say that I assign this comment an 80.Now let’s say that you are reading and you see my comment and think “An 80? Bah! More like a 60!” You can then “downvote” the comment, which nudges the number down, or enter your own (numeric) estimate, which dramatically shifts the value towards your estimate (similar to a “strong” vote). Behind the scenes, the site tracks the disagreement. Each user is essentially making a bet around the true value of the post’s quality. (The downvote is a bet that it’s “less than 80″.) What are they betting? Reputation as judges! New users start 0 judge-of-quality-reputation, unless they get existing users to vouch for them and donate a bit of reputation. (We can call this “karma,” but I think it is very important to distinguish good-judge karma, from high-quality-writing karma!) When voting/betting on a post/comment, they stake some of that reputation (maybe 10% up to a cap of 50? (Just making up numbers here for the sake of clarity; I’d suggest actually running experiments)).
Then, you have the site randomly sample pieces of writing, weighting the sampling towards those that are most controversial (ie have the most reputation on the line). Have the site assign these pieces of writing to moderators whose sole job is to study that piece of writing and the surrounding context and to score its quality. (Perhaps you want multiple moderators. Perhaps there should be appeals, in the form of people betting against the value set by the moderator. Etc. More implementation details are needed.) That judgment then resolves all the bets, and results in users gaining/losing reputation.
Users who run out of reputation can’t actually bet, and so lose the ability to influence the quality-indicator. However, all people who place bets (or try to place bets when at zero/negative reputation) are subsidized a small amount of reputation just for participating. (This inflation is a feature, encouraging participation in the site.) Thus, even a new user without any vouch can build up ability to influence the signal by participating and consistently being right.
Update: I decided that I like the grass south of the baseball diamond better. Let’s meet there.
Hey all, Max here. I was bad/busy on the weekend when I was supposed to provide a more specific location, so I’ve updated the what3words to a picnic table near the dog/skate park. I reserve the right to continue to adjust the meetup location in the coming weeks if I find even better places, so be sure to check on the 18th for specifics.
I’m an AI safety researcher and author of Crystal Society. I did a bunch of community leading/organizing in Ohio, including running a rationality dojo. I moved out to the bay area in 2016, and to Grass Valley in June. If you feel like introducing yourself in the comments here, please do! (But also no pressure.)Do people want food? I’ll probably make it happen, so if you have preferences, let me know ahead of time by email or by comment here. (No need to request vegetarian options; that’s a given.)
Grass Valley, CA – ACX Meetups Everywhere 2021
Issue 2 is about to be fixed: https://github.com/Discordius/Lesswrong2/pull/188
I picked 7 Habits because it’s pretty clearly rationality in my eyes, but is distinctly not LW style Rationality. Perhaps I should have picked something worse to make my point more clear.
Ah, perhaps I misunderstood the negative perception. It sounds like you see him as incompetent, and since he’s working with a subject that you care about that registers as disgusting?
I can understand cringing at the content. Some of it registers that way to me, too. I think Gleb’s admitted that he’s still working to improve. I won’t bother copy-pasting the argument that’s been made elsewhere on the thread that the target audience has different tastes. It may be the case that InIn’s content is garbage.
I guess I just wanted to step in and second jsteinhardt’s comment that Gleb is a very growth-oriented and positive, regardless of whether his writing is good enough.
I agree! Having good intentions does not imply the action has net benefit. I tried to communicate in my post that I see this as a situation where failure isn’t likely to cause harm. Given that it isn’t likely to hurt, and it might help, I think it makes sense to support in general.
(To be clear: Just because something is a net positive (in expectation) clearly doesn’t imply one ought to invest resources in supporting it. Marginal utility is a thing, and I personally think there are other projects which have higher total expected-utility.)
Okay well it seems like I’m a bit late to the discussion party. Hopefully my opinion is worth something. Heads up: I live in Columbus Ohio and am one of the organizers of the local LW meetup. I’ve been friends with Gleb since before he started InIn. I volunteer with Intentional Insights in a bunch of different ways and used to be on the board of directors. I am very likely biased, and while I’m trying to be as fair as possible here you may want to adjust my opinion in light of the obvious factors.
So yeah. This has been the big question about Intentional Insights for its entire existence. In my head I call it “the purity argument”. Should “rationality” try to stay pure by avoiding things like listicles or the phrase “science shows”? Or is it better to create a bridge of content that will move people along the path stochastically even if the content that’s nearest them is only marginally better than swill? (<-- That’s me trying not to be biased. I don’t like everything we’ve made, but when I’m not trying to counteract my likely biases I do think a lot of it is pretty good.)
Here’s my take on it: I don’t know. Like query, I don’t pretend to be confident one way or the other. I’m not as scared of “horrific long-term negative impact”, however. Probably the biggest reason why is that rationality is already tainted! If we back off of the sacred word, I think we can see that the act of improving-how-we-think exists in academia more broadly, self-help, and religion. LessWrong is but a single school (so to speak) of a practice which is at least as old as philosophy.
Now, I think that LW style rationality is superior than other attempts at flailing at rationality. I think the epistemology here is cleaner than most academic stuff and is at least as helpful as general self-help (again: probably biased; YMMV). But if the fear is that Intentional Insights is going to spoil the broth, I’d say that you should be aware that things like https://www.stephencovey.com/7habits/7habits.php already exist. As Gleb has mentioned elsewhere on the thread, InIn doesn’t even use the “rationality” label. I’d argue that the worst thin InIn does to pollute the LW meme-pool is that there are links and references to LW (and plenty of other sources, too).
In other words, I think at worst* InIn is basically just another lame self-help thing that tells people what they want to hear and doesn’t actually improve their cognition (a.k.a. the majority of self-help). At best, InIn will out-compete similar things and serve as a funnel which pulls people along the path of rationality, ultimately making the world a nicer, more sane place. Most of my work with InIn has been for personal gain; I’m not a strong believer that it will succeed. What I do think, though, is that there’s enough space in the world for the attempt, the goal of raising the sanity waterline is a good one, and rationalists should support the attempt, even if they aren’t confident in success, instead of getting swept up in the typical-mind fallacy and ingroup/outgroup and purity biases.
* - Okay, it’s not the worst-case scenario. The worst-case scenario is that the presence of InIn aggravates the lords of the matrix into torturing infinite copies of all possible minds for eternity outside of time. :P
(EDIT: If you want more evidence that rationality is already a polluted activity, consider the way in which so many people pattern-match LW as a phyg.)
I just wanted to interject a comment here as someone who is friends with Gleb in meatspace (we’re both organizers of the local meetup). In my experience Gleb is kinda spooky in the way he actually updates his behavior and thoughts in response to information. Like, if he is genuinely convinced that the person who is criticizing him is doing so out of a desire to help make the world a more-sane place (a desire he shares) then he’ll treat them like a friend instead of a foe. If he thinks that writing at a lower-level than most rationality content is currently written will help make the world a better place, he’ll actually go and do it, even if it feels weird or unpleasant to him.
I’m probably biased in that he’s my friend. He certainly struggles with it sometimes, and fails too. Critical scrutiny is important, and I’m really glad that Viliam made this thread, but it kinda breaks my heart that this spirit of actually taking ideas seriously has led to Gleb getting as much hate as it has. If he’d done the status-quo thing and stuck to approved-activities it would’ve been emotionally easier.
(And yes, Gleb, I know that we’re not optimizing for warm-fuzzies. It still sucks sometimes.)
Anyway, I guess I just wanted to put in my two (biased) cents that Gleb’s a really cool guy, and any appearance of a status-hungry manipulator is just because he’s being agent-y towards good ends and willing to get his hands dirty along the way.
Impostor entries were generally more convincing than genuine responses. I chalk this up to impostors trying harder to convince judges.
But who knows? Maybe you were a vegetarian in a past life! ;)
You’re right, but I’m pretty confident that the difference isn’t significant. We should probably see it as evidence that rationalists omnivores are about as capable as rationalist vegetarians.
If we look at average percent of positive predictions (predictions that earn more than 0 points):
Omnivores: 51%
Vegetarians: 46%
If we look at non-negative predictions (counting 50% predictions):
Omnivores: 52%
Vegetarians: 49%
As Douglas_Knight points out, it’s only 10⁄12, a probability of ~0.016. In a sample of ~50 we should see about one person at that level of accuracy or inaccuracy, which is exactly what we see. I’m no more inclined to give #14 a medal than I am to call #43 a dunce. See the histogram I stuck on to the end of the post for more intuition about why I see these extreme results as normal.
I absolutely will fess up to exaggerating in that sentence for the sake of dramatic effect. Some judges, such as yourself, were MUCH less wrong. I hope you don’t mind me outing you as one of the people who got a positive score, and that’s a reflection of you being better calibrated. That said, if you say “I’m 70% confident” four times, and only get it right twice, that’s evidence that you were still (slightly) overconfident when you thought “decently able to discern genuine writing from fakery”.
In retrospect I ought to have included options closer to 50%. I didn’t expect that they’d be so necessary! You are absolutely right, though.
A big part of LessWrong, I think, is learning to overcome our mental failings. Perhaps we can use this as a lesson that the best judge writes down their credence before seeing the options, then picks the option that is the best match to what they wrote. I know that I, personally, try (and often fail) to use this technique when doing multiple-choice tests.
Every judge being close to 50% would be bizarre. If I flip 13 coins 53 times I would expect that many of those sets of 13 will stray from the 6.5/13 expected ratio. The big question is whether anyone scored high enough or low enough that we can say “this wasn’t just pure chance”.
Nice. Thank you. How would you feel about me writing a top-level post reconsidering alternative systems and brainstorming/discussing solutions to the problems you raised?