I still like the site, though I had to set the AI tag to −100 this year. One thing I wish was a bit different is that I’ve posted a whole bunch of LW-site-relevant feedback in comments (my natural inclination is to post comprehensive feedback on whatever content I interact with), and for a good fraction of them I’ve received no official reaction whatsoever. I don’t know if the comments got ignored because the LW team didn’t see them, or didn’t have the time to act on them, or whatever, but I still wish I’d gotten some kinds of reactions on them.
I’m not asking for my feedback to be implemented[1], but when I post feedback comments on site posts by LW team members, I do wish I got some kind of acknowledgement, even if it always turned out to be “we’ve seen this feedback, but we have bigger fish to fry”.
As an example, here are all my unanswered feedback requests since 2023-08-01 (arbitrary cutoff from when I got bored browsing my comments history):
Most important: My comment thread which requested that the Forum Magnum Github list of open issues be cleaned up, and which also included a smaller & actionable ask to sync the issue statuses from the private Lightcone Asana back to the public Github.
Incidentally, lack of reactions to some of my feedback is one reason why I wish that issue tracker was useful, rather than being in the sorry state it’s in. My LW-related feedback comments are like posting Github issues on an open-source project: i.e. I try to make the site better, and expect a reply like “we’ll implement this” or “we won’t implement this” or “we haven’t seen this, but someone will look at this eventually” or something. Plus I want to be able to check back on the issues months later, and neither comments nor Intercom are the right format for that. Whereas on Github, one can easily find all one’s feedback in one place; the status of each issue is transparently clear; and I get update notifications.
The New User’s Guide to LessWrong asked for feedback, I posted three huge feedback comments (1, 2, 3), and I’m not sure anyone ever saw them, let alone reacted to them.
Or this comment on the Carving of Reality book where I pointed out that the books actually can be purchased in a few countries apart from the US, and so the FAQ warranted an update. Raemon even responded to the thread, but the FAQ is still incomplete as written.
This question on an Open Thread: “What’s the minimum one would have to learn to productively contribute to open Github issues in the LW / EA Forum codebase?”
These questions on the process of making the “The Carving of Reality” book set.
Just as an FYI: pinging us on Intercom is a much more reliable way of ensuring we see feature suggestions or bug reports than posting comments. Most feature suggestions won’t be implemented[1]; bug reports are prioritized according to urgency/impact and don’t always rise to the level of “will be addressed” (though I think >50% do).
At least not as a result of a single person suggesting them; we have ever made decisions that were influenced on the margin by suggestions from one or more LW users.
Genuine question: Why Intercom? What’s so good about it?
It advertises urgency (“we’ll be back in X hours”), which seems unnecessary for almost all bug reports and feature requests. When I post a non-time-sensitive bug report, I just want to know that someone will look at it eventually; I don’t need a reply within 24h from LW team members whose time is valuable.
My list of previous Intercom messages (20 threads total) is sorted in chronological order of last reply, cannot be rearranged, has no subject headings and is unsearchable. So I have to click into each thread to know what it’s about.
I can’t delete or archive old Intercom threads, so this list becomes increasingly unwieldy over time.
Old Intercom threads have poor timestamps, only being accurate to a week.
Or consider this bug report, which I forwarded on Intercom, but which only got resolved because another user contributed their experience in the comments. That couldn’t have happened if the bug report was restricted to a 1-to-1 chat within Intercom.
Re: reliability & follow-ups:
I asked to have the Filan podcast transcript not set to Personal Blog status, and didn’t receive a reply.
Or, a year ago, I reported a (non-time-sensitive) bug that I was unable to use the Reset Password button. Someone took a look, I mentioned it wasn’t time-sensitive, and the exchange ended on me saying “Just let me know when it becomes possible again to reset my password.” and them saying “Cool, will do”. Naturally I never received a further update on this; the button just got fixed at some point. I’m mentioning this not to blame the LW team member, but to indicate that the Intercom medium is just not the right tool for anything that requires long-term follow-up.
Similarly, I just bumped another Intercom question which I’d originally asked in August.
Re: issue trackers:
The second-ever Intercom message I received (322w / 6 years ago) began with: “Hey! If you’ve found a bug on the site, please feel free to file it as an issue on our Github.” So at least back then, Github did get used. What changed?
I’ve made a few bug reports where the Intercom reply (IIRC by you) was “Will put it in the queue”. Which is appreciated, but which also implies that there is a queue, which kind of sounds like an issue tracker (?), except that it’s not public.
Intercom has the benefit of acting as an inbox on our side, unlike comments posted on LW (which may not be seen by any LW team member).
In an ideal world, would Github Issues be better for tracking bug reports? Probably, yes. But Github Issues require that the user reporting an issue navigate to a different page and have a Github account, which approximately makes it a non-starter as the top-of-funnel.
Intercom’s message re: response times has some limited configurability but it’s difficult to make it say exactly the right thing here. Triaging bug reports from Intercom messages is a standard part of our daily workflow,so you shouldn’t model yourself as imposing unusual costs on the team by reporting bugs through Intercom.
re: reliability—yep, we are not totally reliable here. There are probably relatively easy process improvements here that we will end up not implementing because figuring out & implementing such process improvements takes time, which means it’s competing with everything else we might decide to spend time on. Nevertheless I’m sorry about the variety of dropped balls; it’s possible we will try to improve something here.
re: issue tracker—right now our process is approximately “toss bugs into a dedicated slack channel, shared with the EA forum”. The EA forum has a more developed issue-tracking process, so some of those do find their way to Github Issues (eventually).
Thanks for the reply. I think we’ve reached the limits of what can be discussed in a comment thread. Would you be interested in doing a dialogue on this topic? I’m thinking of a somewhat broader phrasing, something like: “Would better support for open-source contributions free up or cost LW team resources?” or “LW and open-source contributions: costs & benefits”, or similar.
(And, re: “I’m sorry about the variety of dropped balls”, I want to be clear that I appreciate everything you and the team do, and I understand that you’re a small team with a big mission. The reason why I gave examples of when the Intercom process was less than 100% reliable was not meant as blame, but just to support my argument that the tool seems ill-suited for certain kinds of reliability, like follow-ups.)
If the Github issue tracker is indeed not in use, then I find that very disappointing.
Intercom may be a more reliable channel for reporting bugs than the alternatives (though even on Intercom, I’ve still had things slip through the cracks), but it can’t replace an issue tracker. Besides, not all feedback constitutes bug reports; others require a back-and-forth, or input from multiple people, or follow-up to ask “hey, what’s the status here?”, and all of that works much better when it’s asynchronous and in public, not in a private chat interface.
And this very comment thread is a good illustration for why open thread comments also don’t work for this purpose: they might not get noticed; the threads are kind of ephemeral; feedback is mixed with non-feedback; the original poster has no way to keep track of their feedback (I had to skim through all my recent comments to find the ones that were feedback); not everyone related to an issue gets notified when someone comments on the issue; if issues are discussed in disparate threads, there’s no bi-directional crosslinking (if Github issue A links to B, then B displays the link, too); etc.
Ultimately whatever tools the LW team use to manage the website development may work well for them. But when I want to help as an outsider, I feel like the tools I’m given are not up to snuff.
It seems to me like a public issue tracker is an obvious solution to this problem, so I’m kind of incredulous that there isn’t really one. What gives?
It’s (as a descriptive fact) not a priority to support external contributions to the codebase. My guess is that it’s also correct not to prioritise that.
I understand that that’s obviously the counter-perspective, it just seems so wild to me. I’d love to see or do a dialogue on this, with anyone on the team where it would matter if they changed their mind on deprioritising this topic.
I still like the site, though I had to set the AI tag to −100 this year. One thing I wish was a bit different is that I’ve posted a whole bunch of LW-site-relevant feedback in comments (my natural inclination is to post comprehensive feedback on whatever content I interact with), and for a good fraction of them I’ve received no official reaction whatsoever. I don’t know if the comments got ignored because the LW team didn’t see them, or didn’t have the time to act on them, or whatever, but I still wish I’d gotten some kinds of reactions on them.
I’m not asking for my feedback to be implemented[1], but when I post feedback comments on site posts by LW team members, I do wish I got some kind of acknowledgement, even if it always turned out to be “we’ve seen this feedback, but we have bigger fish to fry”.
As an example, here are all my unanswered feedback requests since 2023-08-01 (arbitrary cutoff from when I got bored browsing my comments history):
Most important: My comment thread which requested that the Forum Magnum Github list of open issues be cleaned up, and which also included a smaller & actionable ask to sync the issue statuses from the private Lightcone Asana back to the public Github.
Incidentally, lack of reactions to some of my feedback is one reason why I wish that issue tracker was useful, rather than being in the sorry state it’s in. My LW-related feedback comments are like posting Github issues on an open-source project: i.e. I try to make the site better, and expect a reply like “we’ll implement this” or “we won’t implement this” or “we haven’t seen this, but someone will look at this eventually” or something. Plus I want to be able to check back on the issues months later, and neither comments nor Intercom are the right format for that. Whereas on Github, one can easily find all one’s feedback in one place; the status of each issue is transparently clear; and I get update notifications.
The New User’s Guide to LessWrong asked for feedback, I posted three huge feedback comments (1, 2, 3), and I’m not sure anyone ever saw them, let alone reacted to them.
Or there’s this minor feedback comment on your post on the Dialogue Matching feature.
Or this comment on the Carving of Reality book where I pointed out that the books actually can be purchased in a few countries apart from the US, and so the FAQ warranted an update. Raemon even responded to the thread, but the FAQ is still incomplete as written.
And this unofficial thread on the LW Wiki contains feedback by me and others, which I wish had received some kind of attention.
This question on an Open Thread: “What’s the minimum one would have to learn to productively contribute to open Github issues in the LW / EA Forum codebase?”
These questions on the process of making the “The Carving of Reality” book set.
This request for a retrospective for Good Heart Week 2022.
Finally, if someone on the LW team was interested, it could be neat to dialogue on a topic like “LW and open-source contributions”.
Though it occasionally does get implemented: like fixes to this bug report on comment reactions, or to a report on the comments counter being bugged in discussions.
Just as an FYI: pinging us on Intercom is a much more reliable way of ensuring we see feature suggestions or bug reports than posting comments. Most feature suggestions won’t be implemented[1]; bug reports are prioritized according to urgency/impact and don’t always rise to the level of “will be addressed” (though I think >50% do).
At least not as a result of a single person suggesting them; we have ever made decisions that were influenced on the margin by suggestions from one or more LW users.
Genuine question: Why Intercom? What’s so good about it?
It advertises urgency (“we’ll be back in X hours”), which seems unnecessary for almost all bug reports and feature requests. When I post a non-time-sensitive bug report, I just want to know that someone will look at it eventually; I don’t need a reply within 24h from LW team members whose time is valuable.
My list of previous Intercom messages (20 threads total) is sorted in chronological order of last reply, cannot be rearranged, has no subject headings and is unsearchable. So I have to click into each thread to know what it’s about.
I can’t delete or archive old Intercom threads, so this list becomes increasingly unwieldy over time.
Old Intercom threads have poor timestamps, only being accurate to a week.
Or consider this bug report, which I forwarded on Intercom, but which only got resolved because another user contributed their experience in the comments. That couldn’t have happened if the bug report was restricted to a 1-to-1 chat within Intercom.
Re: reliability & follow-ups:
I asked to have the Filan podcast transcript not set to Personal Blog status, and didn’t receive a reply.
Or, a year ago, I reported a (non-time-sensitive) bug that I was unable to use the Reset Password button. Someone took a look, I mentioned it wasn’t time-sensitive, and the exchange ended on me saying “Just let me know when it becomes possible again to reset my password.” and them saying “Cool, will do”. Naturally I never received a further update on this; the button just got fixed at some point. I’m mentioning this not to blame the LW team member, but to indicate that the Intercom medium is just not the right tool for anything that requires long-term follow-up.
Similarly, I just bumped another Intercom question which I’d originally asked in August.
Re: issue trackers:
The second-ever Intercom message I received (322w / 6 years ago) began with: “Hey! If you’ve found a bug on the site, please feel free to file it as an issue on our Github.” So at least back then, Github did get used. What changed?
I’ve made a few bug reports where the Intercom reply (IIRC by you) was “Will put it in the queue”. Which is appreciated, but which also implies that there is a queue, which kind of sounds like an issue tracker (?), except that it’s not public.
Intercom has the benefit of acting as an inbox on our side, unlike comments posted on LW (which may not be seen by any LW team member).
In an ideal world, would Github Issues be better for tracking bug reports? Probably, yes. But Github Issues require that the user reporting an issue navigate to a different page and have a Github account, which approximately makes it a non-starter as the top-of-funnel.
Intercom’s message re: response times has some limited configurability but it’s difficult to make it say exactly the right thing here. Triaging bug reports from Intercom messages is a standard part of our daily workflow,so you shouldn’t model yourself as imposing unusual costs on the team by reporting bugs through Intercom.
re: reliability—yep, we are not totally reliable here. There are probably relatively easy process improvements here that we will end up not implementing because figuring out & implementing such process improvements takes time, which means it’s competing with everything else we might decide to spend time on. Nevertheless I’m sorry about the variety of dropped balls; it’s possible we will try to improve something here.
re: issue tracker—right now our process is approximately “toss bugs into a dedicated slack channel, shared with the EA forum”. The EA forum has a more developed issue-tracking process, so some of those do find their way to Github Issues (eventually).
Thanks for the reply. I think we’ve reached the limits of what can be discussed in a comment thread. Would you be interested in doing a dialogue on this topic? I’m thinking of a somewhat broader phrasing, something like: “Would better support for open-source contributions free up or cost LW team resources?” or “LW and open-source contributions: costs & benefits”, or similar.
(And, re: “I’m sorry about the variety of dropped balls”, I want to be clear that I appreciate everything you and the team do, and I understand that you’re a small team with a big mission. The reason why I gave examples of when the Intercom process was less than 100% reliable was not meant as blame, but just to support my argument that the tool seems ill-suited for certain kinds of reliability, like follow-ups.)
Crossposting my comment from here:
kave’s reply:
my reply: