Just as an FYI: pinging us on Intercom is a much more reliable way of ensuring we see feature suggestions or bug reports than posting comments. Most feature suggestions won’t be implemented[1]; bug reports are prioritized according to urgency/impact and don’t always rise to the level of “will be addressed” (though I think >50% do).
At least not as a result of a single person suggesting them; we have ever made decisions that were influenced on the margin by suggestions from one or more LW users.
Genuine question: Why Intercom? What’s so good about it?
It advertises urgency (“we’ll be back in X hours”), which seems unnecessary for almost all bug reports and feature requests. When I post a non-time-sensitive bug report, I just want to know that someone will look at it eventually; I don’t need a reply within 24h from LW team members whose time is valuable.
My list of previous Intercom messages (20 threads total) is sorted in chronological order of last reply, cannot be rearranged, has no subject headings and is unsearchable. So I have to click into each thread to know what it’s about.
I can’t delete or archive old Intercom threads, so this list becomes increasingly unwieldy over time.
Old Intercom threads have poor timestamps, only being accurate to a week.
Or consider this bug report, which I forwarded on Intercom, but which only got resolved because another user contributed their experience in the comments. That couldn’t have happened if the bug report was restricted to a 1-to-1 chat within Intercom.
Re: reliability & follow-ups:
I asked to have the Filan podcast transcript not set to Personal Blog status, and didn’t receive a reply.
Or, a year ago, I reported a (non-time-sensitive) bug that I was unable to use the Reset Password button. Someone took a look, I mentioned it wasn’t time-sensitive, and the exchange ended on me saying “Just let me know when it becomes possible again to reset my password.” and them saying “Cool, will do”. Naturally I never received a further update on this; the button just got fixed at some point. I’m mentioning this not to blame the LW team member, but to indicate that the Intercom medium is just not the right tool for anything that requires long-term follow-up.
Similarly, I just bumped another Intercom question which I’d originally asked in August.
Re: issue trackers:
The second-ever Intercom message I received (322w / 6 years ago) began with: “Hey! If you’ve found a bug on the site, please feel free to file it as an issue on our Github.” So at least back then, Github did get used. What changed?
I’ve made a few bug reports where the Intercom reply (IIRC by you) was “Will put it in the queue”. Which is appreciated, but which also implies that there is a queue, which kind of sounds like an issue tracker (?), except that it’s not public.
Intercom has the benefit of acting as an inbox on our side, unlike comments posted on LW (which may not be seen by any LW team member).
In an ideal world, would Github Issues be better for tracking bug reports? Probably, yes. But Github Issues require that the user reporting an issue navigate to a different page and have a Github account, which approximately makes it a non-starter as the top-of-funnel.
Intercom’s message re: response times has some limited configurability but it’s difficult to make it say exactly the right thing here. Triaging bug reports from Intercom messages is a standard part of our daily workflow,so you shouldn’t model yourself as imposing unusual costs on the team by reporting bugs through Intercom.
re: reliability—yep, we are not totally reliable here. There are probably relatively easy process improvements here that we will end up not implementing because figuring out & implementing such process improvements takes time, which means it’s competing with everything else we might decide to spend time on. Nevertheless I’m sorry about the variety of dropped balls; it’s possible we will try to improve something here.
re: issue tracker—right now our process is approximately “toss bugs into a dedicated slack channel, shared with the EA forum”. The EA forum has a more developed issue-tracking process, so some of those do find their way to Github Issues (eventually).
Thanks for the reply. I think we’ve reached the limits of what can be discussed in a comment thread. Would you be interested in doing a dialogue on this topic? I’m thinking of a somewhat broader phrasing, something like: “Would better support for open-source contributions free up or cost LW team resources?” or “LW and open-source contributions: costs & benefits”, or similar.
(And, re: “I’m sorry about the variety of dropped balls”, I want to be clear that I appreciate everything you and the team do, and I understand that you’re a small team with a big mission. The reason why I gave examples of when the Intercom process was less than 100% reliable was not meant as blame, but just to support my argument that the tool seems ill-suited for certain kinds of reliability, like follow-ups.)
If the Github issue tracker is indeed not in use, then I find that very disappointing.
Intercom may be a more reliable channel for reporting bugs than the alternatives (though even on Intercom, I’ve still had things slip through the cracks), but it can’t replace an issue tracker. Besides, not all feedback constitutes bug reports; others require a back-and-forth, or input from multiple people, or follow-up to ask “hey, what’s the status here?”, and all of that works much better when it’s asynchronous and in public, not in a private chat interface.
And this very comment thread is a good illustration for why open thread comments also don’t work for this purpose: they might not get noticed; the threads are kind of ephemeral; feedback is mixed with non-feedback; the original poster has no way to keep track of their feedback (I had to skim through all my recent comments to find the ones that were feedback); not everyone related to an issue gets notified when someone comments on the issue; if issues are discussed in disparate threads, there’s no bi-directional crosslinking (if Github issue A links to B, then B displays the link, too); etc.
Ultimately whatever tools the LW team use to manage the website development may work well for them. But when I want to help as an outsider, I feel like the tools I’m given are not up to snuff.
It seems to me like a public issue tracker is an obvious solution to this problem, so I’m kind of incredulous that there isn’t really one. What gives?
It’s (as a descriptive fact) not a priority to support external contributions to the codebase. My guess is that it’s also correct not to prioritise that.
I understand that that’s obviously the counter-perspective, it just seems so wild to me. I’d love to see or do a dialogue on this, with anyone on the team where it would matter if they changed their mind on deprioritising this topic.
Just as an FYI: pinging us on Intercom is a much more reliable way of ensuring we see feature suggestions or bug reports than posting comments. Most feature suggestions won’t be implemented[1]; bug reports are prioritized according to urgency/impact and don’t always rise to the level of “will be addressed” (though I think >50% do).
At least not as a result of a single person suggesting them; we have ever made decisions that were influenced on the margin by suggestions from one or more LW users.
Genuine question: Why Intercom? What’s so good about it?
It advertises urgency (“we’ll be back in X hours”), which seems unnecessary for almost all bug reports and feature requests. When I post a non-time-sensitive bug report, I just want to know that someone will look at it eventually; I don’t need a reply within 24h from LW team members whose time is valuable.
My list of previous Intercom messages (20 threads total) is sorted in chronological order of last reply, cannot be rearranged, has no subject headings and is unsearchable. So I have to click into each thread to know what it’s about.
I can’t delete or archive old Intercom threads, so this list becomes increasingly unwieldy over time.
Old Intercom threads have poor timestamps, only being accurate to a week.
Or consider this bug report, which I forwarded on Intercom, but which only got resolved because another user contributed their experience in the comments. That couldn’t have happened if the bug report was restricted to a 1-to-1 chat within Intercom.
Re: reliability & follow-ups:
I asked to have the Filan podcast transcript not set to Personal Blog status, and didn’t receive a reply.
Or, a year ago, I reported a (non-time-sensitive) bug that I was unable to use the Reset Password button. Someone took a look, I mentioned it wasn’t time-sensitive, and the exchange ended on me saying “Just let me know when it becomes possible again to reset my password.” and them saying “Cool, will do”. Naturally I never received a further update on this; the button just got fixed at some point. I’m mentioning this not to blame the LW team member, but to indicate that the Intercom medium is just not the right tool for anything that requires long-term follow-up.
Similarly, I just bumped another Intercom question which I’d originally asked in August.
Re: issue trackers:
The second-ever Intercom message I received (322w / 6 years ago) began with: “Hey! If you’ve found a bug on the site, please feel free to file it as an issue on our Github.” So at least back then, Github did get used. What changed?
I’ve made a few bug reports where the Intercom reply (IIRC by you) was “Will put it in the queue”. Which is appreciated, but which also implies that there is a queue, which kind of sounds like an issue tracker (?), except that it’s not public.
Intercom has the benefit of acting as an inbox on our side, unlike comments posted on LW (which may not be seen by any LW team member).
In an ideal world, would Github Issues be better for tracking bug reports? Probably, yes. But Github Issues require that the user reporting an issue navigate to a different page and have a Github account, which approximately makes it a non-starter as the top-of-funnel.
Intercom’s message re: response times has some limited configurability but it’s difficult to make it say exactly the right thing here. Triaging bug reports from Intercom messages is a standard part of our daily workflow,so you shouldn’t model yourself as imposing unusual costs on the team by reporting bugs through Intercom.
re: reliability—yep, we are not totally reliable here. There are probably relatively easy process improvements here that we will end up not implementing because figuring out & implementing such process improvements takes time, which means it’s competing with everything else we might decide to spend time on. Nevertheless I’m sorry about the variety of dropped balls; it’s possible we will try to improve something here.
re: issue tracker—right now our process is approximately “toss bugs into a dedicated slack channel, shared with the EA forum”. The EA forum has a more developed issue-tracking process, so some of those do find their way to Github Issues (eventually).
Thanks for the reply. I think we’ve reached the limits of what can be discussed in a comment thread. Would you be interested in doing a dialogue on this topic? I’m thinking of a somewhat broader phrasing, something like: “Would better support for open-source contributions free up or cost LW team resources?” or “LW and open-source contributions: costs & benefits”, or similar.
(And, re: “I’m sorry about the variety of dropped balls”, I want to be clear that I appreciate everything you and the team do, and I understand that you’re a small team with a big mission. The reason why I gave examples of when the Intercom process was less than 100% reliable was not meant as blame, but just to support my argument that the tool seems ill-suited for certain kinds of reliability, like follow-ups.)
Crossposting my comment from here:
kave’s reply:
my reply: