Epistemic status: worried about effort/time lost.
I am by no means experienced with any of this, and seriously considered not writing anything at all. But it only takes me a bit of time (an hour max) to write why I feel why the odds are very strongly against you, and if you are serious about pursuing this idea then even if at low probability that my comment is helpful to you it’s worth writing it on average. So here we go.
During my read of the post, top-to-bottom, at the part
On matters of truth, it needs to support epistemic arguments for why we should believe or not believe particular claims. On matters of action, it needs to provide important pro/cons of taking that action. Site must have a method of allowing the best arguments to rise to the top.
my internal monologue went “The first bit is difficult but perhaps possible. The second is a mess. Oh dear, the third is basically impossible!”. The sentence immediately after, explaining this functionality would be the bare basics, shocked me quite a lot. I think aiming for the quoted section is nigh-impossible, and then we haven’t started on the possible additional features you mention. Your post strongly reminds me of Benjamin Hoffman’s piece on Anglerfish (in my opinion worth reading in full), and also a bit of a segment (near the start) in one of Eliezer’s posts on security mindset—where the character Amber makes the mistake of thinking that the critical part of her startup is the technology, where really it is the security. I think in a similar manner your Site would, besides depending on the UI, the back-end, the marketing etc. also depend critically on its ability to continue growing during certain critical phases, and the lack of discussion on this as a plausible failure mode is making me rather pessimistic.
In my mind, conditional on Site eventually operating as intended, it should grow through several phases. First you have a low number of users (~100 regular users? Sorry, I don’t have experience with this) who basically filtered in from your social circles, and are able to aggregate their opinions/thoughts as intended. Then in the next phase Site grows more popular as people notice this is a valuable source of truth/plans/speculation, and they provide new questions and answers covering broader topics. After that there should be some third phase where Site is diverse and big enough that all those extra features you mentioned might become plausible to implement (I’ll come back to this later).
My problem lies with the second phase. Benjamin’s piece suggests that as soon as Site is big enough to have any real value, this immediately creates incentives for outsiders to try to abuse/free-ride on the project (for example through manipulating the questions or voting). This would be worse on discussions on *actions*, which is why at the start I mentioned that that is more difficult than discussing *truth*. Your wish to keep Site crowd-sourced makes it more difficult to guard against this phenomenon, and to me Eliezer’s writing on security mindset suggests that if you don’t treat this problem as central the odds are strongly against you. It is unclear to me what motivates people to keep coming back to Site in this second phase if they disagree with a large part of the demographic/consensus, or in general why echo-chamber effects would not apply. In fact, it is unclear to me why people would spend time participating in discussions outside their immediate interests at all (see also for example evaporative cooling).
Lastly I think a large part of Site would only function after you have some critical mass of users to have sufficient discussion on a lot of different topics. This is troubling as it means those parts existing at all is conditional on Site being a success. In the spirit of “If you’re not growing you’re shrinking” I think a lot more time and effort should be focused on figuring out how to obtain and keep a userbase, and introducing fancy features is downstream from this.
Sorry for being so critical and nonconstructive. I don’t know how to solve any of these problems, but like I said at the start it felt like a wrong strategy to just stay quiet. I hope I’m wrong about most/all of this, and let me as a closure mention again that I don’t have experience with this at all.
Those are all concerns I share. I don’t have solutions either. I feel like my choice is to either build the website despite the lack of solutions and the high risk—or settle for not having anything that does what I want.
If I tried to do research on how to make websites grow, I would expect to encounter a lot of advice that’s based on survivorship bias, and therefore unreliable. (I mostly expect that luck is a/the dominant factor.) Do you think research on that would produce helpful results?
Moderation, on the other hand, is probably something that I could start with some research on, to see what might or might not be possible/helpful.
Well, I still don’t have any experience with this. But maybe possible avenues include:
Looking into moderation rules.
Including some kind of reputation/point/reward system, and other methods to keep your users engaged.
Tracking metrics on the growth of the Site, and ideally having some advance expectations/plans on how to respond to different rates of growth/decline.
A more radical approach might be to give up the phase 2 and beyond in their entirety, and settle for a target audience of people close enough to you that you can reasonably trust them.
The survivorship bias is a very valid point, but [not doing research on how to make websites grow] is also a poor strategy. Personally I’d still look into the advice, but I’m afraid what you’re trying to do is simply very difficult.
Epistemic status: worried about effort/time lost.
I am by no means experienced with any of this, and seriously considered not writing anything at all. But it only takes me a bit of time (an hour max) to write why I feel why the odds are very strongly against you, and if you are serious about pursuing this idea then even if at low probability that my comment is helpful to you it’s worth writing it on average. So here we go.
During my read of the post, top-to-bottom, at the part
my internal monologue went “The first bit is difficult but perhaps possible. The second is a mess. Oh dear, the third is basically impossible!”. The sentence immediately after, explaining this functionality would be the bare basics, shocked me quite a lot. I think aiming for the quoted section is nigh-impossible, and then we haven’t started on the possible additional features you mention. Your post strongly reminds me of Benjamin Hoffman’s piece on Anglerfish (in my opinion worth reading in full), and also a bit of a segment (near the start) in one of Eliezer’s posts on security mindset—where the character Amber makes the mistake of thinking that the critical part of her startup is the technology, where really it is the security. I think in a similar manner your Site would, besides depending on the UI, the back-end, the marketing etc. also depend critically on its ability to continue growing during certain critical phases, and the lack of discussion on this as a plausible failure mode is making me rather pessimistic.
In my mind, conditional on Site eventually operating as intended, it should grow through several phases. First you have a low number of users (~100 regular users? Sorry, I don’t have experience with this) who basically filtered in from your social circles, and are able to aggregate their opinions/thoughts as intended. Then in the next phase Site grows more popular as people notice this is a valuable source of truth/plans/speculation, and they provide new questions and answers covering broader topics. After that there should be some third phase where Site is diverse and big enough that all those extra features you mentioned might become plausible to implement (I’ll come back to this later).
My problem lies with the second phase. Benjamin’s piece suggests that as soon as Site is big enough to have any real value, this immediately creates incentives for outsiders to try to abuse/free-ride on the project (for example through manipulating the questions or voting). This would be worse on discussions on *actions*, which is why at the start I mentioned that that is more difficult than discussing *truth*. Your wish to keep Site crowd-sourced makes it more difficult to guard against this phenomenon, and to me Eliezer’s writing on security mindset suggests that if you don’t treat this problem as central the odds are strongly against you. It is unclear to me what motivates people to keep coming back to Site in this second phase if they disagree with a large part of the demographic/consensus, or in general why echo-chamber effects would not apply. In fact, it is unclear to me why people would spend time participating in discussions outside their immediate interests at all (see also for example evaporative cooling).
Lastly I think a large part of Site would only function after you have some critical mass of users to have sufficient discussion on a lot of different topics. This is troubling as it means those parts existing at all is conditional on Site being a success. In the spirit of “If you’re not growing you’re shrinking” I think a lot more time and effort should be focused on figuring out how to obtain and keep a userbase, and introducing fancy features is downstream from this.
Sorry for being so critical and nonconstructive. I don’t know how to solve any of these problems, but like I said at the start it felt like a wrong strategy to just stay quiet. I hope I’m wrong about most/all of this, and let me as a closure mention again that I don’t have experience with this at all.
Those are all concerns I share. I don’t have solutions either. I feel like my choice is to either build the website despite the lack of solutions and the high risk—or settle for not having anything that does what I want.
If I tried to do research on how to make websites grow, I would expect to encounter a lot of advice that’s based on survivorship bias, and therefore unreliable. (I mostly expect that luck is a/the dominant factor.) Do you think research on that would produce helpful results?
Moderation, on the other hand, is probably something that I could start with some research on, to see what might or might not be possible/helpful.
Well, I still don’t have any experience with this. But maybe possible avenues include:
Looking into moderation rules.
Including some kind of reputation/point/reward system, and other methods to keep your users engaged.
Tracking metrics on the growth of the Site, and ideally having some advance expectations/plans on how to respond to different rates of growth/decline.
A more radical approach might be to give up the phase 2 and beyond in their entirety, and settle for a target audience of people close enough to you that you can reasonably trust them.
The survivorship bias is a very valid point, but [not doing research on how to make websites grow] is also a poor strategy. Personally I’d still look into the advice, but I’m afraid what you’re trying to do is simply very difficult.