Might this be one of those cases where the full Bayesian treatment can be used? Using an occamanian prior over functions over your-featuresspace generated by some ML model?
I like this approach because it follows the approach of Taking the Obvious Advice, and because of its focus on operationalising rationality rather than seeking insight porn.
As a short-term solution, would a Google Sheet work? I believe that you could then use a Google Form to populate the sheet. Here’s your example data in a spreadsheet.
I will return to this thread on December 11, 2017 to see if anyone else has subscribed to this project. I’m unable to commit any time prior to that date.
I second the notion of a spreadsheet, or a database. Aggregating and making available the information is much, much more valuable than a website. The information is the thing, the website is the symbolic representation of the thing.
I daresay what distinguishes this tribe is preventing symbols from having more value than the reality to which they point.
That being said, I am not saying don’t polish—it is just a popular candidate for bikeshedding and I think you can deliver a hard majority of the value sans polish because of the makeup of the community. I would expect similar projects that developed the data to a good state before starting on a platform for presenting the data to succeed more often than projects which do the reverse, or work on them simultaneously.
But what if a neat 80⁄20 of good presentation is a way to get people to add more data? I see it more likely that a pretty website will act as a schelling point compared to some obscure sheet. Especially if we plug it into lesserwrong.com.
This is a good point, and if we were talking about a tool aimed at the general public, I would agree with you.
It is also true that when we are talking about voluntary collaboration, it can easily be the case where someone pops up for whom a website is very easy.
But I urge you to consider that the second person to respond to your post went ahead and put the data in a spreadsheet anyway on their own initiative, and what this implies about the userbase you are after. I notice that whenever I read a post about a trick or method on here or elsewhere in the diaspora, people commonly respond with whether it works for them, what other method they used to receive the same goal, etc. I expect the fundamental challenge in getting user data to be that users want to see the results of the data but not offer it themselves, but in this environment you have users who habitually offer the relevant data anyway.
I therefore expect you could get a reasonable dataset together using this same format. In fact, I suggest a Sequence of Cookbook Development might do the trick. This is the kind of thing I want to see in Frontpage, which would have the added benefit of wide visibility in the community. You could do one for objectives and methods (what data is useful? how much should we ask for? about what?). Then you could do a Data I post where you ask for the data, in the previously determined format, on the previously determined topics, and for more things people are interested in. You could run as many additional Data posts as you wanted, as long as good info kept rolling in. Another post could present the scores, updated every so often with new data.
When that sequence ran its course, you’d probably have a reasonable chunk of data and a goodly chunk of feedback, for zero infrastructure or presentation investment (beyond what you are making anyway). The important thing is that getting this far would be cheap and add value. Then a website becomes a question of adding more value or not.
Of course it is completely acceptable to just really want to do a website, and if it will make you happy you should do it. I sort of assumed this would all be drudgery of one sort or other, which may be completely wrong!
Me and the folks over at FortForecast have actually been building something adjacent to this. It’s a preregistration database for LW Diaspora projects and experiments. Bendini’s original proposal for it looked a lot like your proposed cookbook, and since it actually has no application-specific parts in it we could totally make another form for personal outcomes on things like weight loss techniques.
If you’d be interested in that at all, email me at jd@fortforecast.com
We make a tool to let non-programmers build things like this kind of project. Think SquareSpace for web applications with real, custom logic. We’re accessible to anyone with enough computer skills to, for example, build formulas in excel.
I started Bubble precisely because I had a bunch of ideas for similar things—collaboration sites, smart todos lists, etc. -- and even as an experienced programmer, I found that the effort of building each one was deterring me from experimenting. I wanted something that would let me do something like this in 30 minutes instead of a week or two.
I’d call it a solid success: we use Bubble to build all our own internal tools, of course, and we recently built a lightweight JIRA knock-off for project management with v1 finished in an hour, and probably 3 − 4 more hours of ongoing refinements.
Beyond my own selfish interest in seeing the business take off, I’d love to see more awareness of Bubble in the rationalist community, because I think it’s a great fit: a lot of people experimenting ideas for improving discourse / thinking / knowledge-sharing, with a high-ratio of technically-inclined people, but not necessarily too many people who could develop applications like this full time. A lot of making these kinds of experiments work is having a low cost of trying them, so I think more adoption could lead to a bunch of cool things happening.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful melaton
I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful information for both groups (assuming the difference is statistically significant), and would be lovely to include in our cookbook.
This would require more information-gathering about individual users (and we should definitely have a “decline to disclose” option, particularly for more sensitive topics). If we want to be able to change what data we are collecting later (imagine that we suddenly have reason to believe that hair color is relevant to melatonin impact), we will need to store individual usernames in order to contact participants later.
I would be interested in helping with this project. My employer currently owns anything software-related I produce, but is willing to make reasonable exceptions where a project does not intersect with its business; if this project does materialize in a more concrete form, I would be able to present it to my employer and ask for permission to contribute. So if someone starts it, I would like to support it.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful information for both groups (assuming the difference is statistically significant), and would be lovely to include in our cookbook.
This would require more information-gathering about individual users (and we should definitely have a “decline to disclose” option, particularly for more sensitive topics). If we want to be able to change what data we are collecting later (imagine that we suddenly have reason to believe that hair color is relevant to melatonin impact), we will need to store individual usernames in order to contact participants later.
I apologize for the formatting; I tried to copy and paste from another app to get around the character-eating behavior of the comment box on mobile, and it seems to have resulted in this monstrosity which is immune to edits.
Perhaps I ought to just start posting comments as links to Google Docs.
True, and it’s a good 80⁄20 of what we want. Is it worth improving upon? Maybe:
Their categories only focus on solving problems and not on improving above baseline.
Creating our own software and integrating it with the LW 2.0 website might provide a schelling point, causing many more rationalists to actually work with the data.
They don’t do a lot of analysis. There might be a huge improvement if we condition on people’s features and do the actual ML/bayesian statistics to predict what will work *for them*.
Many rationalist try the weirdest stuff and I don’t expect to find that data on a normie website.
I expect adaptive problem solving in general to be improved if you could run it through the cookbook before generating your own solutions.
I think curetogether is a good start, but really would like a much bigger list of problems/benefits/interventions, plus expected costs of trying each one.
Sure, hit me up. What system are you using? At first I thought of this as a local project too, but more data is better. As long as we can ensure it’s quality.
a combination of turnkey systems eg. wiki, docs, spreadsheets during development, we will likely also be using this preregistration database when it is a bit more polished and we have experiements suited for it
It’s worth using Laplace’s Rule of Succession, so something that worked for 3⁄3 people would be listed as 80% instead of 100%.
Might this be one of those cases where the full Bayesian treatment can be used? Using an occamanian prior over functions over your-featuresspace generated by some ML model?
I like this approach because it follows the approach of Taking the Obvious Advice, and because of its focus on operationalising rationality rather than seeking insight porn.
As a short-term solution, would a Google Sheet work? I believe that you could then use a Google Form to populate the sheet. Here’s your example data in a spreadsheet.
I will return to this thread on December 11, 2017 to see if anyone else has subscribed to this project. I’m unable to commit any time prior to that date.
I second the notion of a spreadsheet, or a database. Aggregating and making available the information is much, much more valuable than a website. The information is the thing, the website is the symbolic representation of the thing.
I agree with your premises, but not your conclusion. Symbols are important. We want LW to be a tribe, not a company.
We can start with a spreadsheet, but let’s make it look nice and polished eventually.
I daresay what distinguishes this tribe is preventing symbols from having more value than the reality to which they point.
That being said, I am not saying don’t polish—it is just a popular candidate for bikeshedding and I think you can deliver a hard majority of the value sans polish because of the makeup of the community. I would expect similar projects that developed the data to a good state before starting on a platform for presenting the data to succeed more often than projects which do the reverse, or work on them simultaneously.
But what if a neat 80⁄20 of good presentation is a way to get people to add more data? I see it more likely that a pretty website will act as a schelling point compared to some obscure sheet. Especially if we plug it into lesserwrong.com.
This is a good point, and if we were talking about a tool aimed at the general public, I would agree with you.
It is also true that when we are talking about voluntary collaboration, it can easily be the case where someone pops up for whom a website is very easy.
But I urge you to consider that the second person to respond to your post went ahead and put the data in a spreadsheet anyway on their own initiative, and what this implies about the userbase you are after. I notice that whenever I read a post about a trick or method on here or elsewhere in the diaspora, people commonly respond with whether it works for them, what other method they used to receive the same goal, etc. I expect the fundamental challenge in getting user data to be that users want to see the results of the data but not offer it themselves, but in this environment you have users who habitually offer the relevant data anyway.
I therefore expect you could get a reasonable dataset together using this same format. In fact, I suggest a Sequence of Cookbook Development might do the trick. This is the kind of thing I want to see in Frontpage, which would have the added benefit of wide visibility in the community. You could do one for objectives and methods (what data is useful? how much should we ask for? about what?). Then you could do a Data I post where you ask for the data, in the previously determined format, on the previously determined topics, and for more things people are interested in. You could run as many additional Data posts as you wanted, as long as good info kept rolling in. Another post could present the scores, updated every so often with new data.
When that sequence ran its course, you’d probably have a reasonable chunk of data and a goodly chunk of feedback, for zero infrastructure or presentation investment (beyond what you are making anyway). The important thing is that getting this far would be cheap and add value. Then a website becomes a question of adding more value or not.
Of course it is completely acceptable to just really want to do a website, and if it will make you happy you should do it. I sort of assumed this would all be drudgery of one sort or other, which may be completely wrong!
Hi there,
Me and the folks over at FortForecast have actually been building something adjacent to this. It’s a preregistration database for LW Diaspora projects and experiments. Bendini’s original proposal for it looked a lot like your proposed cookbook, and since it actually has no application-specific parts in it we could totally make another form for personal outcomes on things like weight loss techniques.
If you’d be interested in that at all, email me at jd@fortforecast.com
Re: “Any coders that would like to jump into this?”
I hope it’s okay to pitch my own startup for this: https://bubble.is
We make a tool to let non-programmers build things like this kind of project. Think SquareSpace for web applications with real, custom logic. We’re accessible to anyone with enough computer skills to, for example, build formulas in excel.
I started Bubble precisely because I had a bunch of ideas for similar things—collaboration sites, smart todos lists, etc. -- and even as an experienced programmer, I found that the effort of building each one was deterring me from experimenting. I wanted something that would let me do something like this in 30 minutes instead of a week or two.
I’d call it a solid success: we use Bubble to build all our own internal tools, of course, and we recently built a lightweight JIRA knock-off for project management with v1 finished in an hour, and probably 3 − 4 more hours of ongoing refinements.
Beyond my own selfish interest in seeing the business take off, I’d love to see more awareness of Bubble in the rationalist community, because I think it’s a great fit: a lot of people experimenting ideas for improving discourse / thinking / knowledge-sharing, with a high-ratio of technically-inclined people, but not necessarily too many people who could develop applications like this full time. A lot of making these kinds of experiments work is having a low cost of trying them, so I think more adoption could lead to a bunch of cool things happening.
See curetogether.com—it seems like that would be what you want if they changed their categories a bit.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful melaton
I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.I think that an important addition would be other data about the participants in a given intervention, that could ideally help newcomers filter out interventions which are reasonably likely to have a positive effect in the general population but unlikely to apply to some subset of people.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful information for both groups (assuming the difference is statistically significant), and would be lovely to include in our cookbook.
This would require more information-gathering about individual users (and we should definitely have a “decline to disclose” option, particularly for more sensitive topics). If we want to be able to change what data we are collecting later (imagine that we suddenly have reason to believe that hair color is relevant to melatonin impact), we will need to store individual usernames in order to contact participants later.
I would be interested in helping with this project. My employer currently owns anything software-related I produce, but is willing to make reasonable exceptions where a project does not intersect with its business; if this project does materialize in a more concrete form, I would be able to present it to my employer and ask for permission to contribute. So if someone starts it, I would like to support it.
For example, let’s imagine that melatonin is effective for 60% of all people: 80% of people who describe themselves as “morning people”, but only 40% of people who do not. This is useful information for both groups (assuming the difference is statistically significant), and would be lovely to include in our cookbook.
This would require more information-gathering about individual users (and we should definitely have a “decline to disclose” option, particularly for more sensitive topics). If we want to be able to change what data we are collecting later (imagine that we suddenly have reason to believe that hair color is relevant to melatonin impact), we will need to store individual usernames in order to contact participants later.
I apologize for the formatting; I tried to copy and paste from another app to get around the character-eating behavior of the comment box on mobile, and it seems to have resulted in this monstrosity which is immune to edits.
Perhaps I ought to just start posting comments as links to Google Docs.
This is an interesting idea but I think something like it already exists: http://curetogether.com/conditions
True, and it’s a good 80⁄20 of what we want. Is it worth improving upon? Maybe:
Their categories only focus on solving problems and not on improving above baseline.
Creating our own software and integrating it with the LW 2.0 website might provide a schelling point, causing many more rationalists to actually work with the data.
They don’t do a lot of analysis. There might be a huge improvement if we condition on people’s features and do the actual ML/bayesian statistics to predict what will work *for them*.
Many rationalist try the weirdest stuff and I don’t expect to find that data on a normie website.
I expect adaptive problem solving in general to be improved if you could run it through the cookbook before generating your own solutions.
I think curetogether is a good start, but really would like a much bigger list of problems/benefits/interventions, plus expected costs of trying each one.
edit: whoops, thought you were AndHisHorse, although they are also welcome to contact me if interested in craft rationality
Sure, hit me up. What system are you using? At first I thought of this as a local project too, but more data is better. As long as we can ensure it’s quality.
a combination of turnkey systems eg. wiki, docs, spreadsheets during development, we will likely also be using this preregistration database when it is a bit more polished and we have experiements suited for it