How much does it cost to run one instance of this right now?
One obvious idea is to build integrations for Manifold, Fatebook, Metaculus etc where you automatically have this bot give an answer. I’m curious how much that costs when you’re just doing it all over the place.
(I don’t know that I believe the current generation of bots are as good as implied here for reasons Habryka/Gwern are saying, but, regardless, I expect at least pretty decent bots of this form to be available within a year if they aren’t already, and for it to be useful for various sites to have SOTA forecasting bots plugged into them if they’re at least nontrivially better than chance)
A thing that is different from a forecasting bot, is an “operationalization” bot, that scans threads with disagreements for plausible operationalizations of the disagreement.
I think good discourse is partly bottlenecked on “it’s pretty expensive and annoying to operationalize many disagreements, in ways that actually help get to the heart of the matter.”
I’d be pretty interested in experimenting with operationalization-bot and forecasting-bot combos (once they get sufficiently good to not be annoying), that scan disagreements, suggest little cloud-bubbles of “is this a cruxy operationalization you could bet on?”, which maybe disappear if nobody clicks “yep”.
Then, people are encouraged to put their own probabilities on it, and then, afterwards (to avoid anchoring), a forecasting bot gives it’s own guess.
How much does it cost to run one instance of this right now?
One obvious idea is to build integrations for Manifold, Fatebook, Metaculus etc where you automatically have this bot give an answer. I’m curious how much that costs when you’re just doing it all over the place.
(I don’t know that I believe the current generation of bots are as good as implied here for reasons Habryka/Gwern are saying, but, regardless, I expect at least pretty decent bots of this form to be available within a year if they aren’t already, and for it to be useful for various sites to have SOTA forecasting bots plugged into them if they’re at least nontrivially better than chance)
Some additional thoughts:
A thing that is different from a forecasting bot, is an “operationalization” bot, that scans threads with disagreements for plausible operationalizations of the disagreement.
I think good discourse is partly bottlenecked on “it’s pretty expensive and annoying to operationalize many disagreements, in ways that actually help get to the heart of the matter.”
I’d be pretty interested in experimenting with operationalization-bot and forecasting-bot combos (once they get sufficiently good to not be annoying), that scan disagreements, suggest little cloud-bubbles of “is this a cruxy operationalization you could bet on?”, which maybe disappear if nobody clicks “yep”.
Then, people are encouraged to put their own probabilities on it, and then, afterwards (to avoid anchoring), a forecasting bot gives it’s own guess.