That might be possible, but it would take a lot of effort to make resolvable. Who is “similar to Eliezer”, and how do we define that in advance? Which forecasting questions are we going to check their Brier score on? Which not-like-Eliezer forecasters are we going to compare them to (since scores are much more informative for ranking forecasters, than in isolation)? Etc. I’d rather people like Eliezer just placed their forecasts properly!
That might be possible, but it would take a lot of effort to make resolvable. Who is “similar to Eliezer”, and how do we define that in advance? Which forecasting questions are we going to check their Brier score on? Which not-like-Eliezer forecasters are we going to compare them to (since scores are much more informative for ranking forecasters, than in isolation)? Etc. I’d rather people like Eliezer just placed their forecasts properly!