Ok, I want to say thank you for this comment because it contains a lot of points I strongly agree with. I think the alignment community needs experimental data now more than it needs more theory.
However, I don’t think this lowers my opinion of MIRI. MIRI, and Eliezer before MIRI even existed yet, was predicting this problem accurately and convincingly enough that people like myself updated. 15 years ago I began studying neuroscience, neuromorphic computing, and machine learning because I believed this was going to become a much bigger deal than it was then.
Now the general gist of the message has absolutely been proven out. Machine learning is now a big impressive thing in the world, and scary outcomes are right around the corner. Forecasting that now doesn’t win you nearly as many points as forecasting that 15 or 20 years ago.
Now we are finally close enough that it makes sense to move from theorizing to experimentation. That doesn’t mean the theorizing was useless. It laid an incredible amount of valuable groundwork. It gave the experimental researchers a server of what they are up against. Laid out the scope of the problem, and made helpful pointers towards important characteristics that good solutions must have.
A call to action for an army of experimentalists to put the theory to test us not evidence against MIRI. Their theory is still useful and helpful, it’s just no longer where we need the most rapid expansion of effort. We are entering a new phase now, gathering experimental data, and we still need MIRI, and others like them, to update and refine the theory in response to the experimental data. Theory used to be 95% of the work going into AGI alignment. Now it needs to become more like 5%. Not by decreasing the work going into theory. We need to increase that! But rather, because now it is time to throw open the floodgates of evidence and unleash the army of experimenters. Engineering work is easier than pure theory to get right, so thankfully, lots more people are qualified to contribute. This is good, because we need a lot. There is so much work to be done.
Hmm, I agree that Eliezer, MIRI and its precursors did a lot of good work raising the profile of this particular x-risk. However, I am less certain of their theoretical contributions, which you describe as
That doesn’t mean the theorizing was useless. It laid an incredible amount of valuable groundwork. It gave the experimental researchers a server of what they are up against. Laid out the scope of the problem, and made helpful pointers towards important characteristics that good solutions must have.
I guess they did highlight a lot of dead ends, gotta agree with that. I am not sure how much the larger AI/ML community values their theoretical work. Maybe the practitioners haven’t caught up yet.
Theory used to be 95% of the work going into AGI alignment. Now it needs to become more like 5%
Well, whatever the fraction, it certainly seems like it’s time to rebalance it, I agree. I don’t know if MIRI has the know-how to do experimental work at the level of the rapidly advancing field.
Ok, I want to say thank you for this comment because it contains a lot of points I strongly agree with. I think the alignment community needs experimental data now more than it needs more theory. However, I don’t think this lowers my opinion of MIRI. MIRI, and Eliezer before MIRI even existed yet, was predicting this problem accurately and convincingly enough that people like myself updated. 15 years ago I began studying neuroscience, neuromorphic computing, and machine learning because I believed this was going to become a much bigger deal than it was then. Now the general gist of the message has absolutely been proven out. Machine learning is now a big impressive thing in the world, and scary outcomes are right around the corner. Forecasting that now doesn’t win you nearly as many points as forecasting that 15 or 20 years ago. Now we are finally close enough that it makes sense to move from theorizing to experimentation. That doesn’t mean the theorizing was useless. It laid an incredible amount of valuable groundwork. It gave the experimental researchers a server of what they are up against. Laid out the scope of the problem, and made helpful pointers towards important characteristics that good solutions must have. A call to action for an army of experimentalists to put the theory to test us not evidence against MIRI. Their theory is still useful and helpful, it’s just no longer where we need the most rapid expansion of effort. We are entering a new phase now, gathering experimental data, and we still need MIRI, and others like them, to update and refine the theory in response to the experimental data. Theory used to be 95% of the work going into AGI alignment. Now it needs to become more like 5%. Not by decreasing the work going into theory. We need to increase that! But rather, because now it is time to throw open the floodgates of evidence and unleash the army of experimenters. Engineering work is easier than pure theory to get right, so thankfully, lots more people are qualified to contribute. This is good, because we need a lot. There is so much work to be done.
Hmm, I agree that Eliezer, MIRI and its precursors did a lot of good work raising the profile of this particular x-risk. However, I am less certain of their theoretical contributions, which you describe as
I guess they did highlight a lot of dead ends, gotta agree with that. I am not sure how much the larger AI/ML community values their theoretical work. Maybe the practitioners haven’t caught up yet.
Well, whatever the fraction, it certainly seems like it’s time to rebalance it, I agree. I don’t know if MIRI has the know-how to do experimental work at the level of the rapidly advancing field.