Sometimes the returns just don’t diminish that fast.
I have a biology degree not mentioned on linkedin. I will say that I think for biology, the returns diminish faster. That is because bioscience knowledge from humans is mostly guesswork and low resolution information. Biology is very complex and the current laboratory science model I think fails to systematize gaining information in a useful way for most purposes. What this means is, you can get “results”, but not gain the information you would need to stop filling morgues with dead humans and animals, at least not without needing thousands of years at the current rate of progress.
I do not think an AGI can do a lot better for the reason that the data was never collected for most of it (the gene sequencing data is good, because it was collected via automation). I think that an AGI could control biology, for both good and bad, but it would need very large robotic facilities to systematize manipulating biology. Essentially it would have had to throw away almost all human knowledge, as there are hidden errors in it, and recreate all the information from scratch, keeping far more data from each experiment than is published in papers.
Using robots to perform the experiments and keeping data, especially for “negative” experiments, would give the information needed to actually get reliable results from manipulating biology, either for good or bad.
It means garage bioweapons aren’t possible. Yes, the last step of ordering synthetic DNA strands and preparing it could be done in a garage, but the information on human immunity at scale, or virion stability in air, or strategies to control mutations so that the lethal payload isn’t lost, requires information humans didn’t collect.
This poster calls this “Diminishing Marginal Returns”. Note that Diminishing marginal returns is empirical reality, it’s not merely an opinion, across most AI papers. (for humans, due to the inaccuracies in trying to assess IQ/talent, it’s difficult to falsify)
Sometimes the returns just don’t diminish that fast.
I have a biology degree not mentioned on linkedin. I will say that I think for biology, the returns diminish faster. That is because bioscience knowledge from humans is mostly guesswork and low resolution information. Biology is very complex and the current laboratory science model I think fails to systematize gaining information in a useful way for most purposes. What this means is, you can get “results”, but not gain the information you would need to stop filling morgues with dead humans and animals, at least not without needing thousands of years at the current rate of progress.
I do not think an AGI can do a lot better for the reason that the data was never collected for most of it (the gene sequencing data is good, because it was collected via automation). I think that an AGI could control biology, for both good and bad, but it would need very large robotic facilities to systematize manipulating biology. Essentially it would have had to throw away almost all human knowledge, as there are hidden errors in it, and recreate all the information from scratch, keeping far more data from each experiment than is published in papers.
Using robots to perform the experiments and keeping data, especially for “negative” experiments, would give the information needed to actually get reliable results from manipulating biology, either for good or bad.
It means garage bioweapons aren’t possible. Yes, the last step of ordering synthetic DNA strands and preparing it could be done in a garage, but the information on human immunity at scale, or virion stability in air, or strategies to control mutations so that the lethal payload isn’t lost, requires information humans didn’t collect.
Same issue with nanotechnology.
Update : https://www.lesswrong.com/posts/jdLmC46ZuXS54LKzL/why-i-m-sceptical-of-foom
This poster calls this “Diminishing Marginal Returns”. Note that Diminishing marginal returns is empirical reality, it’s not merely an opinion, across most AI papers. (for humans, due to the inaccuracies in trying to assess IQ/talent, it’s difficult to falsify)