I think this post would be stronger if it covered at least basic metrology and statistics.
It’s incorrect to say that billions of variables aren’t affecting a sled sliding down a hill—of course they’re affecting the speed, even if most are only by a few planck-lengths per hour. But, crucially, they’re mostly not affecting it to a detectable amount. The detectability threshold is the key to the argument.
For detectability, whether you notice the effects of outside variables is going to come down to the precision of the instrument that you’re using to measure your output. If you’re using a radar gun that gives readings to the nearest MPH, for example, you won’t perceive a difference between 10.1 and 10.2 MPH, and so to you the two are equivalent. Nonetheless, outside variables have absolutely influenced the two readings differently.
Equally critical is the number of measurements that you’re taking. For example, if you’re taking repeated measurements after controlling a certain set of variables, you may be able to say with a certain confidence/reliability that no other variables are causing enough variations in speed to register an output that’s outside of the parameters that you’ve set. But that is a very different thing than saying that those other variables simply don’t exist! One is a statement of probability, another is a statement of certainty. Maybe there’s a confluence of variables that only occur once every thousand times, which you won’t pick up when doing an initial evaluation.
I’d like to propose a test to objectively quantify the average observer’s reaction with regards to skepticism of doomsday prophesizing present in a given text. My suggestion is this: take a text, swap the subject of doom (in this case AGI) with another similar text spelling out humanity’s impending doom—for example, a lecture on Scientology and Thetans or the Jonestown massacre—and present these two texts to independent observers, in the same vein as a Turing test.
If an outside independent observer cannot reliably identify which subject of doom corresponds to which text, then that could serve as an effective way of benchmarking when a specific text has transitioned away from effectively conveying information and towards fearmongering.