Many programs can be proven to have a certain result without any simulation, not even of a subset of the information. For example, think of a program that discovers the first 10,000 primes, increasing a counter by one for each prime it finds, and then stops. You can prove that the counter will equal 10,000 when it stops, without simulating this program.
See, to me that is a mental simulation of the relevant part of the program.
The counter will increase, point by point, it will remain an integer at each point and pass through every integer, and upon reaching 10,000 this will happen.
The fact that the relevant part of the program is as ridiculously simple as a counter just means that the simulation is easy.
See cousin_it’s post: http://lesswrong.com/lw/2ip/ai_cooperation_in_practice/
Many programs can be proven to have a certain result without any simulation, not even of a subset of the information. For example, think of a program that discovers the first 10,000 primes, increasing a counter by one for each prime it finds, and then stops. You can prove that the counter will equal 10,000 when it stops, without simulating this program.
See, to me that is a mental simulation of the relevant part of the program.
The counter will increase, point by point, it will remain an integer at each point and pass through every integer, and upon reaching 10,000 this will happen.
The fact that the relevant part of the program is as ridiculously simple as a counter just means that the simulation is easy.