Optimisers naturally tend to develop instrumental goals to acquire resources—because that helps them to optimise. If you are not talking about an optimiser, you are not talking about an intelligent agent—in which case it is not very clear exactly what you want it for—whereas if you are, then you must face up to the possible resource-grab problem.
Tim:
Doesn’t apply here.
Optimisers naturally tend to develop instrumental goals to acquire resources—because that helps them to optimise. If you are not talking about an optimiser, you are not talking about an intelligent agent—in which case it is not very clear exactly what you want it for—whereas if you are, then you must face up to the possible resource-grab problem.
Do you think Watson and google’s search engine are liable to start grabbing resources ? Do you think they are unintelligent?