That’s a fair bit of additional system complexity (though perhaps similar code-complexity, and fewer actual circuits). More importantly, it really just moves the problem out one level—now you worry about runaway or mutated controllers. You can make a tree of controllers-controlling-controllers, up to a small number of top-level controllers, with “only” logarithmic overhead, but it’s still not clear why a supervisor bot is less risk than a distributed set of bots.
If one level of nanobots mutates, it can pass the mutation on the nanobots “below” it but not other nanobots at the same level, so as long as nanobots “below” it don’t travel too far and wide, it won’t be able to exponentially grow until it ravages large parts of the world.
Of course a mutation at a very high level (maybe the top level) will still be a big problem. I kind of forgot to explain this part, but my idea is that these machines at a very high level will be fewer in number, and bigger, so that they might be easier to control or destroy.
Anyways, I do admit my idea might not be that necessary after reading Thomas Kwa’s post.
That’s a fair bit of additional system complexity (though perhaps similar code-complexity, and fewer actual circuits). More importantly, it really just moves the problem out one level—now you worry about runaway or mutated controllers. You can make a tree of controllers-controlling-controllers, up to a small number of top-level controllers, with “only” logarithmic overhead, but it’s still not clear why a supervisor bot is less risk than a distributed set of bots.
If one level of nanobots mutates, it can pass the mutation on the nanobots “below” it but not other nanobots at the same level, so as long as nanobots “below” it don’t travel too far and wide, it won’t be able to exponentially grow until it ravages large parts of the world.
Of course a mutation at a very high level (maybe the top level) will still be a big problem. I kind of forgot to explain this part, but my idea is that these machines at a very high level will be fewer in number, and bigger, so that they might be easier to control or destroy.
Anyways, I do admit my idea might not be that necessary after reading Thomas Kwa’s post.