My point is that there is motivation to solve the problem of value drift and manipulation both for people and global power, on the defense side. The thing to preserve/anchor could be anything that shouldn’t be lost, not necessarily conventional referents of concern. The conservative choice is trying to preserve everything except ignorance. But even vague preference about experience of others is a direction of influence that could persevere, indifferent to its own form.
If the problem is solved, there is less motivation to keep behavior or cognition predictable to make externalities safe for unprotected others, as there are no unprotected others to worry about.
I absolutely understand that there’s motivation to “solve the problem of value drift and manipulation”. I suspect that the problem is literally unsolvable, and I should be more agnostic about distant values than I seem to be. I’m trying on the idea of just hoping that there are intelligent/experiencing/acting agents for a long long time, regardless of what form or preferences those agents have.
My point is that there is motivation to solve the problem of value drift and manipulation both for people and global power, on the defense side. The thing to preserve/anchor could be anything that shouldn’t be lost, not necessarily conventional referents of concern. The conservative choice is trying to preserve everything except ignorance. But even vague preference about experience of others is a direction of influence that could persevere, indifferent to its own form.
If the problem is solved, there is less motivation to keep behavior or cognition predictable to make externalities safe for unprotected others, as there are no unprotected others to worry about.
I absolutely understand that there’s motivation to “solve the problem of value drift and manipulation”. I suspect that the problem is literally unsolvable, and I should be more agnostic about distant values than I seem to be. I’m trying on the idea of just hoping that there are intelligent/experiencing/acting agents for a long long time, regardless of what form or preferences those agents have.