As an aside (that’s still rather relevant, IMO), it is a huge pet peeve of mine when people use the word “randomly” in technical or semi-technical contexts (like this one) to mean “uniformly at random” instead of just “according to some probability distribution.” I think the former elevates and reifies a way-too-common confusion and draws attention away from the important upstream generator of disagreements, namely how exactly the constitution is sampled.
I wouldn’t normally have said this, but given your obvious interest in math, it’s worth pointing out that the answers to these questions you have raised naturally depend very heavily on what distribution we would be drawing from. If we are talking about, again, a uniform distribution from “the design space of minds-in-general” (so we are just summoning a “random” demon or shoggoth), then we might expect one answer. If, however, the search is inherently biased towards a particular submanifold of that space, because of the very nature of how these AIs are trained/fine-tuned/analyzed/etc., then you could expect a different answer.
Fair point. (I am not convinced by the argument that if the AI’s are trained on human texts and feedback, they are likely to end up with values similar to humans, but that would be a long debate.)
As an aside (that’s still rather relevant, IMO), it is a huge pet peeve of mine when people use the word “randomly” in technical or semi-technical contexts (like this one) to mean “uniformly at random” instead of just “according to some probability distribution.” I think the former elevates and reifies a way-too-common confusion and draws attention away from the important upstream generator of disagreements, namely how exactly the constitution is sampled.
I wouldn’t normally have said this, but given your obvious interest in math, it’s worth pointing out that the answers to these questions you have raised naturally depend very heavily on what distribution we would be drawing from. If we are talking about, again, a uniform distribution from “the design space of minds-in-general” (so we are just summoning a “random” demon or shoggoth), then we might expect one answer. If, however, the search is inherently biased towards a particular submanifold of that space, because of the very nature of how these AIs are trained/fine-tuned/analyzed/etc., then you could expect a different answer.
Fair point. (I am not convinced by the argument that if the AI’s are trained on human texts and feedback, they are likely to end up with values similar to humans, but that would be a long debate.)