This seems directly important and actionable in a way that most Less Wrong posts, while well considered and informative, are not. I have not acquitted myself well in the rationalist community so I would like to offer engagement with this article and its concepts as a surrogate for any engagement with me.
Alephwyr
Karma: −1
Considering coming if I can wake up in time and if nobody has a problem with me shamelessly exploiting you for free labor (project is setting up an OpenBSD darkweb Mastodon server on a ThinkPad T440P.) Due to limited storage space and limited uptime ability the “concept” I am aiming at is fedspace protocol as a method of temporarily deploying a way for people to exchange CWTCH handles for subsequent communication in an adversarial environment. Might like to record the setup process on cell phone camera so others have a tutorial of sorts or just to keep track of ground covered. Is any of that sensible/acceptable?
I am skeptical of any epistemics that conflate memetic survival with truth, even weakly. Mostly because acting on certain beliefs can destroy evidence for other beliefs. Partly because I can think of no reason that all truths should intersect with anthropocentrism. An example of the former might be the destruction of native american agricultural and hunting techniques by the destruction of the environment. An example of the latter might be, more contentiously, natalism vs antinatalism. If antinatalism is true it still loses memetically simply because it self-selects itself away. So I don’t think your heuristics are robust enough, and probably we need people to probe the boundaries of reason from time to time. It would be good to reduce the social costs of this, but that’s different from epistemically consolidating around a small bundle of high equity beliefs and never risking anything on outlandish but potentially important beliefs. That sort of epistemic conservativism seems both exploitable to me, in the long run, and also a kind of death spiral.
Aside from the layer one security considerations, if you can define a minimum set of requirements for safe AI and a clear chain of escalation with defined responses, you can eventually program this into AI itself, pending a solution to alignment. At a certain level of AI development, AI safety becomes self enforcing. At that point the disincentives should be towards non-networked compute capacity, at least beyond the threshold needed for strong AI. At the point at which AI safety becomes self enforcing, the security requirement for state ownership only should become relaxable, albeit within definable limits, and pending control of manufacturing according to security compliant demands. Since manufacturing is physical and capital intensive this is probably fairly easy to achieve, at least when compared to AI alignment itself.