Thought 2: From my experience, AI alignment is a domain of research that intrinsically comes with mental health hazards. First, the possibility of impending doom and the heavy sense of responsibility are sources of stress. Second, research inquiries often enough lead to “weird” metaphysical questions that risk overturning the (justified or unjustified) assumptions we implicitly hold to maintain a sense of safety in life. I think it might be the closest thing in real life to the Lovecraftian notion of “things that are best not to know because they will drive you mad”. Third, the sort of people drawn to the area and/or having the necessary talents seem to often also come with mental health issues (I am including myself in this group).
That sounds like MIRI should have a councillor on it’s staff.
That would make them more vulnerable to claims that they use organizational mind control on their employees, and at the same time make it more likely that they would actually use it.
You would likely hire someone who’s traditionally trained, credentialed and has work experience instead of doing a bunch of your own psych-experiments, likely in a tradition like gestalttherapy that focuses on being nonmanipulative.
There’s an easier solution that doesn’t run the risk of being or appearing manipulative. You can contract external and independent councillors and make them available to your staff anonymously. I don’t know if there’s anything comparable in the US, but in Australia they’re referred to as Employee Assistance Programs (EAPs). Nothing you discuss with the councillor can be disclosed to your workplace, although in rare circumstances there may be mandatory reporting to the police (e.g. if abuse or ongoing risk of a minor is involved).
This also goes a long way toward creating a place where employees can talk about things they’re worried will seem crazy in work contexts.
Solutions like that might work, but it’s worth noting that just having an average therapist likely won’t be enough.
If you actually care about a level of security that protects secrets against intelligence agencies, operational security of the office of the therapist is a concern.
Governments that have security clearances don’t want their employees to talk with therapists who don’t have the secuirty clearances about classified information.
Talking nonjudgmentally with someone who has reasonable fears that the humanity won’t survive the next ten years because of fast AI timelines is not easy.
That sounds like MIRI should have a councillor on it’s staff.
That would make them more vulnerable to claims that they use organizational mind control on their employees, and at the same time make it more likely that they would actually use it.
You would likely hire someone who’s traditionally trained, credentialed and has work experience instead of doing a bunch of your own psych-experiments, likely in a tradition like gestalttherapy that focuses on being nonmanipulative.
There’s an easier solution that doesn’t run the risk of being or appearing manipulative. You can contract external and independent councillors and make them available to your staff anonymously. I don’t know if there’s anything comparable in the US, but in Australia they’re referred to as Employee Assistance Programs (EAPs). Nothing you discuss with the councillor can be disclosed to your workplace, although in rare circumstances there may be mandatory reporting to the police (e.g. if abuse or ongoing risk of a minor is involved).
This also goes a long way toward creating a place where employees can talk about things they’re worried will seem crazy in work contexts.
Solutions like that might work, but it’s worth noting that just having an average therapist likely won’t be enough.
If you actually care about a level of security that protects secrets against intelligence agencies, operational security of the office of the therapist is a concern.
Governments that have security clearances don’t want their employees to talk with therapists who don’t have the secuirty clearances about classified information.
Talking nonjudgmentally with someone who has reasonable fears that the humanity won’t survive the next ten years because of fast AI timelines is not easy.