samuelshadrach
If it take what you’re saying as literally true, your way of thinking is somewhat alien to me. I’m unsure how literally to take it.
There’s a difference between “this person will steal my money with probability less than 1%” and “this person is physically incapable of running the thought process that leads to them stealing my money”.
Maybe I live in a lower trust culture than yours and this influences me. I might have to see actual video or irl examples of what you talk about.
Or maybe I do see human brains as hackable machines to a greater extent than someone in your culture. I think more-or-less anyone can be motivated to turn on their neighbour with either a sufficiently large incentive (money, fear of social disapproval, safety), or a sufficiently persuasive ideology. And sure, some ideologies are extremely hard to persuade someone out of (let’s say they are religious) but hard is not the same as impossible.
can you trust a random person with your most embarrassing secrets is a different matter from whether you can trust them to not abscond with your wallet—both require trust, but of a different kind.
So there is one thing, which is having an institution that can send policemen after the person and retrieve the money for you. Some countries have better institutions for this than others.
If the other person can take your money and get away with it, is trusting them with money really all that different from trusting them with information? Both require studying a) are they inherently aligned with you in some way and b) can you threaten them with consequences, how will the gossip network around both of you react if you told them what had happened.
And that’s also part of the problem, as then you have to invest in defenses against other people’s envy or requests.
Yes agree!
Glad we agree :)
Real-time voice translation
That being said, Duncan is pointing at something crucial here—lots of people can’t even comprehend such a society, and they act as prions.
I think sharing raw data of how high trust societies work on youtube is a high leverage intervention. You are right that failure of imagination is huge, people tend to believe what they’ve seen work in practice.
This is more of a matter of a minimum being supplied, though.
How much is the minimum?
Here’s a possible crux: I think a society where everyone has 5 years financial runaway is likely to be higher trust than a society where everyone has 6 months financial runway. And yes, there are many people even in the US who have less than 6 months runway.
I have a lot of personal anecdotes as someone living in India, but some of them I can’t share for privacy reasons and others will take time to share even if I could.
I agree that if everyone had shared values we could build a higher trust society than if everyone had different values.
I’m experimenting with opening up on a bunch of stuff on the public internet. Example
You get a much better understanding of exactly which pieces of info are not safe to share, and why.
My reaction to this sort of thing is that reducing the cost of solar energy is a high leverage intervention here.
Making everyone wealthier is on one pathway to get to a high-trust society for 8 billion people. Do you have any other scalable solution?
I would rather prefer insults to the silence.
I feel same.
I think EA funders are more willing than many other non-profit funders to be the first person to fund your org, without anyone else supporting.
P.S. I didn’t downvote your comments.
and if you do declare it classified but don’t actually protect it, that means that the whistleblower is dealing with a different landscape.
I’m saying this has happened multiple times in the past already, and has high probability of happening again
We seem to be mostly on the same page here honestly.
And I think at least part of the reason he got away with it was that particular people (and courts) at the time felt constrained by the rule of law. This may not always apply.
There is a more recent list of examples of people who also got away with publicly leaking summaries but not leaking classified information. Your point is also true, but I think classified information being leaked is a bigger factor. It’ll be easier for me to argue all this once I publish an actual list of case studies.
Sorry, I need more time for that.
For technical information, that means not something entitled “How to HaxOr on the Dark Web lol”, but the manual for the OS. The manuals for servers and monitoring systems. The protocol specifications. Security product literature. Technical security standards. Architecture textbooks. There’s no royal road; you have to actually understand how things work.
Yes I agree this is required for the multiple books opsec guide, not the 10-page quick guide. Again, the question comes back to probabilities. These are very much not the actual numbers but if P(no guide, no prison) = 50%, P(10-page guide, no prison) = 65%, P(1000-page guide, no prison) = 80%, then it is worth publishing a 10-page guide.
I would tend to say that if you’re not already pretty sophisticated, maybe you shouldn’t do it. Not just because you don’t know your risks, not just because you may be unprepared either to stay anonymous or to deal with the consequences of not being… but also because you may not understand the actual impact, or lack thereof, that your disclosure will have when it hits the real world.
You can have wide uncertainty on every single one of these questions and still conclude that whistleblowing is the correct decision. See the example probabilities above.
I agree that my guide should include a section on predicting potential outcomes after you whistleblow. I will definitely do this. Thank you for the suggestion.
Basically every modern institution, even non-secretive ones, logs internal downloads (and external network connections, and more). You can end up doing that without even trying, just because of the software defaults. I believe the NSA does not usually have access to such internal logs. But, again, for this case, they’re on the same side as the people who do.
The people who matter most in terms of noticing your initial acquisition of whatever information will be your organization’s internal security and counterintelligence functions.
Where you get into trouble with the NSA is out on the public Internet. After you exfiltrate your data, the NSA is the agency that may be able to figure out where they went, or link your activities outside of the organization you’re whistleblowing on with your activities inside of it.
I agree this model is valuable to have, and I appreciate you writing this up.
I honestly do think though that by 2027, there is a good chance internal security for AI companies will directly report to the head of the NSA and ultimately the president, not to the head of labs.
And not just in a legal abstract sense like how Lockheed Martin’s various security teams may be accountable to the govt, but in the sense of how Manhattan project security directly reported to US military generals who frequently visited the sites for inspections.
I would not be surprised if NSA leaders built war rooms and private residences inside the datacenter compound, although this might be me speculating too far. If it happens though it may take longer than 2027, maybe 2029? It depends on timelines honestly.
Other than stuff like those questions, I don’t know a way. I think it’s partly about seeing these sorts of games played a lot of ways, and partly about the innate ability to think through all the possibilities with the right kind of paranoia. Paranoia that’s skeptical of itself.
I’ll try to think of a way though. It seems valuable to do.
I think the technological change is really the key point
Agree!
They were openly imposed by official policy from above and outside both agencies (even though they were probably welcomed on both sides).
Snowden in Permanent Record considers Dick Cheney, vice president and Michael Hayden, NSA director co-conspirators. I don’t have a lot of data to argue this but I do think the FBI director was intentionally kept out of the matter.
Your description is not wrong, there’s nuances here I haven’t tried to fully understand yet either. Thanks for the pointer that this may be worth doing.
This makes sense. Paying someone to do more of something they care about is very different from paying someone to do something they don’t otherwise want to do. That would include providing regular feedback.
I don’t think this is a good description of how big EA funders operate? This might be true for other orgs. Do you have any data to back this up?
Please post your hourly rate either here or even better, on your personal website. It’ll make it easier to contact people for this.
Related: Beta Readers by Holden Karnofsky
I’ve been informed that getting approval of 10-20 people on lesswrong for a project is a good way of getting funding from the bigger EA funders.
Can I just pay $50 each to 20 people for 0.5 hours of their time? Has this been tried?
(Ofcourse they can choose to disapprove the project, and accept the payment regardless.)
I think the biggest meta input I’ve gotten from your feedback, is that I need to publish a redteaming document for the theory of change of this work.
Also copied from another document. Sorry, I may need to publish all my work more clearly first, before soliciting expert feedback. Keen on your thoughts.
AI capability increases will outpace ability of US intelligence circles to adapt. Lots of information won’t become classified. - Weakly disagree
I have low (but not zero) probability we get ASI by 2027. If we get ASI by 2030, I think there’s enough time for them to adapt.
Classifying information is possible without significant changes in org structure or operational practices of AI labs. This means it can be done very quickly.
Classification is a legal tool.
The actual operational practices to defend information can take multiple years to implement, but this can come after information is already marked classified in terms of legality.
US govt can retroactively classify information after it has already been leaked.
This allows the US govt to pursue a legal case against the whistleblower under Espionage Act and prevent them from presenting evidence in court because it is now classified information.
The specific detail of whether information was classified at the time of leaking is less important than whether it poses a national security threat as deemed by US intelligence circles. (Law does not matter when it collides with incentives, basically.)
Case studies
Mark Klein’s 2006 leak of AT&T wiretapping—retroactively classified
Hillary Clinton 2016 email leak—retroactively classified
Abu Ghraib abuse photographs 2004 - retroactively classified
Sgt. Bowe Bergdahl 15-6 investigation file 2016 - retroactively classified
Scrape of many lesswrong blogs
(I did not put much effort in this, and am unlikely to fix errors. Please fork the list if you want something higher quality. Used only public data to make this.)
https://www.yudkowsky.net
https://gwern.net/
https://kajsotala.fi
https://www.astralcodexten.com
http://www.weidai.com/
https://lukemuehlhauser.com
https://www.mccaughan.org.uk/g/
https://vladimirslepnev.me
https://sethaherd.com
https://jimrandomh.tumblr.com
https://muckrack.com/thane-ruthenis/articles
https://ailabwatch.substack.com/
https://www.lsusr.com
https://theeffortlessway.com
https://www.cognitiverevolution.ai/mind-hacked-by-ai-a-cautionary-tale-from-a-lesswrong-users-confession/
http://zackmdavis.net/blog/
https://benjaminrosshoffman.com
https://acesounderglass.com/
https://www.benkuhn.net/
https://benlandautaylor.com/
https://thezvi.wordpress.com/
https://eukaryotewritesblog.wordpress.com/
https://flightfromperfection.com/
https://meteuphoric.wordpress.com/
https://srconstantin.wordpress.com/
https://sideways-view.com/
https://rationalconspiracy.com/
http://unremediatedgender.space/
https://unstableontology.com/
https://sjbyrnes.com/agi.html
https://sites.google.com/view/afdago/home
https://paulfchristiano.com
https://paisri.org/
https://katjagrace.com
https://blog.ai-futures.org/p/our-first-project-ai-2027
https://www.mariushobbhahn.com/aboutme/
http://rootsofprogress.org/
https://www.bhauth.com/
https://turntrout.com/research
https://www.jefftk.com
https://virissimo.info/documents/resume.html
https://bmk.sh
https://metr.org/
https://kennaway.org.uk
http://1a3orn.com/
https://acritch.com
https://lironshapira.substack.com
https://coral-research.org
https://substack.com/@theojaffee
https://matthewbarnett.substack.com/
https://www.beren.io/
https://www.cold-takes.com
https://newsletter.safe.ai
https://www.metaculus.com/accounts/profile/116023/
https://www.patreon.com/profile/creators?u=132372822
https://medium.com/inside-the-simulation
https://www.lesswrong.com/users/unexpectedvalues?from=search_page
http://markxu.com/about
https://www.overcomingbias.com
https://www.vox.com/authors/miranda-dixon-luinenburg
https://www.getkratom.com
https://github.com/YairHalberstadt
https://www.scott.garrabrant.com
https://arundelo.com
http://unstableontology.com/
https://mealsquares.com/pages/our-team
https://evhub.github.io
https://formethods.substack.com/
https://www.nosetgauge.com
https://substack.com/@euginenier
https://drethelin.com
https://entersingularity.wordpress.com
https://doofmedia.com
https://mindingourway.com/about/
https://jacquesthibodeau.com
https://www.neelnanda.io/about
https://niplav.site/index.html
https://jsteinhardt.stat.berkeley.edu
https://www.jessehoogland.com
http://therisingsea.org
https://www.stafforini.com/
https://acsresearch.org
https://elityre.com
https://www.barnes.page/
https://peterbarnett.org/
https://joshuafox.com/
https://itskatydee.com/
https://ethanperez.net/
https://owainevans.github.io/
https://chrislakin.blog/
https://colewyeth.com/
https://www.admonymous.co/ryankidd44
https://ninapanickssery.substack.com/
https://joecarlsmith.com
http://coinlist.co/
https://davidmanheim.com
https://github.com/SarahNibs
https://malmesbury.substack.com
https://www.admonymous.co/rafaelharth
https://dynomight.net/
http://nepenthegame.com/
https://github.com/RDearnaley
https://graehl.org
https://nikolajurkovic.com
https://www.julianmorrison.com
https://avturchin.livejournal.com
https://www.perfectlynormal.co.uk
https://www.250bpm.com
https://www.youtube.com/@TsviBT
https://adamjermyn.com
https://www.elilifland.com/
https://zhd.dev
https://ollij.fi
https://arthurconmy.github.io/about/
https://www.youtube.com/@RationalAnimations/featured
https://cims.nyu.edu/~sbowman/
https://crsegerie.github.io/
https://escapingflatland.substack.com/
https://qchu.wordpress.com
https://dtch1997.github.io/
https://math.berkeley.edu/~vaintrob/
https://mutualunderstanding.substack.com
https://longerramblings.substack.com
https://peterwildeford.substack.com
https://juliawise.net
https://uli.rocks/about/
https://stephencasper.com/
https://engineeringideas.substack.com/
https://homosabiens.substack.com
https://martin-soto.com/
https://www.tracingwoodgrains.com
https://www.brendanlong.com
https://foresight.org/fellowship/2024-fellow-bogdan-ionut-cirstea/
https://davekasten.substack.com
https://datapacrat.com
http://admonymous.co/nat_m
https://mesaoptimizer.com/
https://ae.studio/team
https://davidad.org
https://heimersheim.eu
https://nunosempere.com/
https://www.thinkingmuchbetter.com/nickai/
http://kilobug.free.fr/code/
http://vkrakovna.wordpress.com/
https://www.conjecture.dev/
https://ejenner.com
https://morphenius.substack.com/
https://gradual-disempowerment.ai
https://www.clubhouse.com/@patrissimo
https://mattmacdermott.com
https://knightcolumbia.org
https://www.openphilanthropy.org/about/team/lukas-finnveden/