Ironically, I do not know who to attribute to the notion that ‘all problems are credit assignation problems.’
romeostevensit
Karma: 9,472
I’ve read leaked emails from people in similar situations before that made a couple things apparent:
Power talk happens on the phone for paper trail reasons
There is no meeting where an actual rational discussion of considerations and theories of change happens, everything really is people flying by the seat of their pants even at highest level. Talk of ethics usually just gets you excluded from the power talk.
I concluded this from the lack of any such talk in meeting minutes that are recorded, and the lack of any reference to such considerations in ‘previous conversations’ or requests to set up such meetings.
This elides the original argument by assuming the conclusion: that countermanding efforts remain cheap relative to the innovations. But the whole point is that significant shifts in costs associated with defense of a certain level can change behaviors and which plans and supply chains are economically defensible a lot.
Relatedly: people often discount improvements with large startup costs even if those costs are one time cost for an ongoing benefit. One of the worst is when it’s something one is definitely going to do eventually, so delaying paying the startup cost is simply reducing the amount of time for diffuse benefits. Exercise and learning to cook are like this.
One operationalization is splitting out positive and negative predictions/models in all three questions (or cost benefit etc).
when you’re stuck at the bottom of an attractor a hard kick to somewhere else can be good enough even with unknown side effects.
I have attempted to communicate to ultra-high-net-worth individuals, seemingly to little success so far, that given the reality of limited personal bandwidth, with over 99% of their influence and decision-making typically mediated through others, it’s essential to refine the ability to identify trustworthy advisors in each domain. Expert judgment is an active field of research with valuable, actionable insights.
It’s worth noting that many therapists break therapeutic alliance for ideological or liability reasons and this is one of the reasons that self therapy, peer therapy, llms, and workbooks can sometimes be better.
Agree with the approach with the caveat that some people in group 2 are naive cooperators and therefore second order defectors since they are suckers for group 1. Eg the person who will tell the truth to the Nazis out of mistaken theories of ethics or just behavioral conditioning.
I was reading this earlier and it dovetails very well with this post. Framing defending yourself against hostile people and processes as primarily selfish itself serves the hostile.
‘In essence, it is viewed as a form of adhamma (not-Dhamma) or misconduct to teach someone who is uninterested or unreceptive, since doing so does not respect the individual’s disposition and may lead to misunderstanding or conflict rather than enlightenment.’ (commentary on Akkosa Sutta (SN 7.2))
Yes, though this often involves some self deception about your true utility function. I suspect that some ace people did this to themselves to avoid zero sum competition they expect to painfully lose.
I can secondhand lend some affirmation to the newcomb case. A friend with DID from a childhood with a BPD mom later became a meditator and eventually rendered transparent the shell game that was being played with potentially dangerous preferences and goals to keep them out of consciousness, since the mom was extremely good at telepathy and was hostile for the standard BPD reason: other beings with other goals are inherently threatening to their extremely fragile sense of their own preferences and goals.
Another solution is illegible-ization/orthogonalization of preferences to the hostile telepath so that you don’t overlap in anything they might care about or overpower you with. I think this is one of the things to think about in terms of rationalist avoidance of conflict theory.
I think this is an important topic and am glad to see substantial scholarship efforts on it.
Wrt AI relevance: I think the meme that it matters a lot who builds the self activating doomsday device has done potentially quite a bit of harm and may be a main contributor to what kills us.
Wrt people detecting these traits: I personally feel that the self domestication of humans has made us easier targets for such people, and undermined our ability to even think of doing anything about them. I don’t think this is entirely random.
I propose a new term, gas bubble, to describe the spate of scams we’re about to see. It’s a combination of gas lighting and filter bubble.
like the calibration game but for a variety of decision problems, where the person has to assign probabilities to things at different stages based on what information is available. Afterwards they get an example brier score based on the average of what people with good prediction track records set at each phase.
I’ve thought about this for a long time and I think one of the big issues is lack of labelled training data in many domains. E.g. people made calibration toys and that helped a lot for that particular dimension. Ditto the tests on which studies replicated. In many cases we’d want more complex blinded data for people to practice on, and that requires, like in games, someone to set up all the non-fun backend for them.
I tried them for a while and was unimpressed. Plus some need to be loaded quite heavy, risking injury.
Reasonable
Thank you!