IFF I’m going to die with P>80% in the next 10 years while >80% (modulo natural death rate) of the rest of humanity survives for at least 5 more years then, was what killed me in the reference class:
disease
mechanical/gross-physical accident
murdered
other
Repeat to drill down and know the most important hedges for personal survival.
The “rest of humanity survives” condition reduces the chance the question becomes entangled with the eschaton.
i.e. I’m pointing out that selfish utility functions are less personally or humanity-existentially dangerous to ask the oracle questions relevant to in cases where concerns are forced to be local (in this case, forced-local because you died before the eschaton). However the answers still might be dangerous to people near you.
i.e. Selfish deals with the devil might not destroy the world if they’re banal in the grand scheme of things.
See the edit, and make sure you “decide on the length of each episode, and how the outcome is calculated. The Oracle is run once an episode only (and other Oracles can’t generally be used on the same problem; if you want to run multiple Oracles, you have to justify why this would work), and has to get objective/loss/reward by the end of that episode, which therefore has to be estimated in some way at that point.”
Submission: low bandwidth oracle, ask:
IFF I’m going to die with P>80% in the next 10 years while >80% (modulo natural death rate) of the rest of humanity survives for at least 5 more years then, was what killed me in the reference class:
disease
mechanical/gross-physical accident
murdered
other
Repeat to drill down and know the most important hedges for personal survival.
The “rest of humanity survives” condition reduces the chance the question becomes entangled with the eschaton.
i.e. I’m pointing out that selfish utility functions are less personally or humanity-existentially dangerous to ask the oracle questions relevant to in cases where concerns are forced to be local (in this case, forced-local because you died before the eschaton). However the answers still might be dangerous to people near you.
i.e. Selfish deals with the devil might not destroy the world if they’re banal in the grand scheme of things.
See the edit, and make sure you “decide on the length of each episode, and how the outcome is calculated. The Oracle is run once an episode only (and other Oracles can’t generally be used on the same problem; if you want to run multiple Oracles, you have to justify why this would work), and has to get objective/loss/reward by the end of that episode, which therefore has to be estimated in some way at that point.”