Interested in big picture considerations and thoughtful action.
ektimo
Karma: 179
Care to explain? Is the Servant God an ASI and the true makers the humans that built it? Why did the makers hide their deeds?
Thanks for the riff!
Note, I wasn’t sure how to convey it but in the version I wrote, I didn’t mean it as a world where people have god-like powers. The only change intended was that it was a world where it was normal for six-year-olds to be able to think about multiple universes and understand what counts as advanced math for us, like Group Theory. There were a couple things I was thinking about:
I was musing on a possible solution to the measure problem that our universe is an actual hypothetical/mathematical object and there a finite number of actual hypotheticals such that having a copy of a universe would make no more sense than having a copy of a number. (The mathematical object only needs to be as real as we are within it.)
I was also asking if it would be possible to have a world where it was normal for six-year-olds to be that much better at math (and presumably get better as they grow up) in the same way that a six-year-old is that much better at conceptual math than a chimpanzee. Would it have to be creepy or could they still be relatable? (The girl was smiling because she knew she was being silly.)
Disclaimer: I’m not a Group Theorist and the LLM I asked said it would take ten plus years if ever for me to be able to derive the order of the Fischer–Griess monster group from first principles (but it’s normal that the child could do this).
Prompt: write a micro play that is both disturbing and comforting
--Title: “The Silly Child”
Scene: A mother is putting to bed her six-year-old child
CHILD: Mommy, how many universes are there?
MOTHER: As many as are possible.
CHILD (smiling): Can we make another one?
MOTHER (smiling): Sure. And while we’re at it, let’s delete the number 374? I’ve never liked that one.
CHILD (excited): Oh! And let’s make a new Fischer-Griess group element too! Can we do that Mommy?
MOTHER (bops nose) That’s enough stalling. You need to get your sleep. Sweet dreams, little one. (kisses forehead)
End
Thank you for your clear response. How about another example? If somebody offers to flip a fair coin and give me $11 if Heads and $10 if Tails then I will happily take this bet. If they say we’re going to repeat the same bet 1000 times then I will take this bet also and I expect to gain and unlikely to lose a lot. If instead they show me five unfair coins and say they are weighted from 20% Heads to 70% Heads then I’ll be taking on more risk. The other three could be all 21% Heads or all 69% Heads but if I had to pick then I’ll pick Tails because if I know nothing about the other three and I know nothing about if the other person wants me to make or lose money then I’d figure the other three are randomly biased within that range (even though I could be playing a loser’s game for 1000 rounds with flips of those coins if each time one of the coins is selected randomly to flip, but it’s still better than picking Heads). Is this the situation we’re discussing?
Maximality seems asymmetrical and losing information?
Maybe it will help me to have an example though I’m not sure if this is a good one… if I have two weather forecasts that provide different probabilities for 0 inches, 1 inch, etc but I have absolutely no idea about which forecast is better, and I don’t want to go out if there is greater than 20% probability of more than 2 inches of rain then I’d weigh each forecast equally and calculate the probability from there. If the forecasts themselves provide a high/low probabilities for 0 inches, 1 inch, etc then I’d think this isn’t a very good forecast since the forecaster should either have combined all their analysis into a single probability (say 30%) or else given the conditions under which they give their low end (say 10%) or high end (say 40%) and then if I didn’t have any opinions on the probability of those conditions then I would weigh the low and high equally (and get 25%). Do you think I should be doing something different (or what is a better example)?
This seems like 2 questions:
Can you make up mathematical counterfactuals and propagate the counterfactual to unrelated propositions? (I’d guess no. If you are just breaking a conclusion somewhere you can’t propagate it following any rules unless you specify what those rules are, in which case you just made up a different mathematical system.)
Does the identical twin one shot prisoners dilemma only work if you are functionally identical or can you be a little different and is there anything meaningful that can be said about this? (I’m interested in this one also.)
I donated. I think Lightcone is helping strike at the heart of questions around what we should believe and do. Thank you for making LessWrong work so well and being thoughtful around managing content, and providing super quality spaces both online and offline for deep ideas to develop and spread!
What is your tax ID for people wanting to donate from a Donor Advised Fund (DAF) to avoid taxes on capital gains?
Cool. Is this right? For something with a 1/n chance of success I can have a 95% chance of success by making 3n attempts, for large values of n. About what does “large” mean here?
A small improvement to Wikipedia page on Pareto Efficiency
I’m confused by what you mean by “non-pragmatic”. For example, what makes “avoiding dominated strategies” pragmatic but “deference” non-pragmatic?
(It seems like the pragmatic ones help you decide what to do and the non-pragmatic ones help you decide what to believe, but then this doesn’t answer how to make good decisions.)
I meant this as a joke since if there’s one universe that contains all the other universes since it isn’t limited by logic, and that one doesn’t exist then that would mean I don’t exist either and wouldn’t have been able to post this. (Unless I only sort-of exist in which case I’m only sort-of joking.)
We can be virtually certain that 2+2=4 based on priors. This is because it’s true in the vast multitude of universes. In fact all the universes except the one universe that contains all the other universes. And I’m pretty sure that one doesn’t exist anyway.
Code here,
The link to code isn’t working for me. (Update: Worked on Safari but not Chrome)
How about a voting system where everyone is given 1000 Influence Tokens to spend across all the items on the ballot? This lets voters exert more influence on the things they care more about. Has anyone tried something like this?
(There could be tweaks like if people are avoiding spending on winners it could redistribute margin of victory, or if avoiding spending on losers it could redistribute tokens when losing, etc. but I’m not sure how much that would happen. The more interesting thing may be how does it influence everyone’s sense of what they are doing?)
Thanks for your reply! Yes, I meant identical as in atoms not as in “human twin”. I agree it would also depend on what the payout matrix is. My margin would also be increased by the evidentialist wager.
Should you cooperate with your almost identical twin in the prisoner’s dilemma?
The question isn’t how physically similar they are, it’s how similar their logical thinking is. If I can solve a certain math problem in under 10 seconds, are they similar enough that I can be confident they will be able to solve it in under 20 seconds? If I hate something will they at least dislike it? If so, then I would cooperate because I have a lot of margin on how much I favor us both to choose cooperate over any of the other outcomes so even if my almost identical twin doesn’t favor it quite as much I can predict they will still choose cooperate given how much I favor it (and more-so that they will also approach the problem this same way; if I think they’ll think “ha, this sounds like somebody I can take advantage of” or “reason dictates I must defect” then I wouldn’t cooperate with them).
A key question is how prosaic AI systems can be designed to satisfy the conditions under which the PMM is guaranteed (e.g., via implementing surrogate goals)
Is something like surrogate goals needed, such that the agent would need to maintain a substituted goal, for this to work? (I don’t currently fully understand the proposal but my sense was the goal of renegotiation programs is to not require this?)
Thank you @GideonF for taking the time to post this! This deserved to be said and you said it well.
Yes, and also just that I find it a little creepy/alien to imagine a young child that could be that good at math.