I noticed the tag posts imported from Arbital that haven’t been edited on LW yet can’t be found when searching those tags from the “Add Tags” button above posts. Adding ineffective edits like spaces at the end of a paragraph seems to fix that problem.
Morpheus
Karma: 676
I didn’t downvote, but my impression is the post seems to hand-wave away a lot of problems and gives the impression you haven’t actually thought clearly and in detail about whether the ideas you propose here are feasible.
Some people have been thinking for quite some time now that an AI that wants to be changed would be great, but that it’s not that easy to create one, so how is your proposal different? Maybe checkout the corrigibility tag. Figuring out which desiderata are actually feasible to implement and how is the hard part. Same goes for your Matroshka bunkers. What useful work are you getting out of your 100% safe Matroshka bunkers? After you thought about that for 5 minutes+, maybe checkout the AI boxing tag and the AI oracle tag. Maybe there is something to the reversibility idea ¯\_(ツ)_/¯.
Also using so many tags gives a bad impression (“AI Timelines”? “Tiling Agents”? “Infinities in Ethics”?). Read the description of the tags.
Finding some some friend (or language model?) to play Zendo (the science game) with makes this really intuitive on a gut level. Guessing a rule based on whether a sequence of 3 integers is either accepted or rejected works pretty well via text.
NOTE: I posted this to LW and I’m new here so I don’t totally know the cross-posting policies. Hope it’s alright that I posted here too!
It seems you posted on LW twice instead or in addition to cross-posting to the EA forum.
I just tried this with o3-mini-high and o3-mini. o3-mini-high identified and prevented the fork correctly, while o3-mini did not even correctly identify it lost.
Yes!
I can only see the image of the 5-d random walk. The other images aren’t rendering.
I was already sold on singularity. For what it’s worth I found the post and comments very helpful for why you would want to take the sun apart in the first place and why it would be feasible and desirable for superintelligent and non-superintelligent civilization (Turning the sun into a smaller sun that doesn’t explode seems nicer than having it explode. Fusion gives off way more energy than lifting the material. Gravity is the weakest of the 4 forces after all. In a superintelligent civilization with reversible computers, not taking apart the sun will make readily available mass a taut constraint).
One thing I am pretty confident about is that methylation patterns are downstream, not upstream. Methyl group turnover time is far too fast to be a plausible root cause of aging. (In principle, there could be some special methyl groups which turn over slowly, but I would find that very surprising.)
My possibly wrong understanding here is that there are histone modifications and other proteins (like CTCF) that make methylation patterns way more stable? Which leads to some methylation patterns like imprinting for genes like IGF2 to be stable in most tissues over ~decades. Nevertheless, loss of imprinting and epigenetic marks still doesn’t necessarily seem like the most likely root cause of aging to me.
This argument against subagents is important and made me genuinely less confused. I love the concrete pizza example and the visual of both agent’s utility in this post. Those lead me to actually remember the technical argument when it came up in conversation.
I found Steven Byrnes valence concept really useful for my own thinking about psychology more broadly and concretely when reading text messages from my contextualizing friend (in that when a message was ambiguous, guessing the correct interpretation based on valence worked surprisingly well for me).
I ended up dodging the bullet of loosing money here, because I was a bit worried that Nate Silvers model might have been behind, because the last poll then was on the 23rd. I was also too busy with other important work to resolve my confusions before the election. My current two best guesses are:
The French whale did not have an edge,
The neighbour polling method is a just-so story to spread confusion, but he actually did have an edge
I don’t understand correctly how this neighbour polling method is supposed to work.
In any case, if Polymarket is still legal in 4 years I expect the prediction market on the election to be efficient relative to me and I will not bet on it.
I had a discussion with @Towards_Keeperhood what we would expect in the world where orcas either are or aren’t more intellectually capable than humans if trained. Main pieces I remember were: Orcas already dominating the planet (like humans do), large sea creatures going extinct due to orcas (similar to how humans drove several species extinct (Megalodon? Probably extinct for different reasons, weak evidence against? Most other large whales are still around)). I argued that @Towards_Keeperhood was also underestimating the intricacies that hunter-gatherers are capable of, and gave the book review for the secret of our success as an example. I think @Towards_Keeperhood did update in that direction after reading that post. I also reread that post and funnily enough stumbled over some evidence that orcas might have fallen into a similar “culture attractor” for intelligence, like humans:
Learn from old people. Humans are almost unique in having menopause; most animals keep reproducing until they die in late middle-age. Why does evolution want humans to stick around without reproducing?
Because old people have already learned the local culture and can teach it to others. Heinrich asks us to throw out any personal experience we have of elders; we live in a rapidly-changing world where an old person is probably “behind the times”. But for most of history, change happened glacially slowly, and old people would have spent their entire lives accumulating relevant knowledge. Imagine a world where when a Silicon Valley programmer can’t figure out how to make his code run, he calls up his grandfather, who spent fifty years coding apps for Google and knows every programming language inside and out.
Quick google search revealed Orcas have menopause too! While chimpanzees don’t! I would not have predicted that.
Typo in the linked document:
There is no one is coming to save us.
Can someone who is already trading on Polymarket or is planning to do so soon tell me if there are any hidden fees (or ways my money might be locked up for longer than I expect) if I trade on Polymarket? Four years ago I got hit by enormous ether gas fees on Augur, which still made my bet positive EV, but only barely so (I had to wait quite a while for the gas cost to go that low and was loosing out on investing the money and my attention). I plan to bet ~$3K-$7K and think Kamala Harris has a 45% chance of winning. Is that enough for all the transaction costs to vanish?
One confounder: depression/mania. Recently (the last ~two weeks) I have been having bad sleep (waking up 3-7 am and not feeling sleepy anymore (usually I sleep from midnight to 9). My current best guess is that the problem is that my life has been going too well recently, leading to a self-sustaining equilibrium where I have little sleep and mania. Reduced my medication today (~55mg instead of 70mg) which seems to have helped with the mania. I had another day with slight mania 1 month ago when sleeping little in order to travel to a conference, so in the future I’ll probably reduce my medication dose on such days. Took a friend describing his symptoms on too much medication for me to realize what is going on.
I am also interested in finding a space to explore ideas which are not well-formed. It isn’t clear to me that this is intended to be such a space. This may simply be due to my ignorance of the mechanics around here.
For not well-formed ideas, you can write a Quick Take (can be found by clicking on your profile name in the top right corner) or starting a dialogue if you want to develop the idea together with someone (can be found in the same corner).
I feel like there should exist a more advanced sequence that explains problems with filtered evidence leading to “confirmation bias”. I think the Luna sequence is already a great step in the right direction. I do feel like there is a lack of the equivalent non-fiction version, that just plainly lays out the issue. Maybe what I am envisioning is just a version of What evidence filtered evidence with more examples of how to practice this skill (applied to search engines, language models, someone’s own thought process, information actively hidden from you, rationality in groups etc.).
adult augmentation 2-3std for the average person seems plausible, but for the few +6std people on earth it might just give +0.2std or +0.3std, which tbc I think is incredibly worthwhile.
Such high diminishing returns in g based on genes seems quite implausible to me, but would be happy if you can point to evidence to the contrary. If it works well for people with average Intelligence, I’d expect it to work at most half as well with +6sd.
I noticed the tag posts imported from Arbital that haven’t been edited on LW yet can’t be found when searching those tags from the “Add Tags” button above posts. Adding ineffective edits like spaces at the end of a paragraph seems to fix that problem.