Excellent post. Please write more on ethics as safety rails on unseen cliffs.
Recovering_irrationalist
Karma: 295
Nazir, a secret hack to prevent Eliezer from deleting your posts is here. #11.6 is particularly effective.
Ah, I see...
other events may be offered at the same time, and I can not predict such events.
As far as Eliezer is currently aware, Saturday night should be clear.
I meant some of you singularity-related guys may want to meet me at other times, possibly at my apartment.
I’d love to come to another meet, Anna would too, probably others. I just wasn’t sure there’d be enough people for two, so focused on making at least one happen.
I guess this was not the right place to post such an offer.
If the invite extends to OB readers, you’re very welcome to share this page. If it’s just for us Singularitarians, it’s probably better to plan elsewhere and post a link here.
Oops, misinterpreted tags. Should read:
It’s 3am and the lab calls. Your AI claims [nano disaster/evil AI emergence/whatever] and it must be let out to stop it. It’s evidence seems to check out.
Even if we had the ultimate superintelligence volunteer to play the AI and we proved a gatekeeper strategy “wins” 100% (functionally equal to a rock on the “no” key) that wouldn’t show AI boxing can possibly be safe.
It’s 3am and the lab calls. Your AI claims and it must be let out to stop it. It’s evidence seems to check out...
If it’s friendly, keeping that lid shut gets you just as dead as if you let it out and it’s lying. That’s not safe. Before it can hide it’s nature, we must know it’s nature. The solution to safe AI is not a gatekeeper no smarter than a rock!
Besides, as Drexler said, Intelligent people have done great harm through words alone.
If there’s a killer escape argument it will surely change with the gatekeeper. I expect Eliezer used his maps the arguments and psychology to navigate reactions & hesitations to a tiny target in the vast search space.
A gatekeeper has to be unmoved every time. The paperclipper only has to persuade once.
I’m not saying this is wrong, but in its present form, isn’t it really a mysterious answer to a mysterious question? If you believed it, would the mystery seem any less mysterious?
Hmm. You’re right.
Darn.
it doesn’t explain why we find ourselves in a low-entropy universe rather than a high-entropy one
I didn’t think it would solve all our questions, I just wondered if it was both the simplest solution and lacking good evidence to the contrary. Would there be a higher chance of being a Boltzmann brain in a universe identical to ours that happened to be part of a what-if-world? If not, how is all this low-entropy around me evidence against it?
Just because what-if is something that humans find deductively compelling does not explain how or why it exists Platonically
How would our “Block Universe” look different from the inside if it was a what-if-Block-Universe? It all adds up to...
Not trying to argue, just curious.
Eliezer: imagine that you, yourself, live in a what-if world of pure mathematics
Isn’t this true? It seems the simplest solution to “why is there something rather than nothing”. Is there any real evidence against our apparently timeless, branching physics being part of a purely mathematical structure? I wouldn’t be shocked if the bottom was all Bayes-structure :)
If there is that ‘g’/unhappiness correlation, maybe the causality is: unhappiness->‘g’. The overly happy, seeing less problems, get less problem solving practice, whereas a tendency to be analytical could boost ‘g’ over a lifetime, though perhaps not effective intelligence.
I wouldn’t expect this to apply to most readers, who get particular pleasure from solving intelligent problems. Think general population.
I wouldn’t assume a process seeming to churn through preference cycles to have an inconsistent preference ranking, it could be efficiently optimizing if each state provides diminishing returns. If every few hours a jailer offers either food, water or a good book, you don’t pick the same choice each time!
I’ve spent some time online trying to track down the exact moment when someone noticed the vastly tangled internal structure of the brain’s neurons, and said, “Hey, I bet all this giant tangle is doing complex information-processing!”
My guess is Ibn al-Haytham, early 11thC while under house arrest after realizing he couldn’t, as claimed, regulate the Nile’s overflows.
Wikipedia: “In the Book of Optics, Ibn al-Haytham was the first scientist to argue that vision occurs in the brain, rather than the eyes. He pointed out that personal experience has an effect on what people see and how they see, and that vision and perception are subjective. He explained possible errors in vision in detail, and as an example described how a small child with less experience may have more difficulty interpreting what he or she sees. He also gave an example of how an adult can make mistakes in vision due to experience that suggests that one is seeing one thing, when one is really seeing something else.”
Eliezer: The overall FAI strategy has to be one that would have turned out okay if Archimedes of Syracuse had been able to build an FAI.
I’d feel a lot safer if you’d extend this back at least to the infanticidal hunter-gatherers, and preferably to apes fighting around the 2001 monolith.
Are you’re rationally taking into account the biasing effect your heartfelt hopes exert on the set of hypotheses raised to your conscious attention as you conspire to save the world?
Recovering, in instances like these, reversed stupidity is not intelligence; you cannot say, “I wish fast takeoff to be possible, therefore it is not”.
Indeed. But you can, for example, say “I wish fast takeoff to be possible, so should be less impressed, all else equal, by the number of hypothesis I can think of that happen to support it”.
Do you wish fast takeoff to be possible? Aren’t then Very Horrid Singularities more likely?
All you can do is try to acquire the domain knowledge and put your mind into forward form.
Yes, but even then the ballot stuffing is still going on beneath your awareness, right? Doesn’t that still count as some evidence for caution?
Will Pearson: When you were figuring out how powerful AIs made from silicon were likely to be, did you have a goal that you wanted? Do you want AI to be powerful so it can stop death?
Eliezer: …”Yes” on both counts....
I think you sidestepped the point as it related to your post. Are you’re rationally taking into account the biasing effect your heartfelt hopes exert on the set of hypotheses raised to your concious attention as you conspire to save the world?
Carl Shulman: Occam’s razor makes me doubt that we have two theoretical negative utilitarians (with egoistic practice) who endorse Pascal’s wager, with similar writing styles and concerns, bearing Internet handles that begin with ‘U.’ michael vassar: Unknown and Utilitarian could be distinct but highly correlated (we’re both here after all). In principle we could see them as both unpacking the implications of some fairly simple algorithm.
With thousands of frequent-poster-pairs with many potentially matchable properties, I’m not too shocked to find a pair that match on six mostly correlated properties.
For example, I would be substantially more alarmed about a lottery device with a well-defined chance of 1 in 1,000,000 of destroying the world, than I am about the Large Hadron Collider switched on. If I could prevent only one of these events, I would prevent the lottery.
On the other hand, if you asked me whether I could make one million statements of authority equal to “The Large Hadron Collider will not destroy the world”, and be wrong, on average, around once, then I would have to say no.
Hmm… might this be the heuristic that makes people prefer a 1% chance of 1000 deaths to a definite death for 5? The lottery would definately destroy worlds, with as many deaths as killing over six thousand people in each Everett branch. Running the LHC means a higher expected number of dead worlds by your own estimates, but it’s all or nothing across universes. It will most probably just be safe.
If you had a definate number for both P(Doomsday Lottery Device Win) and P(Doomsday LHC) you’d shut up and multiply, but you haven’t so you don’t. But you still should because you’re pretty sure P(D-LHC) >> P(DLDW) even if you don’t know a figure for P(DLHC).
This assumes Paul’s assumption, above.
I doubt my ability to usefully spend more than $10 million/year on the Singularity. What do you do with the rest of the money?
Well I admit it’s a hell of a diminishing returns curve. OK… Dear Santa, please can I have an army of minions^H^H^H^Htrusted experts to research cool but relatively neglected stuff like intelligence enhancement, life extension (but not nanotech) & how to feed and educate the third world without screwing it up. And deal with those pesky existential risks. Oh, and free cryonics for everyone—let’s put those economies of scale to good use. Basically keep people alive till the FAI guys get there. Then just enjoy the ride, cos if I’ve just been handed $10^13 I’m probably in a simulation. More so than usual.
Anonymous: I’d hire all AI researchers to date to work under Eliezer and start seriously studying to be able to evaluate myself whether flipping the “on” switch would result in a friendly singularity.
(emphasis mine)I doubt this is the way to go. I want a medium sized, talented and rational team who seriously care, not every AI programmer in the world who smells money. I bring Eliezer a blank cheque and listen to his and those he trusts arguments for best use, though he’d have to convince me where we disagreed, he seems good at that.
Also, even after years of studying for it I wouldn’t trust me, or anyone else for that matter, to make that switch-on decision alone.
So far hardly any feedback on places & no restaurant recommendations. If I get no more responses by tomorrow I’ll just search the net for a well-reviewed restaurant that’s walkable-to from Montgomery Theater, good for groups, accepting of casual attire and hopefully not too crowded/noisy (with a private room?), book it for Saturday probably around 7pm for 21 people, post the details and directions and hope everyone turns up.
If you’d rather a different time, or have any preferences at all, please let me know before I do that. So far no one’s mentioned vegetarian, parking or wheelchair access needs, or preference for or against any food except one vote for pizza. How do you feel about Chinese? Italian? Mexican?