MMath Cambridge. Currently studying postgrad at Edinburgh.
Donald Hobson
Karma: 3,083
The first merger event that Ligo detected was 1 billion ly away and turned 1 solar mass into gravitational waves. at a distance of so energy flux received is approx The main peak power output from the merging black holes lasted around one second. A full moon illuminates earth with around . So even if the aliens are great at making gravitational waves, they aren’t a good way to communicate. If they send a gravitational wave signal just powerful enough for us to detect with our most sensitive instruments, with the same power as light they could outshine the moon. Light is just more easily detected.
Milky way
Mass of observable universe
Radius of observable universe
Lets suppose that the milky way has (5% of all stars) stars suitable for life. (because some stars are too small or close to the galactic core.
Scaling up by mass gives around stars of interest.
As planetary location is not known, they must fill the entire habitable zone with energy. Radius of earths orbit so area around
This gives total that it must illuminate to hit us.
If they want to broadcast the galaxies mass-energy over 3billion years () then flux is
, 100X a full moons illumination.
If they choose to send a 1 second pulse of energy every 30 years, That is time ratio so the flux for that second can be , This is comparable to the beam of current laser weapons or being within 1km of the trinity test, and would likely set fire to most exposed organics and heat the surface of black rocks to a red glow. As we can see, the aliens can reduce power usage by orders of magnitude and still be highly noticeable.
From a recent online article, now removed, or a spoof, can you tell which?
“After hearing about protests in Catelonia, Trump asked for riot police to be sent in to deal with the problem. An advisor tried to explain that Spain was an independant country he didn’t have any power over. It apeared Trump hadn’t realized that many, independent countries existed, each with their own power structure. White house staff had to make fake TV news, showing american police dispelling the protests to keep Trump happy.”
That was what I was considering. I was hoping the aliens had telescopes that could see the collapsing cloud of gas and work out where the star would end up.
There exist some maths problems that even ASI can’t solve, because they require more computation than fits in the universe. To prove this, consider the set of all programs that take in an arbitrary turing machine and return “halt” “no halt” or “unsure”. Rule out all the programs that are ever wrong. Rule out all the programs that require more computation than fits in the universe. Consider a program that take in a turing machine and applies all such programs to it. If any of them return “halt” then you have worked out that it halts in finite time. If any return “no halt” then you know it does not halt. As the halting problem can’t be solved, then the program must sometimes return unsure. That is there must exist instances of the halting problem that no program that fits in the universe can solve. (Assuming the universe contains a finite amount of computation)
These problems aren’t actually that important to the real world. They are abstract mathematical limitations that wouldn’t stop the AI from achieving a decisive strategic advantage. There are limits, but they aren’t very limiting.
The AI needs at least some data to deduce facts about the world. This is also not very limiting. Will it need to build huge pieces of physics equipment to work out how the universe works, or will it figure it out from the data we have already gathered? Could it figure out string theory from a copy of Kepler’s notes? We just don’t know. It depends if there are several different theories that would produce similar results.
What is this incoherent rubbish doing here? It has a lot of math but no description whatsoever of what the math is about, for all we know, it could be a random arithmetic problem.
The problem with such arguments is that they shoot out their own support. If I am in a simulation, then the external universe can be any size and shape it likes for all I know.
But Statement 1 doesn’t imply caution is always best. A plausibly friendly AI based on several dubious philosophical assumptions (50% chance of good outcome) is better than taking our time to get it right(99%) if someone else will make a paperclip maximizer in the mean time. We want to maximize the likelyhood of the first AI being good, which may mean releasing a sloppily made, potentially friendly AI in race conditions. (assuming the other side can’t be stopped)
The potential exploitability of infinite options
“If the subject is Paul Christiano, or Carl Shulman, I for one am willing to say these humans are reasonably aligned; and I’m pretty much okay with somebody giving them the keys to the universe in expectation that the keys will later be handed back.”
I’m not saying that this isn’t true, but it sounds like a world wrecking assumption that can’t be mathematically proved, so it looks worthwhile to question it. Suppose you take the policy that the goal function of everyone in the AI alignment—transhumanism movement is basically similar, and you will let individuals get that sort of power. You have created a huge incentive for people with other goals to pretend to have that goal, work your way into the community, and then turn round and do something different.
If we consider blog posts as a good indication of someones values, which can be used to decide who gets such power, then they stop becoming good indicators as people lie. If you don’t hand people that power just because they claim to have good values,then the claimed values do indicate real values. Goodharts law in action.
It’s even possible that no-one can be trusted with that power. Suppose that Fair Utopia has a utility of 99 to everyone, and person X in charge has a utility of 100 to person X, and 0 to everyone else.
Of course, you have to wonder, are all those copies of Paul Christiano suffering. They were said to be very similar to the original. Probably no more than the original felt, slightly bored or itchy leg ect. If they really are perfect copies, could they realize they were that, and have an existential freak out? Is it murder to turn off all those similar copies once the jobs done? Will the copies think it is and go on slow strike to live longer?
It is possible to make a Turing machine that returns “HALT”, “RUNS FOREVER” or “UNSURE” and is never wrong. If the universe as we know it runs on quantum mechanics, it should be possible to simulate a mathematician in a box with unlimited data storage and time, using a finite Turing machine. This would imply the existence of some problem that would leave the mathematician stuck forever. There are plenty of Turing machines where we have no Idea if they halt. If you describe the quantum state of the observable universe to plank length resolution, what part of humans supposedly infinite minds is not described in these 2^2^100 bits of information?
If anyone wants a go, here’s a Turing machine to try on. Does it halt?
Written in python for the convenience of coders.
A lot of people value making things, but the type of things I want to make is different from the type of things I want to use. Many people want to make art of various kinds, fewer would value making disposable plastic packaging. There is also the fact that if a non expert is making something for the fun of it, there is a good chance it just won’t work. I might want to play a functional flute despite having made a flute that sounds like strangled cat. And if you want to build anything high tech, you can’t easily work your way up from raw materials. Many people would like to do some programming without having to build microelectronic components.
Human values are fairly complex and fragile. Most human values are focused around points in mind design space that are similar to ours. We should expect a randomly generated AI to not be a good successor. Any good successor would have to result from some process that approximately copies our values. This could be rerunning evolution to create beings with values similar to ours, or it could be an attempt at alignment that almost worked.
I’m not sure what simulating our civilization is supposed to achieve? If it works, we would get beings who were basically human. This would double the population, and get you some digital minds. Much the same thing could be achieved by developing mind uploading and a pro-natal culture. Neither will greatly help us to build an aligned super intelligence, or stop people building an unaligned one.
On the partially aligned AI, this just means we don’t need to get AI perfectly aligned for the future to be good, but the closer we get, the better it gets. An AI that’s running a hard coded set of moral rules wont be as good as one that lets us think about what we want to do, but if those rules are chosen well, they could still describe most of human value. (eg CelestAI from friendship is optimal)
If we believe that morally valuable alien life probably could exist in our future light cone, then an expansionist AI that has no moral value is much worse than blowing ourselves up with nukes.
“safe” Is a value loaded word. To make a “safe” car, all you need is the rough approximation to human values that says humans value not being injured. To make a “safe” super intelligence, you need a more detailed idea of human values. This is where the philosophy comes in, to specify exactly what we want.
I would argue that a prisoners dilemma situation applies. Assume that most people want party A to win, but don’t care enough that the tiny chance of getting a deciding vote is worth it.
If all sensible people decide not to vote, the vote is ruled by a few nutcases voting for party N. Suppose there are a million sensible people voting, they get utility +1000 from A winning and −1 from voting. Suppose that the number of nutters voting N is evenly distributed between 0 and a million. Staying home gets you 1 utility but costs everyone 0.001 utility. If everyone does it, they are all worse off.
There are several problems with this argument, firstly the AI has code describing its goal. It would seem much easier to copy this code across than to turn our moral values into code. Secondly the AI doesn’t have to be confident in getting it right. A paperclipping AI has two options, it can work at a factory and make a few paperclips, or it can self improve, but it has a chance of the resultant AI won’t maximize paperclips. However the amount of paperclips it could produce if it successfully self improves is astronomically vast. If its goal function is linear in paperclips, it will self improve if it thinks it has any chance of getting it right. If it fails at preserving its values as its self improving then the result looks like a staple maximizer.
Humans (at least the sort thinking about AI) know that we all have roughly similar values, so if you think you might have solved alignment, but aren’t sure, it makes sense to ask for others to help you, to wait for someone else to finish solving it.
However a paperclipping AI would know that no other AI’s had its goal function. If it doesn’t build a paperclipping super-intelligence, no one else is going to. It will therefore try to do so even if unlikely to succeed.
https://github.com/DonaldHobson/AI-competition-2