(I am genuinely curious about reasons behind downvotes)
quetzal_rainbow
Idea for experiment: take a set of coding problems which have at least two solutions, say, recursive and non-recursive. Prompt LLM to solve them. Is it possible to predict which solution LLM will generate from activations due to first token generation?
If it is possible, it is the evidence against “weak forward pass”.
I’ve seen LLMs generating text backwards. Theoretically, LLM can keep pre-image in activations, calculate hash and then output in order hash, pre-image.
“Frontiers” is known to publish various sorts of garbage. If somebody comes to you to argue some controversial point with article from Frontiers, you can freely assume it to be wrong.
Bandits have obvious cost: if they kill all farmers, from whom are they going to take stuff?
(This text should be understood as written from within the frame of OP: I do not endorse it fully, I just accept the premise and infer conclusions)
The reason why extreme suffering and injustice exist is interpersonal justification of Goodness of Reality hypothesis: the first thing upon hearing it is to dismiss it as naive, romantic, squamish, lacking skin in the game. It is easy to forgive when you have never been wronged. Enduring extreme suffering gives the possibility to answer: “We suffered and we forgive, why can’t you?”
Humans write computer viruses for far more money than price of token generation.
Quoting the bill:
“Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative.
I don’t understand how one part of the post relates to another. Yeah, sure, computational irreducibility of the world can make understanding of the world impossible and this would be sad. But I don’t see what it has to do with “Platonic laws of physics”. Current physics, if anything else, is sort of antiplatonic: it claims that there are several dozens of independent entities, actually existing, called “fields”, which produce the entire range of observable phenomena via interacting with each other, and there is no “world” outside this set of entities. “Laws of nature” are just “how this entities are”. Outside very radical skepticism I don’t know any reasons to doubt this worldview.
Multiple comments in one trenchcoat:
I am fine with your iterated mugging? “Forever” without discount implies infinities which is not nice, but we can assume that I’ll live 50 years more, so I either need to pay 100*365*50 = 182500$ with delay or get 18250000$ with delay, which sounds like pretty good bet? The only issue I see here is necessity to suspend prior disbelief in such bets and reasonable doubt in honesty of person who does not warn in advance about future payments.
***
Note that betting is a sort of counterfactual mugging in itself: you agree to pay in case of one outcome conditional on payment in case of the other outcome. If resulting outcome is not win and there is no causal enforcement, then your decision to pay or not to pay is similar to decision to pay or not to pay in counterfactual mugging.
***
I think that the simplest way to ignore weird hypotheses is to invoke computing power.
Roughly: there are many worlds that require less computing power to consider which are also more probable than lizard world, so you redirect computing power from considering lizard world to get more utility.
Correspondingly, if lizard world is sufficiently similar to our world and has similar restrictions on computing power, it will likely not consider our world, reducing incentive to analyze it further.
Conversively, if we imagine the world with vastly more computing power than we have, we likely won’t be able to analyze it, which incentivizes such world to not try to influence our policies.
To finish, if we imagine the world where it is possible to actually build literal Solomonoff induction which is capable to consider all computable universes in finite time, such world is possibly sufficiently weird to justify considering all possible weird hypotheses, because such world probably has multiple AIXIs which have multiple different utility functions and different priors and run multiple computable copies of you in all sorts of weird circumstances, even if we don’t invoke any multiversal weirdness. (It probably sucks to be finite computable structure is such worlds.)
In the end, actual influence on our updateless decisions is limited by very narrow set of worlds, like “different outcomes of real analogs of Prisoner’s dilemma with people on the same planet with you” and we don’t need to worry about lizard worlds until we solve all our earthly problems and start to disassemble galaxies into computronium.
Roughly, Earth average temperature:
Where j is dissipating power per area and sigma is Stephan-Boltzmann constant.
We can estimate j as
Where is a solar constant 1361 W/m^2. We take all incoming power and divide it by Earth surface area. Earth albedo is 0.31.
After substitution of variables, we get Earth temperature 254K (-19C), because we ignore greenhouse effect here.
How much humanity power consumption contributes to direct warming? In 2023 Earth energy consumption was 620 exajoules (source: first link in Google), which is 19TW. Modified rough estimation of Earth temperature is:
Human power production per square meter is, like, 0.04W/m^2, which gives us approximately zero effect of direct Earth heating on Earth temperature. But what happens if we, say, increase power by factor x1000? We are going to get increase of Earth temperature to 264K, by 10K, again, we are ignoring greenhouse effect. But qualitatively, increasing power consumption x1000 is likely to screw the biosphere really hard, if we count increasing amount of water vapor, CO2 from water and methane from melting permafrost.
How is it realistic to get x1000 increase in power consumption? Well, @Daniel Kokotajlo at least thought that we are likely to get it somewhere in 2030s.
Certainly one of the strongest indicators of autism is that both autistic and non-autistic people tend to agree that autistic people have poor social skills, but quantitative psychologists often struggle with operationalizing social skills, and in my own brief research on the topic, I haven’t found a clear difference in social performance between autistic and non-autistic people. What’s going on?
I think there is a difference between skills (which are trainable), outcomes (which can be deliberately optimized), and innate ability (which seems to be actual source of struggle).
My current friends are usually very surprised when they learn that I was absolutely insufferable to interact with when I was 18 years old, but I just speedran social adaptation by hanging out with people tolerant of my sort of autism, and I still find it very tiring to be around people not from tech/science/artistic subcultures.
I don’t think you actually prove your point.
The principle of alpha-beta pruning is, roughly, “search for moves until it’s clear that all other moves are bad”. It means that if you find sufficiently good move on first iteration of search, you just stop search and execute this move. And if you have sufficiently good search-ordering heuristics, you won’t have much of tree search.
Why do you assume that open source equates to small hacker groups??? The largest supplier of open weights is Meta AI and their recent Llama-405B rivals SOTA models.
Are you implying that there are multiple hidden violent criminals who also don’t fit sociological profile for prosecuted violent criminals?
To clarify in more general way: Earth is bad in coordination in a sense that you can’t expect that if industrial producer dumps toxic waste into environment, special government agency will walk in with premise: “hey, you seem to destroy environmental commons to get profits, let’s negotiate point on Pareto frontier in space of environmental commons-profits which leave both of us not very upset”.
I agree that the word “any” is wrong here. I used “sure” in the sense of “reasonably everyday I-won’t-get-hit-by-a-car-if-I-cross-the-road-now sure,” not in the “100% math-proof sure” sense.
By “vibe-based,” I refer to the features that you mention. Yes, there is a lot of talk about consequentialism and efficiency in rationalist/EA-adjacent AIS circles, and consequentialism gives a base for power-seeking actions, but it’s unclear how much this talk leads to actual power-seeking actions and how much it’s just local discourse framing.
The same applies to the feel of urgency.
Funnily, consequentialism can lead to less power-seeking if we define the problem in a less open-ended manner. If your task is to “maximize the number of good things in the world,” you benefit from power-seeking. If your task is to “design a safe system capable of producing superintelligent work,” you are, in fact, interested in completing this task with minimal effort and, therefore, minimal resources.
I think that the broad environmentalist movement is at least in the same tier as the AIS movement for all of the mentioned features.
Power-seeking inferred from consequentialism? Environmentalism is a movement about political control from the very start. The Club of Rome was initially founded inside the OECD and tried to influence its policy towards degrowth.
Consequentialism leading to questionable practices? Mass sterilizations in the 1980s are far beyond whatever AI Safety has done to date.
Urgency? You can look at all the people claiming that 2030 is a point of no return for climate change.
Focus of elites? The Club of Rome and the Sierra Club are literally what is written in the name—they are elite clubs.
I think I would agree if you say that there are a lot of nuances and the chance of AIS being more power-seeking than the environmentalism movement is non-negligible, but to measure power-seeking with necessary accuracy, we would need not a blog post written by one person, but the work of an army of sociologists.
I think majority of nations would support dismantling their governments in favor of benevolent superintelligence, especially given correct framework. And ASI can simply solve problem of meat by growing brainless bodies.
I think your example is closer to outer alignment failure—model was RLHFed to death to not offend modern sensibilites and developers clearly didn’t think about preventing this particular scenario.
My favorite example of pure failure of moral judgement is this post.
I don’t quite understand why “fast” and “slow” are called such. Hansons “Economic growth given machine intelligence” assumes just efficient conversion of capital into human-level labor and it gets doubling time of economy 18 month, which is pretty much your “flash economy” story?
First season of Game of Thrones was released in 2011, and first book was written in 1996, I think we got rrrrreally desensitized here.