JBlack

Karma: 2,058

JBlack 19 Dec 2024 3:49 UTC
3 points
0
in reply to: RussellThor’s comment on: RussellThor’s Shortform
Temporarily adopting this sort of model of “AI capabilities are useful compared to human IQs”:
With IQ 100 AGI (i.e. could do about the same fraction of tasks as well as a sample of IQ 100 humans), progress may well be hyper exponentially fast: but the lead-in to a hyper-exponentially fast function could be very, very slow. The majority of even relatively incompetent humans in technical fields like AI development have greater than IQ 100. Eventually quantity may have a quality of its own, e.g. after there were very large numbers of these sub-par researcher equivalents running at faster than human and coordinated better than I would expect average humans to be.
Absent enormous numerical or speed advantages, I wouldn’t expect substantial changes in research speed until something vaguely equivalent to IQ 160 or so.
Though in practice, I’m not sure that human measures of IQ are usefully applicable to estimating rates of AI-assisted research. They are not human, and only hindsight could tell what capabilities turn out to be the most useful to advancing research. A narrow tool along the lines of AlphaFold could turn out to be radically important to research rate without having anything that you could characterize as IQ. On the other hand, it may turn out that exceeding human research capabilities isn’t practically possible from any system pretrained on material steeped in existing human paradigms and ontology.

JBlack 18 Dec 2024 0:56 UTC
LW: 2 AF: 1
0
AF
in reply to: Donald Hobson’s comment on: How counterfactual are logical counterfactuals?
If they have source code, then they are not perfectly rational and cannot in general implement LDT. They can at best implement a boundedly rational subset of LDT, which will have flaws.
Assume the contrary: Then each agent can verify that the other implements LDT, since perfect knowledge of the other’s source code includes the knowledge that it implements LDT. In particular, each can verify that the other’s code implements a consistent system that includes arithmetic, and can run the other on their own source to consequently verify that they themselves implement a consistent system that includes arithmetic. This is not possible for any consistent system.
The only way that consistency can be preserved is that at least one cannot actually verify that the other has a consistent deduction system including arithmetic. So at least one of those agents is not a LDT agent with perfect knowledge of each other’s source code.
We can in principle assume perfectly rational agents that implement LDT, but they cannot be described by any algorithm and we should be extremely careful in making suppositions about what they can deduce about each other and themselves.

JBlack 17 Dec 2024 23:39 UTC
2 points
0
in reply to: notfnofn’s comment on: notfnofn’s Shortform
Oh, I see that I misread.
One problem is that “every possible RNG call” may be an infinite set. For a really simple example, a binary {0,1} RNG with program “add 1 to your count if you roll 1 and repeat until you roll 0″ has infinitely many possible rolls and no maximum output. It halts with probability 1, though.
If you allow the RNG to be configured for arbitrary distributions then you can have it always return a number from such a distribution in a single call, still with no maximum.

JBlack 17 Dec 2024 3:01 UTC
2 points
0
in reply to: notfnofn’s comment on: notfnofn’s Shortform
My guess is “no” because both of you would die first. In the context of “largest numbers” 10^10^100 is baby’s first step, but is still a number with more digits than you will ever succeed in printing.
In principle the “you” in this scenario could be immortal with unbounded resources and perfect reliability, but then we may as well just suppose you are a superintelligence smarter than the AI in the problem (which isn’t looking so ‘S’ anymore).

JBlack 17 Dec 2024 2:34 UTC
LW: 10 AF: 3
0
AF
on: How counterfactual are logical counterfactuals?
Truly logical counterfactuals really only make sense in the context of bounded rationality. That is, cases where there is a logically necessary proposition, but the agent cannot determine it within their resource bounds. Essentially all aspects of bounded rationality have no satisfactory treatment as yet.
The prisoners’ dilemma question does not appear to require dealing with logical counterfactuals. It is not logically contradictory for two agents to make different choices in the same situation, or even for the same agent to make different decisions given the same situation, though the setup of some scenarios may make it very unlikely or even direct you to ignore such possibilities.

JBlack 5 Dec 2024 1:46 UTC
9 points
0
in reply to: Lorec’s comment on: Lorec’s Shortform
1. It’s an arbitrary convention. We could have equally well chosen a convention in which a left hand rule was valid. (Really a whole bunch of such conventions)
2. In the Newtonian 2-point model gravity is a purely radial force and so conserves angular momentum, which means that velocity remains in one plane. If the bodies are extended objects, then you can get things like spin-orbit coupling which can lead to orbits not being perfectly planar if the rotation axes aren’t aligned with the initial angular momentum axis.
  If there are multiple bodies then trajectories can be and usually will be at least somewhat non-planar, though energy losses without corresponding angular momentum losses can drive a system toward a more planar state.
  Zero dimensions would only be possible if both the net force and initial velocity were zero, which can’t happen if gravity is the only applicable force and there are two distinct points.
  In general relativity gravity isn’t really a force and isn’t always radial, and orbits need not always be planar and usually aren’t closed curves anyway. Though again, many systems will tend to approach a more planar state.

JBlack 4 Dec 2024 2:00 UTC
4 points
−5
in reply to: Decaeneus’s comment on: Decaeneus’s Shortform
I believe that there is already far too much “hate sharing”.
Perhaps the default in a social media UI should be that shared content includes a public endorsement of whatever content it links to, and if you want to “hate share” anything without such an endorsement, you have to fight a hostile UI to do so.
In particular, “things that are worth sharing” absolutely should not overlap with “want to see less of”. If you want to see less of some type of thing, it’s self-defeating to distribute more copies of it. Worse, if you even suspect that any of your own readers are anything like you, why are you inflicting it on them?

JBlack 2 Dec 2024 2:03 UTC
4 points
0
on: Is malice a real emotion?
Yes, it is a real emotion. I have felt it on some rare occasions. I do not act on it, though on such occasions I cannot rule out the possibility that it may influence me in less direct ways.
I don’t know what you mean by “best way to interpret it”. What sort of interpretation are you looking for? For example, what are your best ways of interpreting other emotions?

JBlack 28 Nov 2024 23:22 UTC
7 points
0
on: Is the mind a program?
The conclusion does not follow from the argument.
The argument suggests that it is unlikely that a perfect replica of the functioning of a specific human brain can be emulated on a practical computer. The conclusion generalizes that out to no conscious emulation of a human brain, at all.
These are enormously different claims, and neither follows from the other.

JBlack 27 Nov 2024 5:56 UTC
3 points
0
on: What epsilon do you subtract from “certainty” in your own probability estimates?
For all practical purposes, such credences don’t matter. Such scenarios certainly can and do happen, but in almost all cases there’s nothing you can do about them without exceeding your own bounded rationality and agency.
If the stakes are very high then it may make sense to consider the probability of some sort of trick, and attempt to get further evidence of the physical existence of the coin and that its current state matches what you are seeing.
There is essentially no point in assigning probabilities to hypotheses of failures of your mind itself. You can’t reason your way out of serious mind malfunction using arithmetic. At best you could hope to recognize that it is malfunctioning, and try not to do anything that will make things worse. In the case of mental impairment severe enough to have false memories or sensations this blatant, a rational person should expect that a person so affected wouldn’t be capable of correctly carrying out quantified Bayesian reasoning.
My own background credences are generally not insignificant for something like this or even stranger, but they play essentially zero role in my life and definitely not in any probability calculations. Such hypotheses are essentially untestable and unactionable.

JBlack 23 Nov 2024 3:01 UTC
5 points
10
on: Which things were you surprised to learn are metaphors?
In relativity, space and time are just different directions in spacetime with a single pseudometric determining separation between events. With this understanding, the time/space distance metaphor is more literal than most people think.
The correspondence isn’t exact since it’s a pseudometric and not a standard metric, and everyday units of time correspond to much greater than everyday units of distance, but it’s still more than just a metaphor.

JBlack 15 Nov 2024 0:58 UTC
2 points
0
on: The Foraging (Ex-)Bandit [Ruleset & Reflections]
Thanks for making this!
I found it a challenge to deduce strategies over many plays, rather than following the advice “not intended to be replayed”. The first playthrough was pretty much meaningless for me, especially given the knowledge that both time and history could affect the results. I just viewed it as one step of information gathering for the real game.
The suboptimal zones weren’t obviously suboptimal from a single pass, even Dragon Lake that always yields nothing. For all I knew, it could have yielded 5000 food with quite a low probability (and still be always optimal), or lesser amounts of food at specific combinations of time and day, or only when matching some rule based on the previous results of foraging in other zones.
After many runs I did settle on a strategy, and mentally scored myself by looking at the source to see whether there was anything that I should have spotted but didn’t. As it happened, my final strategy was almost optimal though I stayed on the rats for a few more hours than ideal.

JBlack 10 Nov 2024 2:40 UTC
4 points
2
in reply to: RHollerith’s comment on: lc’s Shortform
In principle I suppose one could build very large walls around it to reduce heat exchange with the rest of Earth and a statite mirror (or few slowly orbiting ones) to warm it up. That would change the southern hemisphere circulation patterns somewhat, but could be arranged to not affect the overall heat balance of the rest of Earth.
This is very unlikely to happen for any number of good reasons.

JBlack 9 Nov 2024 5:22 UTC
1 point
0
on: The Case Against Moral Realism
Only the first point “Good and evil are objectively real” is a necessary part of moral realism. Sometimes the first half of the third (“We have an objective moral obligation to do good and not do evil”) is included, but by some definitions that is included in what good and evil mean.
All the rest are assumptions that many people who believe in moral realism also happen to hold, but aren’t part of moral realism itself.

JBlack 3 Nov 2024 2:04 UTC
6 points
2
in reply to: Purplehermann’s comment on: Purplehermann’s Shortform
Research companies work best when there’s plenty of infrastructure that can supply stuff they need to do the research. Including, to mention one recent case, electricity. It also helps to be in an area where there is stable government that can protect the research site from civil or military unrest, and too much (or too unpredictable) corruption. You also want it to be a place where your researchers are happy to live while they do their research, and where you can relatively easily recruit other skilled workers.
China does meet these requirements, but it is not exactly lacking in bureaucracy so I’m not sure why it made the list. If you’re doing research involving human trials of some sort, you also want to be able to communicate well with the participants so extensive knowledge of the language and culture will be very useful.
All that said, plenty of organizations do carry out research all over the world, not just in rich countries with a lot of bureaucracy.

JBlack 1 Nov 2024 2:26 UTC
2 points
0
in reply to: jefftk’s comment on: Is the Power Grid Sustainable?
Yes, it definitely does depend upon local conditions. For example if your grid operator uses net metering (and is reliable) then it is not worthwhile at any positive price. This statement was in regard to my disputed upstream comment “Even now at $1000/kW-hr retail it’s almost cost-effective here [...]”.

JBlack 1 Nov 2024 2:07 UTC
2 points
0
in reply to: denkenberger’s comment on: Is the Power Grid Sustainable?
Batteries are primarily used for intra-day time shifting, not weekly. I agree that going completely off grid costs substantially more than being able to use your own generated power for 80-90% of usage. That’s why I focused on the case where home owners remain grid-connected in my top-level comment:
With smart meters and cheaper home battery systems the incentives starts to shift toward wealthier solar enthusiasts buying batteries and selling excess power to the grid at peak times (or consuming it themselves), lowering peak demand at no additional capital or maintenance cost to the grid operators.
The only mention I made regarding completely off-grid power systems was about the counterfactual scenario of $150/kW-hr battery cost, which I have not assumed anywhere else. I didn’t say that it would be marginally cost effective to go completely off grid with such battery prices, just that it would be substantially more cost-effective than buying all my power from the grid. The middle option of 80-90% reduced but not completely eliminated grid use is still cheaper than either of the two extremes, and likely to remain so for any feasible home energy storage system.
That’s what I was referring to regarding $700 kW/hr. At $1000/kW-hr it’s (just barely) not worth even buying batteries to shift energy from daytime generation to night consumption, while at $700/kW-hr it definitely is worthwhile. Do you need the calculation for that?

JBlack 31 Oct 2024 7:16 UTC
4 points
0
in reply to: denkenberger’s comment on: Is the Power Grid Sustainable?
At $150/kW-hr and assuming a somewhat low 3000 cycle lifetime, such batteries would cost $0.05 per cycled kW-hr which is very much cost-effective when paired with the extremely low cost but inconveniently timed nature of solar power. It would drop the amortized cost of a complete off-grid power system for my home to half that of grid power in my area, for example.
Even now at $1000/kW-hr retail it’s almost cost-effective here to buy batteries to time-shift energy from solar generation to time of consumption. At $700/kW-hr it would definitely be cost-effective to do daily load-shifting with the grid as a backup only for heavily cloudy days.
Pumped hydro is already underway in this region, though it’s proving more expensive and time-consuming to build than expected. Have there been some recent advances in compressed air energy storage? The information I read 2-3 years ago did not look promising at any scale.

JBlack 29 Oct 2024 4:46 UTC
2 points
0
in reply to: J Bostock’s comment on: Jemist’s Shortform
How do you construct a maximizer for 0.3X+0.6Y+0.1Z from three maximizers for X, Y, and Z? It certainly isn’t true in general for black box optimizers, so presumably this is something specific to a certain class of neural networks.

JBlack 29 Oct 2024 3:45 UTC
5 points
1
in reply to: denkenberger’s comment on: Is the Power Grid Sustainable?
Battery costs should be lower by now than they are.
For example, in Australia wholesale cell prices are on the order of $150/kW-hr, while installed battery systems are still more than $1000/kW-hr. The difference isn’t just packaging, electrical systems, and installation costs. Packaging doesn’t cost anywhere near that much, installation costs are relatively flat with capacity, and so are electrical systems (for given peak power). Yet battery system costs from almost all suppliers are almost perfectly linear with energy capacity.
I don’t know why there isn’t an alternative decent-quality supplier that would eat their lunch on large-capacity systems with moderate peak power. Such a thing should be still very highly profitable with a much larger market. It could be that there just hasn’t been enough time for such a market to develop, or supply issues, or something else I’m missing?