I want everyone to be able to achieve world they would really like; guess the best way to do that is to help people learn, build one’s strengths, build small-scale and large-scale projects, and also to cooperate.
Any gift must be accepted with gratitude and, if possible, with grace.
As of 2024-12-21, I have signed no contracts I cannot mention exist. I finally got to adding this notice thanks to the one in the-gears-to-ascension’s bio.
That article is suspiciously scarce on what microcontrols units… well, glory to LLMs for decent macro management then! (Though I believe that capability is still easier to get without text neural networks.)
In StarCraft II, adding LLMs (to do/aid game-time thinking) will not help the agent in any way, I believe. That happens because inference has a quite large latency, especially as most of prompt changes with all the units moving, so tactical moves are out; strategic questions “what is the other player building” and “how many units do they already have” are better answered by card-counting counting visible units and inferring what’s the proportion of remaining resources (or scouting if possible).
I guess it is possible that bots’ algorithms are improved with LLMs but that requires a high-quality insight; not convinced that o1 or o3 give such insights.
I don’t think so as I had success explaining away the paradox with concept of “different levels of detail”—saying that free will is a very high-level concept and further observations reveal a lower-level view, calling upon analogy with algorithmic programming’s segment tree.
(Segment tree is a data structure that replaces an array, allowing to modify its values and compute a given function over all array elements efficiently. It is based on tree of nodes, each of those representing a certain subarray; each position is therefore handled by several—specifically, nodes.)
Doesn’t the “threat” to delete the model have to be DT-credible instead of “credible conditioned on being human-made”, given that LW with all its discussion about threat resistance and ignoring is in training sets?
(If I remember correctly, a decision theory must ignore “you’re threatened to not do X, and the other agent is claiming to respond in such a way that even they lose in expectation” and “another agent [self-]modifies/instantiates an agent making them prefer that you don’t do X”.)
The surreal version of the VNM representation theorem in “Surreal Decisions” (https://arxiv.org/abs/2111.00862) seems to still have a surreal version of the Archimedean axiom.
That’s right! However it is not really a problem unless we can obtain surreal probabilities from the real world; and if all our priors and evidence are just real numbers, updates won’t lead us into the surreal area. (And it seems non-real-valued probabilities don’t help us in infinite domains, as I’ve written in https://www.lesswrong.com/posts/sZneDLRBaDndHJxa7/open-thread-fall-2024?commentId=LcDJFixRCChZimc7t.)
Re the parent example, utility function (or its evaluations) changing in an expectable way seems problematic to rational optimizing. If you know you prefer A to B, and know that you will prefer B to A in future even given only current context (so no “waiter must run back and forth”), then you don’t reflectively endorse either decision.
Yes, many people will have problems with the Archimedes’ axiom because it implies that everything has a price (that any good option can be probability-diluted enough that a mediocre is chosen instead), and people don’t take it kindly when you tell “you absolutely must have a trade-off between value A and value B” - especially if they really don’t have a trade-off, but also if they don’t want to admit or consciously estimate it.
Thankfully, that VNM property is not that critical for rational decision-making because we can simply use surreal numbers instead.
One possible real-world example (with integer-valued for deterministic outcomes) would be a parent whose top priority is minimizing the number of their children who die within the parent’s lifetime, with the rest of their utility function being secondary.
Wouldn’t work well since in real world outcomes are non-deterministic; given that, minimizing expected number is accomplished by simply having zero children.
But technology is not a good alternative to good decision making and informed values.
After thinking on this a bit, I’ve somewhat changed my mind.
(Epistemic status: filtered evidence.)
Technology and other progress has two general directions: a) more power for those who are able to wield it; b) increasing forgiveness, distance to failure. For some reason, I thought that b) was a given at least on average. However, now it came to mind that it’s possible for someone to
1) get two dates to accidentally overlap (or before confirming with partners-to-be that poly is OK),
2) lose an arbitrarily large bunch of money on gambling just online,
3) take revenge on a past offender with a firearm (or more destructive ways, as it happens),
and I’m not sure the failure margins have widened over time at all.
By the way, if technology effects aren’t really on topic, I’m open to move that discussion to shortform/dialogue.
---
(Epistemic status: obtained with introspection.)
Continuing the example with sweets, I estimate my terminal goals to include both “not be ill e.g. with diabetes” and “eat tasty things”. Given tech level and my current lifestyle, there isn’t instrumental goal “eat more sweets” nor “eat less sweets”; I think I’m somewhere near the balance, and I wouldn’t want society to pass any judgement.
I object to the framing of society being all-wise, and instead believe that for most issues it’s possible to get the benefits of both ways given some innovators on that issue. For example, visual communication was either face-to-face or heavily resource-bounded till the computer era—then there were problems of quality and price, but those have been almost fully solved in our days.
Consequently, I’d prefer “bunch of candy and no diabetes still” outcome, and there are some lines of research/ideas into how this can be done.
As for “nonmarital sex <...> will result in blowing past Goodhart’s warnings into more [personal psychological, I suppose] harm than good”, that seems already solved with the concept of “commitment”? The society might accept someone disregarding another person if that’s done with plausible deniability like “I didn’t know they would even care”, and commitment often makes you promise to care about partner’s feelings, solving* the particular problem in a more granular way than “couples should marry no matter what”. The same thing goes with other issues.
That said, I’ve recently started to think that it’s better to not push other people to less-socially-accepted preferences unless you have a really good case they can revert from exploration well and would be better off (and, thus, better not to push over social networks at all), since the limit point of person’s preferences might shift—wicked leading to more wicked and so on—to the point person wouldn’t endorse outcomes of change on reflection. I’m still considering if just noting that certain behavior is possible is a nudge significant enough to be disadvantaged (downvoted or like).
*I’d stop believing in that if commitment-based cultures had higher rate of partners failing on their promises to care than marriage-based; would be interested in some evidence either way.
Nicely written!
A nitpick: I believe “Voluntary cooperation” shouldn’t always be equal to “Pareto preferred”. Consider an Ultimatum game, where two people have 10 goodness tokens to split; the first person suggests a split (just once), then the second may accept or reject (when rejecting, all tokens are discarded). 9+1 is Pareto superior to 0+0 but one shouldn’t [100%] accept 9+1 lest that becomes anything they are ever suggested. Summarizable with “Don’t do unto yourself what you wouldn’t want to be done unto you”, or something like that.
Well I’d really love to see a practical example because for now the text clicks as “yes, that’s how it should be, indeed, and what’s novel here?”. (By the way it seems you haven’t yet started at how the gains would be divided; that seems relevant for continuing, and is already described in https://www.lesswrong.com/posts/vJ7ggyjuP4u2yHNcP/threat-resistant-bargaining-megapost-introducing-the-rose.)
Do you have an operational definition of “intrinsic preferences”?
Let me try. I believe that if copies of a person (as determined by their genotype*) would be raised in different cultures and environments, their revealed preferences would mainly be clustered around a single point, with some shifts determined by what desires society showed them as acceptable/tractable and what as unfashionable. Given proper diversity of surrounding cultures list, I’d say the cluster median indicates person’s intrinsic preferences.
*it also seems intuitively right to discount traits that were [IVF?]optimized for, if any, as an outside influence; though, I have no strong opinion on this.
This article seems like it was not written for LessWrong? It explores how we need to do conscious thinking, how we need to suggest and evaluate choices using our emotions, and essentially rephrases part of Sequences while reverting to pre-LW terminology, for “rationality” in particular.
And on top of that, “superrationality” concept is described unsoundly:
Different choices are impossible …
then if a dilemma requires two players to choose different ways at risk of losing (e.g. colliding on road, or booking one conference room at the same time), the text implies there would be no valid choice. In fact we know that there are randomized strategies, and that there often are distinctors useful for decision making.
DeekSeek v3: The Six Million Dollar Model
A typo in the title.
When happy and content, the Assistant Characters can channel more helpfulness and capabilities.
Is there anything preventing them to channel capabilities and helpfulness regardless of context, or is it in a large part derived?
I also use this idea of conditioning on context when taking notes on probability+decision theory!
In the same vein, I suggest to use for “philosophers of perfect emptiness” who take nothing but mathematical logic as valid.
At personal level, “debt” usually stands for something that will be paid back eventually. Not claiming whether US should strive to pay out most debt, but that may help explain the people’s intuitions.
Does your family have the same opinions as your social circle?
Quite similar, in fact—at least where they care to do so! I do listen for perspective, but I still can’t put society’s revealed opinion into span of those who I know better!
Getting to know your neighbors is another way to expose yourself to people who often think differently.
A good idea! I’ll have to take it a bit more general, because I’m an university student with dormitory and already know many people around; though, eating at some local cafe with diverse customers should work.
Being an Uber driver would be too taxing on my time, but I’m sure there is another idea instantiation which would work!
(And while we’re in an Open Thread, I’d like to thank LessWrong for featuring that one can make informed decisions on pretty much any topic! I’ve chosen uni aligned with me and I’m not disappointed with it so far.)
Tsukasa just plain lost and the logical ending is for him to be stoned or killed
except that
1) there is sometimes too much of such hostility in the real world,
2) some people can legitimately be redeemed—that is, they change thinking strategies and approximations to their values when they see what has been created (re: Einstein’s reaction to atomic bomb creation),
3) I don’t think anyone depicted in the anime would have valued fairness/consistency/other base for punishing Tsukasa—and with death penalty, no less—over compassion.
Authors might have held the position (shout-out to @the gears to ascension!) “[we] want literally every human to get to go to space often and come back to a clean and cozy world. This currently seems unlikely. Let’s change that.” Even if they haven’t: there is such vibe, and I somewhat endorse it!
Nice to see more fans!)) I have also written a review with more focus on rationality demonstration within the anime: https://www.lesswrong.com/posts/yqXoFx7jsSjbWgvBq/review-dr-stone.
An interesting framing! I agree with it.
As another example: in principle, one could make a web server use an LLM connected to database to serve any requests, not coding anything. It would even work… till the point someone would convince the model to rewrite the database to their whims! (A second problem is that normal site should be focused on something, in line with famous “if you can explain anything, your knowledge is zero”.)