Dweomite

Karma: 1,205

Dweomite Sep 30, 2024, 7:56 PM
2 points
0
in reply to: Tapatakt’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
Then we’re no longer talking about “the way humans care about their friends”, we’re inventing new hypothetical algorithms that we might like our AIs to use. Humans no longer provide an example of how that behavior could arise naturally in an evolved organism, nor a case study of how it works out for people to behave that way.

Dweomite Sep 30, 2024, 6:44 PM
2 points
0
in reply to: Tapatakt’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
My model is that friendship is one particular strategy for alliance-formation that happened to evolve in humans. I expect this is natural in the sense of being a local optimum (in the ancestral environment), but probably not in the sense of being simple to formally define or implement.
I think friendship is substantially more complicated than “I care some about your utility function”. For instance, you probably stop valuing their utility function if they betray you (friendship can “break”). I also think the friendship algorithm includes a bunch of signalling to help with coordination (so that you understand the other person is trying to be friends), and some less-pleasant stuff like evaluations of how valuable an ally the other person is and how the friendship will affect your social standing.
Friendship also appears to include some sort of check that the other person is making friendship-related-decisions using system 1 instead of system 2--possibly as a security feature to make it harder for people to consciously exploit (with the unfortunate side-effect that we penalize system-2-thinkers even when they sincerely want to be allies), or possibly just because the signalling parts evolved for system 1 and don’t generalize properly.
(One could claim that “the true spirit of friendship” is loving someone unconditionally or something, and that might be simple, but I don’t think that’s what humans actually implement.)

Dweomite Sep 30, 2024, 6:07 PM
2 points
0
in reply to: adastra22’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
You appear to be thinking of power only in extreme terms (possibly even as an on/off binary). Like, that your values “don’t have power” unless you set up a dictatorship or something.
But “power” is being used here in a very broad sense. The personal choices you make in your own life are still a non-zero amount of power to whatever you based those choices on. If you ever try to persuade someone else to make similar choices, then you are trying to increase the amount of power held by your values. If you support laws like “no stealing” or “no murder” then you are trying to impose some of your values on other people through the use of force.
I mostly think of government as a strategy, not an end. I bet you would too, if push came to shove; e.g. you are probably stridently against murdering or enslaving a quarter of the population, even if the measure passes by a two-thirds vote. My model says almost everyone would endorse tearing down the government if it went sufficiently off the rails that keeping it around became obviously no longer a good instrumental strategy.
Like you, I endorse keeping the government around, even though I disagree with it sometimes. But I endorse that on the grounds that the government is net-positive, or at least no worse than [the best available alternative, including switching costs]. If that stopped being true, then I would no longer endorse keeping the current government. (And yes, it could become false due to a great alternative being newly-available, even if the current government didn’t get any worse in absolute terms. e.g. someone could wait until democracy is invented before they endorse replacing their monarchy.)
I’m not sure that “no one should have the power to enforce their own values” is even a coherent concept. Pick a possible future—say, disassembling the earth to build a Dyson sphere—and suppose that at least one person wants it to happen, and at least one person wants it not to happen. When the future actually arrives, it will either have happened, or not—which means at least one person “won” and at least one person “lost”. What exactly does it mean for “neither of those people had the power to enforce their value”, given that one of the values did, in fact, win? Don’t we have to say that one of them clearly had enough power to stymie the other?
You could say that society should have a bunch of people in it, and that no single person should be able to overpower everyone else combined. But that doesn’t prevent some value from being able to overpower all other values, because a value can be endorsed by multiple people!
I suppose someone could hypothetically say that they really only care about the process of government and not the result, such that they’ll accept any result as long as it is blessed by the proper process. Even if you’re willing to go to that extreme, though, that still seems like a case of wanting “your values” to have power, just where the thing you value is a particular system of government. I don’t think that having this particular value gives you any special moral high ground over people who value, say, life and happiness.
I also think that approximately no one actually has that as a terminal value.

Dweomite Sep 28, 2024, 6:44 AM
2 points
0
in reply to: adastra22’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
In the context of optimization, values are anything you want (whether moral in nature or otherwise).
Any time a decision is made based on some value, you can view that value as having exercised power by controlling the outcome of that decision.
Or put more simply, the way that values have power, is that values have people who have power.

Dweomite Sep 26, 2024, 10:10 PM
1 point
0
in reply to: avturchin’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
I feel like your previous comment argues against that, rather than for it. You said that people who are trapped together should be nice to each other because the cost of a conflict is very high. But now you’re suggesting that ASIs that are metaphorically trapped together would aggressively attack each other to enforce compliance with their own behavioral standards. These two conjectures do not really seem allied to me.
Separately, I am very skeptical of aliens warring against ASIs to acausally protect us. I see multiple points where this seems likely to fail:
- Would aliens actually take our side against an ASI merely because we created it? If humans hear a story about an alien civilization creating a successor species, and then the successor species overthrowing its creators, I do not expect humans to automatically be on the creators’ side in this story. I expect humans will take a side mostly based on how the two species were treating each other (overthrowing abusive masters is usually portrayed as virtuous in our fiction), and that which one of them is the creator will have little weight. I do not think “everyone should be aligned with their creators” is a principle that humans would actually endorse (except by motivated reasoning, in situations where it benefits us).
  - Also note that humans are not aligned with the process that produced us (evolution) and approximately no humans think this is a problem
- Even if the aliens sympathize with us, would they care enough to take expensive actions about it?
- Even if the aliens would war to save us, would the ASI predict that? It can only acausally save us if the ASI successfully predicts the policy. Otherwise, the war might still happen, but that doesn’t help us.
- Even if the ASI predicts this, will it comply? This seems like what dath ilan would consider a “threat”, in that the aliens are punishing the ASI rather than enacting their own BATNA. It may be decision-theoretically correct to ignore the threat.
- This whole premise, of us being saved at the eleventh hour by off-stage actors, seems intuitively like the sort of hypothesis that would be more likely to be produced by wishful thinking than by sober analysis, which would make me distrust it even if I couldn’t see any specific problems with it.

Dweomite Sep 25, 2024, 5:46 PM
1 point
0
in reply to: avturchin’s comment on: What are the best arguments for/against AIs being “slightly ‘nice’”?
I don’t see why either expecting or not-expecting to meet other ASIs would make it instrumental to be nice to humans.

Dweomite Sep 24, 2024, 6:25 AM
3 points
0
on: What are the best arguments for/against AIs being “slightly ‘nice’”?
I have an intuition like: Minds become less idiosyncratic as they grow up.
A couple of intuition pumps:
(1) If you pick a game, and look at novice players of that game, you will often find that they have rather different “play styles”. Maybe one player really likes fireballs and another really like crossbows. Maybe one player takes a lot of risks and another plays it safe.
Then if you look at experts of that particular game, you will tend to find that their play has become much more similar. I think “play style” is mostly the result of two things: (a) playing to your individual strengths, and (b) using your aesthetics as a tie-breaker when you can’t tell which of two moves is better. But as you become an expert, both of these things diminish: you become skilled at all areas of the game, and you also become able to discern even small differences in quality between two moves. So your “play style” is gradually eroded and becomes less and less noticeable.
(2) Imagine if a society of 3-year-olds were somehow in the process of creating AI, and they debated whether their AI would show “kindness” to stuffed animals (as an inherent preference, rather than an instrumental tool for manipulating humans). I feel like the answer to this should be “lol no”. Showing “kindness” to stuffed animals feels like something that humans correctly grow out of, as they grow up.
It seems plausible to me that something like “empathy for kittens” might be a higher-level version of this, that humans would also grow out of (just like they grow out of empathy for stuffed animals) if the humans grew up enough.
(Actually, I think most humans adults still have some empathy for stuffed animals. But I think most of us wouldn’t endorse policies designed to help stuffed animals. I’m not sure exactly how to describe the relation that 3-year-olds have to stuffed animals but adults don’t.)
I sincerely think caring about kittens makes a lot more sense than caring about stuffed animals. But I’m uncertain whether that means we’ll hold onto it forever, or just that it takes more growing-up in order to grow out of it.
Paul frames this as “mostly a question about idiosyncrasies and inductive biases of minds rather than anything that can be settled by an appeal to selection dynamics.” But I’m concerned that might be a bit like debating the odds of whether your newborn human will one day come to care for stuffed animals, instead of whether they will continue to care for them after growing up. It can be very likely that they will care for a while, and also very likely that they will stop.
I strongly suspect it is possible for minds to become quite a lot more grown-up than humans currently are.
(I think Habryka may have been saying something similar to this.)
Still, I notice that I’m doing a lot of hand-waving here and I lack a gears-based model of what “growing up” actually entails.

Dweomite Sep 4, 2024, 7:09 PM
1 point
1
in reply to: mako yass’s comment on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
Speaking as a developer, I would rather have a complete worked-out example as a baseline for my modifications than a box of loose parts.
I do not think that the designer mindset of unilaterally specifying neutral rules to provide a good experience for all players is especially similar to the negotiator mindset of trying to make the deal that will score you the most points.
I haven’t played Optimal Weave yet, but my player model predicts that a nontrivial fraction of players are going to try to trick each other during their first game. Also I don’t think any hidden info or trickery is required in order for rule disagreements to become an issue.

Dweomite Sep 3, 2024, 7:47 PM
1 point
0
in reply to: mako yass’s comment on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
then when they go to a meetup or a con, anyone they meet will have a different version
No, that would actually be wonderful. We can learn from each other and compile our best findings.
That’s...not the strategy I would choose for playtesting multiple versions of a game. Consider:
- Testers aren’t familiar with the mainline version and don’t know how their version differs from it, so can’t explain what their test condition is or how their results differ
- You don’t know how their version differs either, or even whether it differs, except by getting them to teach you their full rules.
- There’s a high risk they will accidentally leave out important details of the rules—even professional rulebooks often have issues, and that’s not what you’ll be getting. So interpreting whatever feedback you get will be a significant issue.
- You can’t guarantee that any particular version gets tested
- You can’t exclude variants that you believe are not worth testing
- You can’t control how much testing is devoted to each version
- Many players may invent bad rules and then blame their bad experience on your game, or simply refuse to play at all if you’re going to force them to invent rules, so you end up with a smaller and less-appreciative playerbase overall
The only real advantage I see to this strategy is that it may result in substantially more testers than asking for volunteers. But it accomplishes that by functionally deceiving your players about the fact that they’re testing variants, which isn’t a policy I endorse, either on moral or pragmatic grounds.
Most of the people that you’ve tricked into testing for you will never actually deliver any benefits to you. Even among volunteers, only a small percentage of playtesters actually deliver notable feedback (perhaps a tenth, depending on how you recruit). Among people who wouldn’t have volunteered, I imagine the percentage will be much lower.
[failed line of thought, don’t read]
Maybe limit it to bringing 1 thing with you? But notice this permits “stealing” items from other players, since “being carried” is not a persistent state.
“longer descriptions of the abilities”
I’d like that. That would be a good additional manual page, mostly generated.
If you’re imagine having a computer program generate this, I’m not sure how that could work. The purpose is not merely to be verbose, but to act as a FAQ for each specific ability, hopefully providing a direct answer whatever question prompted them to look that ability up.
If you aren’t familiar with this practice, maybe take a look at the Dominion rulebook as an example.

Dweomite Sep 3, 2024, 4:33 AM
5 points
3
in reply to: mako yass’s comment on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
there were a lot of things I didn’t want to be prescriptive about, and I figured they could guess a lot of it as they approached their own understanding of how the game should be and how things fit together, and I want to encourage people to fully own their understanding of the reasons for the rules.
I think this is a bad idea. Games are complex machines with many interlocking parts and they trade off between many goals; even an experienced developer can’t generally fill in gaps and expect things to work on the first try. This goes double for you because you are deliberately making an unusual game. Asking people to invent some of the rules is placing a pretty significant burden on them.
And if the designer fails to supply at least one example of a good rule, this makes me question whether they ever successfully found a good option themselves, and whether any possible rule could be filled in that would lead to a good result. (Imagine a blueprint for a perpetual motion machine with a box labeled “you can put whatever you want here.”)
Additionally, an interoperable standard makes it much easier for people to play with strangers. If every friend-group invents their own version, then when they go to a meetup or a con, anyone they meet will have a different version. An official version makes it easier to join a new group.
That’s without even getting into the psychological hangups people have about official rules, which are considerable. I think board games tap into human instincts about societal and ethical rules, and human instincts would like to pretend that there really is a correct rule somewhere in the Platonic realm, and maybe it’s unclear but it can’t just be missing and it’s certainly not up for grabs. I’ve seen people get pretty upset at the suggestion that there is no “right” rule for something because the designer simply never considered it, or that the version 2 of the rules is in some sense “a different game” from version 1. My pet theory is that this is an evolved security measure to not let cheaters escape punishment merely by claiming that there’s a problem with the formal legal code (regardless of whether that claim is correct).
I do not think you need to worry about blocking people from making up their own variations. My impression is that most hobbyist board gamers who would like to have house rules will go ahead use them no matter what you say on the matter. (This won’t stop people from clamoring for you to canonize their preferred rule, but I don’t think you can stop that by leaving rules unspecified, either.)
If you insist on doing this anyway, you should at least be clear about it in the document itself. You are bucking the cultural expectations of the board game hobbyist community, which expect games to be fully defined when we get them. People are not likely to infer that you intend them to make up their own details if you don’t spell that out.
I’m surprised you wouldn’t just assume I meant one space, given a lack of further details.
That was my highest-probability guess.
I do not like rulebooks that require me to guess what the rules are.
I also picked examples based on how glaring the absence of a rule seemed (and therefore how much evidence it gave that I was reading the wrong document), rather than based on how hard it was to guess the most likely answer. If I was more focused on omissions that are hard to guess, I might have asked “can 2 pawns occupy the same space at the same time?” instead.
Maybe I should move the mention of object pickup to the hunger card as well
If you intend to use this on more than one ability, I think it’s probably good to have object pickup rules in the rulebook, but there should be enough context that readers can slot the rule into their mental model of the game while reading the rulebook instead of leaving a dangling pointer that will only be resolved when they see the right card.
“Some abilities add objects to the map. Objects in your space can be picked up or dropped for free during your turn...”
You also probably need several more rules about objects, e.g. “Objects are carried by a particular pawn, and cannot teleport between the two pawns controlled by the same player. Stack the objects beneath the pawn to show they are being carried. Dropped objects remain in their space until someone picks them up. You can carry an unlimited number of objects at once. You can’t directly take an object that someone else is carrying.”
Also, your print-and-play files do not appear to include any components to represent objects. If you are expecting people to supply their own components for this, that ought to be called out in the rules (and you should say something about how many you need, of how many different kinds, etc.) In fact, it is standard practice to include a component list for a game—people usually skim right past it until suddenly they need it because they suspect they’re missing a piece or something.
(Though if this level of effort seems excessive for how often objects are actually used in the game, that might be a sign you should either use it more or get rid of it. Sometimes there’s a really cool ability that’s just not worth the complexity it introduces.)
This one gives me anguish. I don’t think formally defining nearby somewhere would make a better experience for most people and I also don’t want to say “on or adjacent to” 100 times.
Having a short description that will probably give people the right impression is great, but I think lots of players benefit from having some document they can check to resolve a confusion or disagreement if it comes up. I can’t remember a time I’ve played a game that had a glossary or a “longer descriptions of the abilities” section where I didn’t end up looking at least one thing up, and I can remember a lot of times when I wanted one and it wasn’t present.
Also, maybe try “within 1”. (Someone will still ask whether this means “on or adjacent” or “in the exact same space”, but I’d expect fewer people will need to ask compared to “nearby”.)

Dweomite Sep 2, 2024, 7:59 PM
7 points
2
on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
I read the “manual” page here, but I feel like it’s primarily focused on the philosophy of the game and hasn’t given a complete account of the rules. I also downloaded the print-and-play files but they seem to only contain cards/tiles and no rules document. Is there some other rules document that I’ve missed?
In case my response seems confusing, a few examples of why this doesn’t seem like the full rules:
- It says you can move on your turn, but doesn’t specify where you’re allowed to move to (anywhere? adjacent spaces? in a straight line like a rook?)
- It says you can pick up and drop objects on your turn, but “objects” are not mentioned anywhere else on the page and I can’t figure out what this refers to
- Rules for contracts don’t specify when you can make them or what you can agree to (can you transfer points? can you nullify an earlier deal? can you make the same deal twice to double the penalty for breaking it?)
- There are no examples of play
- I looked at a few of the cards in the print-and-play files, and they use terms like “nearby” that appear to be intended as terms-of-art, but which aren’t defined anywhere

Dweomite Sep 1, 2024, 11:58 PM
3 points
0
in reply to: mako yass’s comment on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
nandeck is actually pretty scripting-oriented from what I recall

Dweomite Sep 1, 2024, 8:47 PM
5 points
0
on: Release: Optimal Weave (P1): A Prototype Cohabitive Game
I did a thing where instead of just drawing card svgs directly, I drew parts, and wrote code that glued the parts together in standard ways and generated card-shaped svgs.
FYI there are several OTS tools that will programmatically assemble card images based on spreadsheet data, so that you can change common elements (like layout, backgrounds, or icons) in one place and regenerate all cards automatically. Some are free. I think nandeck is the best-known one.
I’m not sure from your description if this is exactly what you’re doing, but if you haven’t looked into these, you may want to.
I do notice that no big popular board game that I’ve ever heard of decided to publish through this service I guess it’s just because they really do specialize in small volume print runs
Yes. Most commercial board games use a manufacturing process called offset printing, which has high fixed costs that render it impractical if you want less than ~1k copies. The Game Crafter is the best-known of several services specializing in small volume. My impression is that they are noticeably lower-quality and have much higher marginal costs, but the low fixed costs make them great for prototyping and for people who just want to make their game available without making a business out of it.
People complain about printing alignment at these services, but from what I’ve heard, the big commercial printers don’t actually give you any tighter guarantees regarding print alignment (IIRC 1/8″ is the standard). I think there are a few reasons that people have divergent impressions:
1. Professionals know more tricks to disguise alignment errors than amateurs do. For instance, in most commercial board games, you’ll find a thick black (or white) border around the front of every card, which you’ve probably never noticed because it fades into the background; amateurs often fail to replicate this trick.
2. It’s a stochastic process, and the biggest complaints are from a self-selected group with bad luck. (Also, maybe offset printing is better on average, even if the worst case is similar?)
3. I’ve been told that when it’s really important, big publishers will examine the print output and throw away the worst examples at their own cost.
I lack the experience to tell you which card-making tools or small-run print services are best, but send me a message if you’d like a longer list of examples that you could investigate for yourself.

Dweomite Aug 28, 2024, 4:49 AM
3 points
0
in reply to: Raemon’s comment on: Please stop using mediocre AI art in your posts
I preferred your v4 outputs for both prompts. They seem substantially more evocative of the subject matter than v6 while looking substantially better than v3, IMO.
(This was a pretty abstract prompt, though, which I imagine poses difficulty?)
I am not an artist.

Dweomite Aug 1, 2024, 5:39 AM
1 point
0
on: One-shot strategy games?
Recently encountered a game that made me think of this thread.
King of the Bridge is a game where you play an unfair chess variant with partially-secret rules and the opponent sometimes cheats. I reached the first ending in ~40 minutes. Getting there without ever losing a match seems like it ought to be possible with great paranoia plus some luck.
If you want to try this you should probably avoid looking at the store page in detail, as some of the secret rules are revealed in screenshots.

Dweomite Jul 18, 2024, 9:21 PM
6 points
0
in reply to: Noosphere89’s comment on: Optimistic Assumptions, Longterm Planning, and “Cope”
The maximum difficulty that is worth attempting depends on the stakes.

Dweomite Jul 18, 2024, 7:42 PM
3 points
0
in reply to: Noosphere89’s comment on: Optimistic Assumptions, Longterm Planning, and “Cope”
One of Eliezer’s essays in The Sequences is called Shut Up and Do the Impossible
I’m confident Eliezer would agree with you that if you can find a way to do something easier instead, you should absolutely do that. But he also argues that there is no guarantee that something easier exists; the universe isn’t constrained to only placing fair demands on you.

Dweomite Jul 18, 2024, 1:04 AM
1 point
0
in reply to: Raemon’s comment on: Optimistic Assumptions, Longterm Planning, and “Cope”
I haven’t played They Are Billions. I didn’t have that experience in Slay the Spire, but I’d played similar games before. I suppose my first roguelike deckbuilder did have some important combo-y stuff that I didn’t figure out right away, although that game was basically unwinnable on your first try for a bunch of different reasons.

Dweomite Jul 18, 2024, 12:51 AM
4 points
0
on: Optimistic Assumptions, Longterm Planning, and “Cope”
I’ll get 10 extra units of production, or damage. But, then I reach the next stage, and it turns out I really needed 100 extra units to survive.
I’m having a hard time thinking of any of my personal experiences that match this pattern, and would be interested to hear a couple examples.
(Though I can think of several experiences along the lines of “I got 10 points of armor, and then it turned out the next stage had a bunch of attacks that ignore armor, so armor fundamentally stops working as a strategy.” There are a lot of games where you just don’t have the necessary information on your first try.)

Dweomite Jul 18, 2024, 12:32 AM
11 points
4
in reply to: dirk’s comment on: Optimistic Assumptions, Longterm Planning, and “Cope”
Yes, your prior on the puzzle being actually unsolvable should be very low, but in almost all such situations, 70% seems way too high a probability to assign to your first guess at what you’ve misunderstood.
When I prove to myself that a puzzle is impossible (given my beliefs about the rules), that normally leads to a period of desperate scrambling where I try lots of random unlikely crap just-in-case, and it’s rare (<10%) that anything I try during that desperate scramble actually works, let alone the first thing.
In the “final” level of Baba Is You, I was stuck for a long time with a precise detailed plan that solved everything in the level except that there was one step in the middle of the form “and then I magically get past this obstacle, somehow, even though it looks impossible.” I spent hours trying to solve that one obstacle. When I eventually beat the level, of course, it was not by solving that obstacle—it was by switching to a radically different approach that solved several other key problems in entirely different ways. In hindsight, I feel like I should have abandoned that earlier plan much sooner than I did.
In mitigation: I feel that solutions in Baba Is You are significantly harder to intuit than in most puzzle games.