Just this guy, you know?
Dagon
Karma: 11,775
Those seem like pretty low bars for “popular and mainstream”.
This seems to be asking from the demand side (“we” being people with lots of money who want to hire trained people), but then switches to demand side (people being turned away looking for training and employment).
I think that’s a hint to your answer: other industries solve it by actually hiring lots of people, and offering training on the job or with regular programs. Oh, and usually waiting for equilibrium to catch up, which is not comfortable for rapid-change requirements.
It would perhaps clarify your question if you gave some examples of industries/topics that HAVE faced and solved the question, and we can focus on the “how”.
Nope! Parfit’s Hitchhiker is designed to show exactly this. A CDT agent will desperately wish for some way to actually commit to paying.
I think some of the confusion in this thread is what “CDT with precommittment (or really, commitment)” actually means. It doesn’t mean “intent” or “plan”. It means “force”—throw the steering wheel out the window, so there IS NO later decision. Note also that humans aren’t CDT agents, they’re some weird crap that you need to squint pretty hard to call “rational” at all.
I should have specified WHO they want to cooperate with in the future. People with lots of money to spend—businesses. Silence is far preferable to badmouthing former coworkers.
First rule of probability and decision theory: no infinities! If you want to postulate very large numbers, go ahead, but be prepared to deal with very tiny probabilities.
Pascal’s wager is a good example—the chance that the wager actually pays off based on this decision is infinitesimal (not zero, but small enough that I can’t really calculate with it), which makes it irrelevant how valuable it is. This gets even easier with the multitude of contradictory wagers on offer—“infinite value” from many different choices, only one of which can you take. Mostly, take the one(s) with lower value but actually believable conditional probability.
There will be no paperclips if planet and maximizer are destroyed.
There might be—some paperclips could survive a comet. More importantly, one paperclip’s worth of resources won’t change the chance of a comet collision by any measurable amount, so the choice is either “completely waste that energy” or “make a paperclip that might survive”.
Only if every other entity’s anti-paperclip stance is known and unchangeable, and if resource->impact is purely linear, can it be assumed that 100% to self-preservation (oh, wait, also to accumulation of power, there’s another balance to be found) is optimal. Neither of these are true, but especially the problem of declining marginal impact.
For any given energy unit decision you could make, there will be a different distribution of future worlds and their number of paperclips. Building one paperclip could EASILY increase the median and average number of future paperclips more than investing one paperclip’s worth of power into comet diversion.
It gets more difficult when coordinating with unaligned agents—one has to decide whether to nudge them toward valuing paperclips, convincing/forcing them to give you more power, or (since they’re unlikely to care as much as you about the glorious clippy future) point THEM at the comet problem so they reduce that risk AND don’t interfere with your paperclips.
If you haven’t played it (it was popular a few years ago in these circles, but I haven’t seen it mentioned recently), it’s worth a run through https://www.decisionproblem.com/paperclips/ . It’s mostly humorous, but based on some very good thinking.
Adversarial action makes this at least an order of magnitude worse. If Carla has to include the chance that Bob could be lying (for attention, for humor, or just pathologically) about his experiences or his history of drug use or hallucinations, she makes an even smaller update. This is especially difficult in group-membership or banning discussions, because LOTS of people lie (or just focus incorrectly on different evidence) for status and for irrelevant-beef reasons.
I don’t think there is a solution, other than to acknowledge that such decisions will always be a balance of false-positive and false-negative, and it’s REQUIRED to ban some innocent (-ish) people in order to protect against the likely-but-not-provably-harmful.
“legal reasons” is pretty vague. With billions of dollars at stake, it seems like public statements can be used against them more than it helps them, should things come down to lawsuits. It’s also the case that board members are people, and want to maintain their ability to work and have influence in future endeavors, so want to be seen as systemic cooperators.
It’s easy to frame Newcomb’s problem such that there’s no opportunity to precommit (and, CDT generally doesn’t see any REASON to precommit there).
Trying to figure out how to update. From the downvotes and comments, I’m clearly considered wrong, but I can’t easily find details on how. Is the statement “We have not and never will take away vested equity” a flat-out lie? I’d expected it was relying heavily on the word “vested”, and what they took away was something non-vested.
Is there a simple link to a specific legal description of what assets a non-signer was entitled to, but lost due to declining to sign?
Edit: Zvi recently linked to OpenAI NDAs: Leaked documents reveal aggressive tactics toward former employees—Vox, which does have pretty compelling references that my assumptions were wrong, that the denial was a verifiably false statement, and they did, in fact, credibly threaten to take back vested equity. I’ve checked my equity in past (private, so not exercisable unless they have a liquidity event) and current (public, so exercisable immediately on vest) employers, and this doesn’t seem possible for them. OpenAI is an outlier in defining their equity that way (such that “vested” is contingent).
I think most people have a different conception of what “trying to be clear” means than I do. They can talk abut true love or what’s “really” love, but the very few who actually try to get an operational definition across usually start by tabooing “love” when trying to communicate specific beliefs and experiences (though still using it in more casual, romantic, or poetic settings).
I recommend this as a technique when trying to draw out clarity from someone as well. Don’t make them define terms, most people aren’t great at the activity of generalizing and re-specifying that language is built on. Instead, ask them do describe what they mean in this context, by asking for statements that do not use “love”.
Note: this rarely leads to kissing. If that’s your goal, I’d advise to delay semantic exploration for another time.
I haven’t followed closely—from outside, it seems like pretty standard big-growth-tech behavior. One thing to keep in mind is that “vested equity” is pretty inviolable. These are grants that have been fully earned and delivered to the employee, and are theirs forever. It’s the “unvested” or “semi-vested” equity that’s usually in question—these are shares that are conditionally promised to employees, which will vest at some specified time or event—usually some combination of time in good standing and liquidity events (for a non-public company).
It’s quite possible (and VERY common) that employees who leave are offered “accelerated vesting” on some of their granted-but-not-vested shares in exchange for signing agreements and making things easy for the company. I don’t know if that’s what OpenAI is doing, but I’d be shocked if they somehow took away any vested shares from departing employees.
It would be pretty sketchy to consider unvested grants to be part of one’s net worth—certainly banks won’t lend on it. Vested shared are just shares, they’re yours like any other asset.
Most people aren’t confused, because they’re not trying to be clear and rational. They are definitely confusing to people who prefer specificity and operationally useful descriptions of experiences.
It is used to mean a very wide range of positive feelings, and should generally be taken as poetry rather than communication. Another framing is that the ambiguity makes it very useful for signaling affection and value, without being particularly binding in specific promises.
Which is NOT to say it isn’t “real”. Affection, caring, enjoyment, and willingness-to-sacrifice-for someone are all things that many individuals experience for other individuals. The exact thresholds and mix of qualia that gets described as “love” varies widely, but the underlying feelings seem near-universal.
[no actual direct knowledge]
Yeah, all things like this follow a power distribution—a few users use it A TON and both median and average are far smaller than you’d think. Assuming they throttle the biggest users in some way, they’re making some money on paid use. But it’s probably irrelevant to their overall strategy and profitability.
I’d guess the biggest value for them is in showing growth and “willingness to pay” for this, more than the actual money collected.
The other common optimization process people generally refer to is “markets”. In the same sense that “evolution” is what happens when variation and selection combine, “market” is what happens when multiple traders repeatedly choose how to exchange things.
If it’s possible for super-intelligent AI to be non-sentient, wouldn’t it be possible for insects to evolve non-sentient intelligence as well? I guess I didn’t assume “non-sentient” in the definition of “unaligned”.
It’s always seemed strange to me what preferences people have for things well outside their own individual experiences, or at least outside their sympathized experiences of beings they consider similar to themselves.
Why would one particularly prefer unthinking terrestrial biology (moss, bugs, etc.) over actual thinking being(s) like a super-AI? It’s not like bacteria are any more aligned than this hypothetical destroyer.
Public discourse norms, especially in the twitter age, are funny. Hanania has a lot of options, none of which really change his status much.
he can ignore you. There’s enough volume that he doesn’t respond to most comments, so this isn’t him dodging a particularly harsh criticism, it’s just not worth his notice.
he can answer cheaply, by rephrasing (or just reposting) previous texts.
he can answer a different question that’s vaguely related.
All models are wrong, some models are useful.
I think this is wrong. Replace “have/has” with “can be modeled as a”. We don’t know if that structure is causal and singular to that output. I think recognizing that inferred structures are not exclusive or complete is useful here—many possible structures can generate very similar outputs. If your goal is to predict future outputs, you probably need to find the actual generator, not pick a structure that could be it.
No actual agents will be exactly any of those structures, it’ll be a mix of them, and some other less-legible ones. I’d probably also argue that ANY of those structures can be scary, and in fact the utility-optimizing agent can use ANY of the decision types (table, rule, or model), because they’re all equivalent, with sufficient complexity of {table, ruleset, or world-model).