If I make a comment, then the author deletes the post, is my comment lost as well? I’m pretty sure I don’t actually need a record of things I’ve said, and that my comments aren’t valuable enough to justify extra effort, but it kind of bugs me.
Has anyone written the scraper or script to save a copy of everything you write on LW, so it’s available even if it gets deleted later? If it archived the post and comment thread you’re responding to, it would be all right.
Not on my profile page, as far as I’ve been able to tell. Not important enough to dig around GW. I think I’ll start just saving local copies of some of my comments, and decide whether to turn them into a shortform if I notice they’re deleted.
Would it be simple for the system to send me a PM with the contents when a comment is deleted by a post author (either directly or by deleting the post that contains it)?
In addition to being able to find your own recent comments, another issue is links to comments dying. For example if I were to link to this comment, I would worry it might quietly disappear at some point.
I love markdown when it works, and it drives me nuts when it doesn’t. All the docs and cheat sheets in the world don’t help.
Would it be possible to add a “view source” option for all posts and comments (not just mine), so I learn from others rather than re-discovering my own mistakes over and over?
I wish I could agree-and-downvote (or agree-and-not-vote) on posts. There are a fair few posts that are well-reasoned and probably correct, but just not important or interesting.
Yes, and the converse: posts that are interesting and provide useful information, but have fundamentally wrong arguments and derive invalid conclusions.
I wish people would be more careful about reinforcing attribution error. We all (including me) have a tendency to talk about types of people rather than behaviors.
I wish we talked about mistake theory rather than mistake theorists. I wish we talked about ideas or discussions at a given simulacra level, rather than people operating at that level. Recently, I wish it were a discussion about “empathetic oofs” rather than “empathetic oofers”.
I find it offputting, and it makes it harder for me to accept the underlying thesis, when such behaviors are attributed to immutable personality types. More importantly, I think it constrains our analysis very severely not to recognize that these are valid (or at least common) strategies, and we should strive to use both when appropriate.
I’m a bit uncomfortable with [edit: wrong word. all discussion is good, but I think there’s a big modeling error being made.] a lot of the discussion about ethics/morals around here, in that it’s often treated as a separate, incomparable domain to other motives/values/utility sources.
It may be, in humans and/or in AI, that there are fairly distinct modules for different domains, which somehow bid/negotiate for which of them influences a given decision by how much. This would make the “morals as distinct evaluations” approach reasonable, though the actual negotiation and power weights among modules seem more important to study.
But if something approaches VNM-rationality, it acts as if it has a unified consistent utility function. Which means an integrated set of values and motivations, not separate models of “what’s the right action in this context for me”.
So, either AI is irrational (like humans), or exploration of morality as a separate topic from status, pleasure, family, or other non-altruistic drives is not very applicable.
It is not obvious that vnm rationality is quite the right thing here. I talked to a game theorist dude at an event a while back and he said of the core assumptions of vnm rationality there’s one not everyone accepts, but I’ve had trouble re-locating what it was he said. In any case, garrabrant said he doesn’t accept vnm rationality for related but iirc slightly-different-maybe-not-sure reasons. But iirc it doesn’t completely invalidate combining everything into a single ranking function.
It might make sense to demand that morality is clearly spelled out in one of the theorems that constrain what utility functions we’d accept, though. That’s a little different from it being truly separate.
I concede that VNM may be too high a bar, and I’ve seen some clever things done by relaxing the independence axiom. I don’t think I’ve seen any that change the basic problem of (acting as if there were) a single value function that the agent is optimizing. To be rational, there can be no separation of moral and non-moral domains.
It’s fascinating to me to see people changing their recommended meta-ethics due to a very public bad actor and related financial meltdown. How does the FTX meltdown and fraud change your fundamental priorities OR your mechanisms for evaluating actions within those priorities?
Sure, adding some humility to the evaluation function is wise. Recognize that things probably aren’t as linear as you think (and certainly not as precise), when you get toward very large or small numbers. Update to focus on nearer-term, safer improvements. And reducing your trust (for those of you who had it) in the assumption that rich people who say comforting words are actually aligned in any useful way is … so obvious that I’m sad to have to say it.
But if you have believed for years that you can aggregate and compare future population utility against current experiences, and predict what actions will have large impacts on that future, it’s really odd if you give that up based on things that happened recently.
[ note: I am not utilitarian, but I am consequentialist. I never made the assumption that SBF was a pure (or even net) good, nor that his methods were likely to sustain. But even if I were surprised by this, I don’t see how it’s new evidence about something so fundamental. ]
I feel like FTX is a point against utilitarianism for the same reasons Bentham is a point for utilitarianism. If you take an ethical system to logical conclusions and anticipate feminism, animal rights, etc. this is evidence for a certain algorithm creating good in practice. If you commit massive fraud this is evidence against.
This also doesn’t shift my meta-ethics much, so maybe I’m not one of the people you’re talking about?
EA folks: is there any donation I can make to increase the expected number of COVID-19 vaccinations given by end of February, either worldwide or for a specific region?
Increasing vaccine production capacity takes longer then two months. The only way you can increase vaccinations in the time frame is reducing the amount of vaccine that’s given per patient or approve more vaccines.
India’s approval of a vaccine without phase 3 trials would be the kind of action that you would want to encourage. 1 dosing instead of 2 dosing is an action that works in the developed world.
Running PR for such actions would be the only viable move. I’m however not aware of individual charities that would use marginal donations for that purpose.
Reminder to self: when posting on utiltarianism-related topics, include the disclaimer that I am a consequentialist but not a utilitarian. I don’t believe there is an objective, or even outside-the-individual perspective to valuation or population aggregation.
Value is relative, and the evaluation of a universe-state can and will be different for different agents. There is no non-indexical utility, and each agent models the value of other agents’ preferences idiosyncratically.
Strong anti-realism here. And yet it’s fun to play with math and devise systems that mostly match my personal intuitions, so I can’t stay away from those topics.
I think I agree that there’s no objectively true universal value aggregation process, but if you can’t find (a method for finding) very broad value aggregation system, then you can’t have peace. You have peace insofar as factions accept the rulings of the system. Simply giving up on the utilitarian project is not really an option.
That’s an interesting reason to seek Utilitarianism that I hadn’t considered. Not “this is true” or “this makes correct recommendations”, but “if it works, it’ll be a better world”. I see some elements of Pascal’s Wager in there, but it’s a novel enough (to me) approach that I need to think more on it.
I do have to point out that perhaps “you can’t have peace” is the actual true result of individual experience. You can still have long periods of semi-peace, when a coalition is strong enough to convince others to follow their regime of law (Pax Romana, or many of today’s nations). But there’s still a lot of individual disagreement and competition for resources, and some individuals use power within the system to allocate resources in ways that other individuals disprefer.
I’m not sure if that is “peace” within your definition. If so, Utilitarian aggregation isn’t necessary—we have peace today without a working Utility calculation. If not, there’s no evidence that it’s possible. It may still be a worthwhile goal to find out.
Shameless self-promotion but I think meta-preference utilitarianism is a way to aggregate value fairly without giving up anti-realism. Also, be sure to check out the comment by Lukas_Gloor as he goes into more depth about the implications of the theory.
Well, that’s one way to lean into the “this isn’t justified by fundamentals or truth, but if people DID agree to it, things would be nicer than today (and it doesn’t violate preferences that I think should be common)”. But I’m not sure I buy that approach. Nor do I know how I’d decide between that and any other religion as a desirable coordination mechanism.
I’m not sure what flavor of moral anti-realism you subscribe to so I can’t really help you there, but I am currently writing a post which aims to destroy a particular subsection of anti-realism. My hope is that if I can destroy enough of meta-ethics, we can hopefully eventually make some sort of justified ethical system. But meta-ethics is hard and even worse, it’s boring. So I’ll probably fail.
Good luck! I look forward to seeing it, and I applaud any efforts in that direction, even as I agree that it’s likely to fail :)
That’s a pretty good summary of my relationship to Utilitarianism: I don’t believe it, but I do applaud it and I prefer most of it’s recommendations to a more nihilistic and purer theory.
There’s a large class of social/cultural behaviors that I think are fine and good when they’re somewhat rare, and I find annoying and then harmful when it becomes very common.
This includes public demonstrations, minor mostly-victimless crimes, many kinds of drug use, tipping expectations, and I’ve just realized that April Fools is now in this category.
It’s amazing when there are a few surprising well-crafted jokes published. It’s annoying when it’s common enough that it’s hard to find the good ones, and even harder to find the non-joke communications.
The problem with this (as with all examples of the phenomenon—things that are justified and good when they’re somewhat rare and intentionally transgressive or risky, but are annoying and harmful when safe and common) is that I really LIKE the gems, and want to (occasionally) partake myself.
It’s a matter of variable and contextual value (generally declining on the margin), and I don’t know any good answers.
Zvi has been writing about topics inspired by the book Moral Mazes, focused on a subset of large corporations where politics rather than productivity are seen as the primary competition dimension and life focus for participants. It’s not clear how universal this is, nor what size and competitive thresholds might trigger it. This is just a list of other Maze-like situations that may share some attributes with those, and may have some shared causes, and I hope solutions.
Consumerism and fashion.
Social status signaling and competition.
Dating, to the extent you are thinking of eligibility pools and quality of partner, rather than individual high-dimensional compatibility.
Acutal politics (elected and appointed-by-elected office).
Academic publishing, tenure, chairs.
Law enforcement. Getting ahead in a police force, prison system, or judiciary (including prosecutors) is almost unrelated to actually reducing crime or improving the world.
Bureaucracies are a related, but different kind of maze. There, it’s not about getting ahead, it’s about … what? Shares the feeling that it’s not primarily focused on positive world impact, but maybe not other aspects of the pathology.
Expand on that a bit—is this an employment/social maze for lawyers and legislators, or a bureaucratic maze for regular citizens, or something else? I’ll definitely add policing (a maze for legal enforcers in that getting ahead is almost unrelated to stopping crime).
Employment and social maze for both lawyers and legislators, and sometimes judges as well a police (as you mention). For lawyers perhaps there is some asymmetry between DA side and Defendant in criminal cases, not sure there. Examples for DA and police are the incentives to get convictions and so the image of making things safe and producing justice leading to some pretty bad cases of bad behavior that amounts to either framing and innocent in some cases, getting a conviction by illegal means (slippery slope situations) which then undermine the entire legal structure/culture of a rule of law society.
Similar for legislators—election results tend to drive behavior with lots of wiggle room for not actually working on delivering campaign promises or not ever actually having intended to push the advertised agenda. The incentives of being a tenured politician, particularly at the federal and state level, related to personal benefit seem poorly aligned with serving the electorate (representing constituents values rather than maximizing the politician’s income, status and the like.
There might be some dynamic related to rent-seeking results too. If rent-seeking is purely a structural reality (standard organizational costs facing special versus general interests) that should stabilize until the underlying costs structures change. But I’m not entirely sure that is the case, if we can make the cases that rent-seeking is growing then perhaps a moral maze would provide a better explanation of that growth where no shift in underlying cost are able to explain the growth.
For the regular citizen certainly the bureaucracy creates problems both for engaging with government as well as perhaps engaging with politics and elections—though I’m less sure this is a real maze problem.
My daily karma tracker showed that 3 comments got downvoted. Honestly, probably justified—they were pretty low-value. No worries, but it got me thinking:
Can I find out how I’m voting (how many positive and negative karma I’ve given) over time periods? Can I find it out for others? I’d love it if I could distinguish a downvote from someone who rarely downvotes from one from a serial downvoter.
I’ve been downvoting less, recently, having realized how little signal is there, and how discouraging even small amounts can be. Silence or comment are the better response, for anything better than pure drivel.
Worth noting that by default the karma notifier doesn’t show downvotes, for basically this reason.
I think it only makes sense to opt into seeing all downvotes if you’re confident you can take this as information without it harming your motivation.
Meanwhile, there are benefits to actually downvoting things that are wrong, or getting undue attention, or subtly making the site worse. It matters how things can sorted for attention allocation, and relative karma scores in a discussion are important, among other things, so that people don’t see bad arguments appearing highly endorsed and then feeling motivated to argue against them (when they wouldn’t ordinarily consider this a good use of their time)
So I think people should actually just downvote more, and not use the “show me all my downvotes” feature, which most people aren’t using and you probably shouldn’t worry about overmuch.
I opted in to seeing downvotes, and I think my curiosity (both about personal interactions on the site and about the general topic of mechanism design) will compel me to continue. I’m not worried about them—as I say, they were low-value comments, though not much different from others that get upvoted more (and IMO shouldn’t be). Mostly it shows that low-number-of-voters is mostly noise, and that’s fine.
My main point was wondering how I can monitor my own voting behavior? (ok, and a little complaining that variable-point voting is confusing.) I think I upvote pretty liberally and downvote very rarely, but I really wonder if that’s true, and I wonder things like how many votes I give (on multiple comments) on the same post.
I do agree that being able to see metrics on your own voting is pretty reasonable, although not something I expect to make it to the top of our priority queue for awhile.
Should I assign a significant chance that the US will actually default on some debts? Betting markets are at 6%, but they’re pretty bad at the tails, for various reasons. CDS spreads (people buying insurance for/against a default) are 172bp, implying a 1.72% chance with a fair bit of real money wagered. It’s also likely somewhat skewed, for regulatory and technical reasons, but less than the betting markets.
This wouldn’t be an existential threat, but it’s definitely a stability and lifestyle threat for a lot of current and future humans, and likely exacerbates other threats.
Not sure I can do much about it, but it’s on my mind.
Here’s a prediction I haven’t seen anywhere else: due to border closures from COVID-19 and Brexit, the value of dual-citizenship is going up (or at least being made visible). This will lead to an uptick in mixed-nationality marriages.
“One of the big lessons of market design is that participants have big strategy sets, so that many kinds of rules can be bent without being broken. That is, there are lots of unanticipated behaviors that may be undesirable, but it’s hard to write rules that cover all of them. (It’s not only contracts that are incomplete...) ”—Al Roth
I think this summarizes my concerns with some of the recent discussions of rules and norm enforcement. People are complicated and the decision space is much larger than usually envisioned when talking about any specific rule, or with rules in general. Almost all interactions have some amount of adversarial (or at least not-fully-aligned) beliefs or goals, which is why we need the rules in the first place.
A general problem with optimization and control (in AI and commerce and everything else) is that it limits breadth of activity in (at least) two ways. The obvious way is that it avoids things which oppose the stated goals. That’s arguably a good and intentional limit.
But it ALSO avoids things that are illegible or unexpected and hard to quantify. I suspect that’s a large majority of experiential value for a large majority of humans.
I don’t know if this can be made rigorous enough to generate a ‘Friendliness Impossibility Theorem’, but it’s probably the thing that keeps me up at night most often about the future of conscious experience in the universe.
The sketch of that argument is that for any given level of conscious compute power (agentic and/or moral patient), MOST of the calculations will be illegible to that entity, and to entities of similar power. The only way to have provable/fully-trustworthy agreement on values is to have full knowledge of all other entities.
It’s possible that there’s a corollary that things CAN be provably friendly only if a majority of total compute power in conscious entities is used for monitoring and understanding each other, rather than for understanding and optimizing the rest of the universe. If that’s the case, there’s a very unfortunate value loss (to me and many current humans) in illegibility and surprise experiences.
tl;dr I like (some amount of hard-to-identify) chaos and surprises, and I don’t know how to square that with the fact that I don’t trust people or future AIs.
Not losing yourself in the long term is a challenge even without others implanting their influence. Brains are not built to avoid value drift for millions of years, learning is change that has no safety or reflective stability guarantees. So getting some sort of intelligent substrate that understands you or a change of cognitive architecture that lets you understand yourself is a relatively urgent need.
But if everyone is protected from external influence in depth as a result (including superintelligent influence), there is no need to guarantee safety of behavior, let alone cognition. Visible problems can always be solved with backups, it’s value drift of global power that would be irreversible.
I’ve lost myself multiple times over, even in my insanely brief (well over half of human expectation, but still massively inadequate) memory. I give a high probability that my personal experiential sequence will end and be lost forever fairly soon. That’s not my main concern here.
My concern is in understanding what I {do|should} care about beyond my own direct experiences. It’s about extrapolating the things I value in my own life to future trillions (or more, eventually) of experiencing entities. Or to a small number of extremely-experiencing entities, maybe—I don’t know what elements are important to my decisions of how to influence the future. I seem to care about “interesting” experiences more than specifically pleasant or painful or difficult ones. And I don’t know how to reconcile that with the goals often stated or implied around here.
My point is that there is motivation to solve the problem of value drift and manipulation both for people and global power, on the defense side. The thing to preserve/anchor could be anything that shouldn’t be lost, not necessarily conventional referents of concern. The conservative choice is trying to preserve everything except ignorance. But even vague preference about experience of others is a direction of influence that could persevere, indifferent to its own form.
If the problem is solved, there is less motivation to keep behavior or cognition predictable to make externalities safe for unprotected others, as there are no unprotected others to worry about.
I absolutely understand that there’s motivation to “solve the problem of value drift and manipulation”. I suspect that the problem is literally unsolvable, and I should be more agnostic about distant values than I seem to be. I’m trying on the idea of just hoping that there are intelligent/experiencing/acting agents for a long long time, regardless of what form or preferences those agents have.
It’s worth remembering, in discussions about decision theory, deontology vs consequentialism, and related topics, that ALL of our actual examples of agents are MISTAKEN or DECEPTIVE about the causes of their actions.
I give some probability space to being a Boltzmann-like simulation. It’s possible that I exist only for an instant, experience one quantum of input/output, and then am destroyed (presumably after the extra-universal simulators have measured something about the simulation).
This is the most minimal form of Solipsism that I have been configured to conceive. It’s also a fun variation of MWI (though not actually connected logically) if it’s the case that the simulators are running multiple parallel copies of any given instant, with slightly different configurations and inputs.
Georgism isn’t taxation, it’s public property ownership.
It finally clicked in my mind why I’ve reacted negatively to the discussion about Georgist land taxes. It’s the same reaction I have to ANY 100% tax rate on anything. That’s not what “ownership” means.
Honestly, I’d probably be OK with (or at least I’d be able to discuss it more reasonably) the idea that land is not privately owned, ever. Only rented/leased from the government, revocable for failure to pay the current rate (equivalent in all ways to the Georgist tax amount, and penalties for failure to pay). I suspect it’s unworkable for the same reasons—the public won’t stand for evicting sympathetic tenants just because they moved in when the rent/tax was lower.
Further, taxation of economic movement or realized profit is pretty easy to justify, and ALWAYS exists. Taxation of durable-ownership valuation is tricky, as there is disagreement over whether it’s “real”, and there’s no guarantee that the investment value, even if correctly assessed, is liquid enough to actually pay the tax.
Edit: this had occurred to me before, but I forgot to put it here. It also suffers from the socialist calculation problem. When self-interested humans aren’t making decisions to maximize value (to themselves) about a thing, the actual most valuable use is very difficult to determine. Holding someone responsible for the theoretical value of something when they don’t actually think it’s the best is really difficult to envision.
Further edit: thanks for the comments—it’s helped me understand my true objection, in addition to the auxiliary problems that often come to mind. I think there are two insurmountable problems, not necessarily in this order:
Taxing anything at 100% is disingenuous about the word “ownership”. It should be called “rent” or something else that is clear that it’s not owned, but merely usable for some purposes until you decline to pay a future arbitrary amount. It would also make clear that you’re also undertaking all of the political-economy influences that make government setting of rent amounts problematic.
Taxing non-realized value is always going to be problematic, both for calculation reasons (valuation is theoretical and debatable), and for the fragility of the value. I think this generalizes to “only collect taxes in-kind (or on liquid enough things that it can be easily converted at tax-collection time)”. Taxing a pile of wheat is easy—take some percentage of it. Taxing a monetary transaction likewise—the price increases and the tax is paid out of that. Taxing a future value is not (generally) feasible—either you’ve just taken someone’s seeds so they can’t plant the crop you’re trying to tax, or you’re demanding cash from someone whose investments will suffer from early liquidation. Deferral of payment can help with this, but if you’re doing that, you should just tax the realized gains at time of liquidity.
the public won’t stand for evicting sympathetic tenants just because they moved in when the rent/tax was lower.
We would really need the technology where you can take a house and move it to a different place. That would also make Georgism easier—when people don’t pay, you simply move them to a cheaper location.
While I think that would be extremely valuable technology, exiling Grandma to Outer Nowhereville, hours away from the rest of the family, is probably not a lot more feasible.
Alternatively, what if the LVT only increased when the property was sold? Buy land, and you lock in the LVT at the point in time when your deal went through.
I guess people would never sell the real estate at good location, only lease it? Thus capturing the increased value of the land without ever paying the tax.
Maybe leases are subject to increased LVT, just not owning it for personal use? I suspect there’s a way to enact something like this, but this isn’t in my wheelhouse.
Rich people and businesses are more strongly motivated and able to find loopholes than government administrators are to design robust mechanisms. Modern fractional ownership and lease arrangements are complicated enough that most any scheme based on a change in contract participants is going to be arbitraged away very quickly.
Right, but I thought the point was to avoid people balking at a law that makes it easier to evict ordinary folks who’ve lived somewhere a long time and don’t want to move? Let the rich people find their loopholes—as long as people support the system and the LVT delivers most of its promise, that’s good enough.
My point is that whatever exclusions are in place to keep poor people in their newly-valuable homes will be used by rich people to avoid taxation. There is no way to avoid this without making the tax variable based on other factors than the land value (like the “owner’s” perceived financial situation).
Either the law is the same for all “pay or GTFO”, or it’s differentially enforced, so the rich and the politically-sympathetic pay less than the middle-class.
It sounds like you think there’s no practical way to introduce exceptions to make an LVT politically palatable without also breaking its ability to motivate efficient land use and generate tax revenue.
My first thought is sure, we have a lot of tax evasion now. But we also do generate lots of tax revenue, most tax revenue comes from the very rich. Why would an LVT be so much more prone to tax evasion than the current system?
Correct, I think there’s no practical or realpolitik-theoretical way to make it work. To the extent that you make exceptions in implementation, it’s no longer a land-value tax, it’s a land-value fig leaf over a “what we can get away with” tax.
to the extent that you make exceptions in implementation, it’s no longer a land-value tax
This is true the same way that to the extent you dilute your cocoa with water, it’s no longer hot chocolate, it’s water :) I don’t know of almost any “pure” real world policies or taxes. I’d need some good evidence to believe that the LVT is that fragile.
I sometimes think about Neanderthal tribes, and what they thought about alignment with Homo Sapiens (in truth, they didn’t; there wasn’t that much interaction and the takeover was over a longer timeframe than any individual of either species could imagine. but I still think about it).
I wonder if we could learn anything from doing a post-mortem (literally) from their viewpoint, to identify anything they could have done differently to make coexistence or long-term dominance over the newer species more likely.
We should probably generalize cryptocurrency mechanisms to “proof of waste”. Fundamentally, the concept of a blockchain is that real-world resources are used for artifically-complex calculations just to show that one doesn’t need those resources for something else.
Interesting! What cryptocurrencies or blockchains are there which are based on something other than proof of {some resource use}?
There are certainly non-crypt-based currencies and centralized data authorities that simply use fiat and access bottlenecks. I’ve only come across distributed authority mechanisms based on usage of some expensive and limited resource.
Cardano and Polkadot both use proof of stake. They are more like proof of resource ownership instead of proof of resource use. Resource is however a bit misleading here given that there are no resources in the classical sense needed to create the coins that are used for the staking in the first place. The coins are created out of nothing. Saying that minting coins out of nothing is creating resources is like saying that the FED printing money creates more resources for the economy.
Besides reducing electricity consumption that has additional advantages such as increase the amount of transactions that can be processed by two orders of magnitude (and additional orders of magnitude if you count side-chains). It also makes the process of validating blocks faster.
Filecoin is also noteworthy in that it uses proof-of-spacetime where miners do valuable work of storing data instead of wasteful work.
I was aware of the proof of spacetime as an alternative to the normal proof-of-computation chains. Unused (for other purposes) disk space definitely falls into the category of “resource”, as much as unused (for other purposes) compute capacity.
A quick glance at polkadot and cardano didn’t jump out as to what limits their tokens and what a “stake” actually is. Thanks for pointing them out, though—I have more to learn :)
In the case of Filecoin it’s not space that’s unused for other purposes. It’s space that’s used to store files that users (that are verified clients) of filecoin want to store.
The basic idea of proof-of-stake is that when it comes to validating a new block, one of the users that stake coins gets randomly chosen weighted by the amount of staked coins. That user is then responsible for validating the next block in a way that’s similar to a miner in Bitcoin being responsible for validating the new block when the miner succeeds in solving the computational problem. If a user validates a block that’s invalid they will lose part of the coins that they staked which incentives them to only validate valid blocks.
So, has the NYT had any reaction, response, or indication that they’re even considering the issue of publicizing private details of a pseudonymous author? Do we know when the article was planned for publication?
Unrelatedly, on another topic altogether, are there any new-and-upcoming blogs about topics of rationality, psychiatry, statistics, and general smart-ness, written by someone who has no prior reputation or ties to anyone, which I should be watching?
Incentives vs agency—is this an attribution fallacy (and if so, in what direction)?
Most of the time, when I see people discussing incentives about LW participation (karma, voting, comment quality and tone), we’re discussing average or other-person incentives, not our own. When we talk about our own reasons for participation, it’s usually more nuanced and tied to truth-seeking and cruxing, rather than point-scoring.
I don’t think you can create alignment or creative cooperation with incentives. You may be able to encourage it, and you can definitely encourage surface-cooperation, which is not valueless, but isn’t what you actually want. CF variants of Goodheart’s law—incentive design is _always_ misguided due to this, as visible incentives are _always_ a bad proxy for what you really want (deep and illegible cooperation).
There’s two sides of discussing incentives, wrt. X:
Incentivize X/Make tools that make it easier for people to do X [1].
Get rid of incentives that push people to not do X[2] /Remove obstacles to people doing X.
Even if alignment can’t be created with incentives, it can be made easier. I’m also curious about how the current incentives on LW are a bad proxy right now.
[1] There’s a moderation log somewhere (whatever that’s for?), GW is great for formatting things like bulleted lists, and we can make Sequences if we want.
[2] For example, someone made a post about “drive by criticism” a while back. I saw this post, and others, as being about “How can we make participating (on LW) easier (for people it’s hard for right now)?”
Comment throttling seems to catch people offguard somewhat often. I suspect this is because the criteria are imperfect, but also because it’s a VERY sharp change from “unlimited” to “quite small”.
Perhaps there should be a default throttle of 5 or 8 comments per day. And this throttle tightens with net downvotes. Maybe increase it with recent net upvotes, or maybe only by an explicit exception request with an explanation of why one wants to make more than that in a day.
edit: oh, better—do away with the daily granularity. Everyone has a leaky-bucket throttle, with say 4 comments max available, which replenish one every 2 hours. If your recent karma average is low, the replenishment slows down. If your recent karma is high, the replenishment speeds up. the bucket size could go up for posters with significant semi-recent upvotes.
Are human megastructures (capitalism, large corporations, governments, religious organizations) forms of AI?
They’re alien and difficult to understand. Hard to tell if they have their own motives, or if they’re reflecting some extracted set of beliefs from humans. They make leaps of action that individual humans can’t. They are at best semi-corrigible. Sounds very similar to LLMs to me.
edit: I know this isn’t a new thought, but it’s becoming more attractive to me as I think about more similarities. Groups do tend to hallucinate, both in policy that seems based on wrong assumptions, and in actual ludicrous statements.
Inspired by a thought I saw on Mastodon:
A running joke amongst my peers is that any AI news headline can replace “AI” with “capitalism” and remain 100% true.
To the extent that this is a valid comparison, it shows a pretty large problem with the idea that a newer/bigger AI can control and undo the effects of a previous one.
I increasingly think EA is just Objectivism for progressives.
From https://gideons.substack.com/p/altruism-wrap, which I found through Zvi’s writeup. I hadn’t heard it put quite this way before, but the “shut up and multiply” mindset (at least with made-up coefficients) really does lead to prioritizing far-mode ideals and shaky projections over actual real living people.
It’s weird that EA lost credibility (with many, at least partly including me) when it drifted away from mosquito nets as the thing it was mocked for talking about.
Mastodon—is there any primer or description of the theory of federation behind it? What decisions about content or discovery are made at the server level? How do I simultaneously have access to everything federated from every server while getting the advantages of having picked a server with my preferred norms/enforcement?
specifically:
Should I set up accounts on multiple servers? Why or why not?
Should I care which server(s) I use at all? Why or why not?
It seems like routing and identity (in terms of who’s following and going to see your posts) are tied to server. There are procedures for migration, but presumably a dead or rogue server would just lose all your data. Is there a secure export available, that would let one seamlessly restore all follows, followers, and posts on a new server?
Should I set up accounts on multiple servers? Why or why not?
Yes, you probably should, primarily as backup options for when your current server goes down. Happened to me twice.
Should I care which server(s) I use at all? Why or why not?
Probably a bit. You don’t want your server to be blocked by many other servers or be politically controversial (a mutual didn’t allow my follow request from my backup account because it was on a libertarian server). Finding interesting people on the fediverse is kind of difficult, starting with an interesting server can speed up that process. But most of the time you’ll probably be interacting with people who are not on your server.
It seems like routing and identity (in terms of who’s following and going to see your posts) are tied to server. There are procedures for migration, but presumably a dead or rogue server would just lose all your data. Is there a secure export available, that would let one seamlessly restore all follows, followers, and posts on a new server?
There is a method for exporting, but it only allows for importing follows on a new server, but not followers (which I think is technically infeasible, because adversarial users could just declare that everyone is following them now) or posts (which looks technically feasible but hasn’t been done yet).
Hmm. I tend to frame deontololgical moral/decision frameworks as evolved heuristics from consequentialism, with lost purposes as to how they came about, and some loss of future-optimization flexibility from that, but also some serious advantages from that in that it reduces computability paralysis AND motivated cognition to justify worse behaviors. So, not “correct”, but “performs better than correct for many parts of normal life”.
The recent discussion of acausal human decisions (which I think is incorrect) has made me wonder—is deontology a form of acausal thinking? “Do this because it’s right”, as distinct from “do this because it makes the world better according to your values”, is a pretty clear denial of causality.
I think this works if you use the adjudicator framing of acausal coordination, where it’s shared ideas that coordinate people, but people don’t themselves coordinate each other. Deontological principles and norms are such shared ideas, though I think mere words/concepts also count.
This becomes a form of consequentialism when the shared ideas are themselves agents, doing consequentialist decision making. Here, asking “What is right?” becomes asking what the updateless policy of the Ideal of Right (as an agent that acts as an adjudicator between people) would say about responding to your circumstance, taking into account the willingness and ability of people in various circumstances to listen to what the Ideal of Right has to say, as they jointly channel its will.
Last week I compared GameStop to Bitcoin. The thing about Bitcoin as a financial asset, I wrote, is that “there is no underlying claim; there is just a widespread acknowledgment that people think it’s valuable.” I suggested that that was a fascinating and powerful innovation,[3] but that once you are accustomed to it you might get a little overconfident and think that any financial asset—GameStop stock, say—could work the same way. People will buy it because they think it will retain value because people will buy it because they think it will retain value because people will buy it because etc., recursing infinitely, with no underlying fundamental rationale.
I thought that that would not work, with the stock of a mall retailer; eventually a bad earnings release or whatever will kill the vibe. But I know nothing. Maybe it can work; maybe GameStop traders can conjure value out of thin air and maintain it. Maybe people will buy GameStop because people buy GameStop, and it will go on forever because they don’t stop. “The thing I like about GameStop is not its underlying cash flows,” a venture capitalist will blog in 2027, “but the fact that it is a scarce digital store of value.”[4]
I don’t think, however many days we are into this nonsense, that GameStop is a particularly important story (though of course it’s a fun one!), or that it points to any deep problems in the financial markets. There have been bubbles, and corners, and short squeezes, and pump-and-dumps before. It happens; stuff goes up and then it goes down; prices are irrational for a while; financial capitalism survives.
But I tell you what, if we are still here in a month I will absolutely freak out. Stock prices can get totally disconnected from fundamental value for a while, it’s fine, we all have a good laugh. But if they stay that way forever, if everyone decides that cash flows are irrelevant and that the important factor in any stock is how much fun it is to trade, then … what are we all doing here?
My prediction is that it’s not sustainable as a base-less store of value or transaction currency in the same way as bitcoin, because there’s SO MUCH overhead in trading. Predicting internet attention span is a mug’s game, but I think it’ll take a little longer than Levine does—I’ll be surprised if it lasts to the end of February, shocked if it goes through March, but won’t absolutely freak out for a year.
In 2030: “People who bought GameStop stock are now billionaires. It was even mentioned on LessWrong, but again we collectively failed at instrumental rationality. Maybe next time.” :D
I always enjoy convoluted Omega situations, but I don’t understand how these theoretical entities get to the point where their priors are as stated (and especially the meta-priors about how they should frame the decision problem).
Before the start of the game, Omega has some prior distribution of the Agent’s beliefs and update mechanisms. And the Agent has some distribution of beliefs about Omega’s predictive power over situations where the Agent “feels like” it has a choice. What experiences cause Omega to update sufficiently to even offer the problem (ok, this is easy: quantum brain scan or other Star-Trek technobabble)? But what lets the Agent update to believing that their qualia of free-will is such an illusion in this case? And how do they then NOT meta-update to understand the belief-action-payout matrix well enough to take the most-profitable action?
I guess I’ve discussed my perspective on the issue of unrealistic hypotheticals here, although you’ve already commented on that post. Beyond that, Scott Alexanders The Least Convenient Possible World is a great post, but I suspect you’ve seen it too.
BTW, I created a wiki page for hypotheticals. I’ve summarised some arguments on why we should pay attention to unrealistic hypothetical, but it’d be useful to have some opposing arguments listed there as well.
Useful pointers. I do remember those conversations, of course, and I think the objections (and valid uses) remain—one can learn from unlikely or impossible hypotheticals, but it takes extra steps to specify why some parts of it would be applicable to real situations. I also remember the decoupling vs contextualizing discussion, and hadn’t connected it to this topic—I’m going to have to think more before I really understand whether Newcomb-like problems have clear enough paths to applicability that they can be decoupled by default or whether there’s a default context I can just apply to make sense of them.
Funny thought—I wonder if people were created (simulated/evolved/whatever) with a reward/utility function that prefers not to know their reward/utility function.
Is the common ugh field around quantifying our motivation (whether anti-economic sentiment, or just punishing those who explain the reasoning between unpleasant tradeoffs) a mechanism to keep us from goodhearting ourselves?
I’m torn between two models of recent technological change. In some sense, the internet from 1992 (hypertext is a cool organizing principle) to early ’00s was a transformative societal event, and it arguably destroyed civilization. It’s not clear to me whether AI is just the final visible piece of our self-destruction, or if it’s a brand new thing, hitting us after we were so weakened by the massive changes previously experienced.
I guess I could assign the breaking point still earlier—mass communication and easy air travel are what really broke through the isolated and understandable lives of most people.
Considering whether to set up (and when to use) an alt account. As LW starts to tolerate more controversial topics, and as the world is getting better at searching and holding people massively over-accountable for things that can be intentionally misconstrued, I find myself self-censoring some of my opinions on those topics.
Perhaps I should just ignore those topics, but I kind of like the interaction with smart people, even on topics I’d rather not be publicly discussing. Perhaps I should make a new throwaway each time (or every quarter, say). Perhaps I should get over myself, and stand behind everything I write.
The last one is kind of funny—I’m ALREADY using a pseudonym. But it’s one I’ve used since pre-internet days, and I use it enough that it wouldn’t take much motivation to connect it to my real-world existence.
If I make a comment, then the author deletes the post, is my comment lost as well? I’m pretty sure I don’t actually need a record of things I’ve said, and that my comments aren’t valuable enough to justify extra effort, but it kind of bugs me.
Has anyone written the scraper or script to save a copy of everything you write on LW, so it’s available even if it gets deleted later? If it archived the post and comment thread you’re responding to, it would be all right.
Currently they are still available but hard to find on greaterwrong. (I think maybe also on your profile page?)
I think probably we should make it so the comment thread structure is preserved on LW, we just haven’t gotten around to it.
Not on my profile page, as far as I’ve been able to tell. Not important enough to dig around GW. I think I’ll start just saving local copies of some of my comments, and decide whether to turn them into a shortform if I notice they’re deleted.
Would it be simple for the system to send me a PM with the contents when a comment is deleted by a post author (either directly or by deleting the post that contains it)?
I think it’s simpler to make them preserved on on your profile and/or on the original post.
In addition to being able to find your own recent comments, another issue is links to comments dying. For example if I were to link to this comment, I would worry it might quietly disappear at some point.
Yeah.
IMO this is correct to fix, there’s just a lot of other stuff to do and it hasn’t made it to the top of the queue
I love markdown when it works, and it drives me nuts when it doesn’t. All the docs and cheat sheets in the world don’t help.
Would it be possible to add a “view source” option for all posts and comments (not just mine), so I learn from others rather than re-discovering my own mistakes over and over?
I wish I could agree-and-downvote (or agree-and-not-vote) on posts. There are a fair few posts that are well-reasoned and probably correct, but just not important or interesting.
Yes, and the converse: posts that are interesting and provide useful information, but have fundamentally wrong arguments and derive invalid conclusions.
I wish people would be more careful about reinforcing attribution error. We all (including me) have a tendency to talk about types of people rather than behaviors.
I wish we talked about mistake theory rather than mistake theorists. I wish we talked about ideas or discussions at a given simulacra level, rather than people operating at that level. Recently, I wish it were a discussion about “empathetic oofs” rather than “empathetic oofers”.
I find it offputting, and it makes it harder for me to accept the underlying thesis, when such behaviors are attributed to immutable personality types. More importantly, I think it constrains our analysis very severely not to recognize that these are valid (or at least common) strategies, and we should strive to use both when appropriate.
I’m a bit uncomfortable with [edit: wrong word. all discussion is good, but I think there’s a big modeling error being made.] a lot of the discussion about ethics/morals around here, in that it’s often treated as a separate, incomparable domain to other motives/values/utility sources.
It may be, in humans and/or in AI, that there are fairly distinct modules for different domains, which somehow bid/negotiate for which of them influences a given decision by how much. This would make the “morals as distinct evaluations” approach reasonable, though the actual negotiation and power weights among modules seem more important to study.
But if something approaches VNM-rationality, it acts as if it has a unified consistent utility function. Which means an integrated set of values and motivations, not separate models of “what’s the right action in this context for me”.
So, either AI is irrational (like humans), or exploration of morality as a separate topic from status, pleasure, family, or other non-altruistic drives is not very applicable.
I’m pretty sure I mostly agree. However:
It is not obvious that vnm rationality is quite the right thing here. I talked to a game theorist dude at an event a while back and he said of the core assumptions of vnm rationality there’s one not everyone accepts, but I’ve had trouble re-locating what it was he said. In any case, garrabrant said he doesn’t accept vnm rationality for related but iirc slightly-different-maybe-not-sure reasons. But iirc it doesn’t completely invalidate combining everything into a single ranking function.
It might make sense to demand that morality is clearly spelled out in one of the theorems that constrain what utility functions we’d accept, though. That’s a little different from it being truly separate.
I concede that VNM may be too high a bar, and I’ve seen some clever things done by relaxing the independence axiom. I don’t think I’ve seen any that change the basic problem of (acting as if there were) a single value function that the agent is optimizing. To be rational, there can be no separation of moral and non-moral domains.
It’s fascinating to me to see people changing their recommended meta-ethics due to a very public bad actor and related financial meltdown. How does the FTX meltdown and fraud change your fundamental priorities OR your mechanisms for evaluating actions within those priorities?
Sure, adding some humility to the evaluation function is wise. Recognize that things probably aren’t as linear as you think (and certainly not as precise), when you get toward very large or small numbers. Update to focus on nearer-term, safer improvements. And reducing your trust (for those of you who had it) in the assumption that rich people who say comforting words are actually aligned in any useful way is … so obvious that I’m sad to have to say it.
But if you have believed for years that you can aggregate and compare future population utility against current experiences, and predict what actions will have large impacts on that future, it’s really odd if you give that up based on things that happened recently.
[ note: I am not utilitarian, but I am consequentialist. I never made the assumption that SBF was a pure (or even net) good, nor that his methods were likely to sustain. But even if I were surprised by this, I don’t see how it’s new evidence about something so fundamental. ]
I feel like FTX is a point against utilitarianism for the same reasons Bentham is a point for utilitarianism. If you take an ethical system to logical conclusions and anticipate feminism, animal rights, etc. this is evidence for a certain algorithm creating good in practice. If you commit massive fraud this is evidence against.
This also doesn’t shift my meta-ethics much, so maybe I’m not one of the people you’re talking about?
I agree, and I’m unsure why EAs are updating so much, short of PR reasons.
This doesn’t change much for me, because it doesn’t change the probabilities of moral realism being true.
EA folks: is there any donation I can make to increase the expected number of COVID-19 vaccinations given by end of February, either worldwide or for a specific region?
Increasing vaccine production capacity takes longer then two months. The only way you can increase vaccinations in the time frame is reducing the amount of vaccine that’s given per patient or approve more vaccines.
India’s approval of a vaccine without phase 3 trials would be the kind of action that you would want to encourage. 1 dosing instead of 2 dosing is an action that works in the developed world.
Running PR for such actions would be the only viable move. I’m however not aware of individual charities that would use marginal donations for that purpose.
Reminder to self: when posting on utiltarianism-related topics, include the disclaimer that I am a consequentialist but not a utilitarian. I don’t believe there is an objective, or even outside-the-individual perspective to valuation or population aggregation.
Value is relative, and the evaluation of a universe-state can and will be different for different agents. There is no non-indexical utility, and each agent models the value of other agents’ preferences idiosyncratically.
Strong anti-realism here. And yet it’s fun to play with math and devise systems that mostly match my personal intuitions, so I can’t stay away from those topics.
I think I agree that there’s no objectively true universal value aggregation process, but if you can’t find (a method for finding) very broad value aggregation system, then you can’t have peace. You have peace insofar as factions accept the rulings of the system. Simply giving up on the utilitarian project is not really an option.
That’s an interesting reason to seek Utilitarianism that I hadn’t considered. Not “this is true” or “this makes correct recommendations”, but “if it works, it’ll be a better world”. I see some elements of Pascal’s Wager in there, but it’s a novel enough (to me) approach that I need to think more on it.
I do have to point out that perhaps “you can’t have peace” is the actual true result of individual experience. You can still have long periods of semi-peace, when a coalition is strong enough to convince others to follow their regime of law (Pax Romana, or many of today’s nations). But there’s still a lot of individual disagreement and competition for resources, and some individuals use power within the system to allocate resources in ways that other individuals disprefer.
I’m not sure if that is “peace” within your definition. If so, Utilitarian aggregation isn’t necessary—we have peace today without a working Utility calculation. If not, there’s no evidence that it’s possible. It may still be a worthwhile goal to find out.
Shameless self-promotion but I think meta-preference utilitarianism is a way to aggregate value fairly without giving up anti-realism. Also, be sure to check out the comment by Lukas_Gloor as he goes into more depth about the implications of the theory.
Well, that’s one way to lean into the “this isn’t justified by fundamentals or truth, but if people DID agree to it, things would be nicer than today (and it doesn’t violate preferences that I think should be common)”. But I’m not sure I buy that approach. Nor do I know how I’d decide between that and any other religion as a desirable coordination mechanism.
I’m not sure what flavor of moral anti-realism you subscribe to so I can’t really help you there, but I am currently writing a post which aims to destroy a particular subsection of anti-realism. My hope is that if I can destroy enough of meta-ethics, we can hopefully eventually make some sort of justified ethical system. But meta-ethics is hard and even worse, it’s boring. So I’ll probably fail.
Good luck! I look forward to seeing it, and I applaud any efforts in that direction, even as I agree that it’s likely to fail :)
That’s a pretty good summary of my relationship to Utilitarianism: I don’t believe it, but I do applaud it and I prefer most of it’s recommendations to a more nihilistic and purer theory.
Wow what a coincidence, moral nihilism is exactly the subject of the post is was talking about. Here it is btw.
There’s a large class of social/cultural behaviors that I think are fine and good when they’re somewhat rare, and I find annoying and then harmful when it becomes very common.
This includes public demonstrations, minor mostly-victimless crimes, many kinds of drug use, tipping expectations, and I’ve just realized that April Fools is now in this category.
It’s amazing when there are a few surprising well-crafted jokes published. It’s annoying when it’s common enough that it’s hard to find the good ones, and even harder to find the non-joke communications.
</rant>
On LW you can add a tag filter for April Fool’s. So you can set that to hidden.
The problem with this (as with all examples of the phenomenon—things that are justified and good when they’re somewhat rare and intentionally transgressive or risky, but are annoying and harmful when safe and common) is that I really LIKE the gems, and want to (occasionally) partake myself.
It’s a matter of variable and contextual value (generally declining on the margin), and I don’t know any good answers.
Making the rounds.
User: When should we expect AI to take over?
ChatGPT: 10
User: 10? 10 what?
ChatGPT: 9
ChatGPT 8
...
There’s a lot of similarity between EMH (efficient markets hypothesis) and Aumann’s agreement theorem.
Both are about updating on other agents’ information.
Both make pretty strong statements about common knowlege.
Neither one applies in the interesting cases.
Types and Degrees of Maze-like situations
Zvi has been writing about topics inspired by the book Moral Mazes, focused on a subset of large corporations where politics rather than productivity are seen as the primary competition dimension and life focus for participants. It’s not clear how universal this is, nor what size and competitive thresholds might trigger it. This is just a list of other Maze-like situations that may share some attributes with those, and may have some shared causes, and I hope solutions.
Consumerism and fashion.
Social status signaling and competition.
Dating, to the extent you are thinking of eligibility pools and quality of partner, rather than individual high-dimensional compatibility.
Acutal politics (elected and appointed-by-elected office).
Academic publishing, tenure, chairs.
Law enforcement. Getting ahead in a police force, prison system, or judiciary (including prosecutors) is almost unrelated to actually reducing crime or improving the world.
Bureaucracies are a related, but different kind of maze. There, it’s not about getting ahead, it’s about … what? Shares the feeling that it’s not primarily focused on positive world impact, but maybe not other aspects of the pathology.
We should also add law and legislation to that list.
Expand on that a bit—is this an employment/social maze for lawyers and legislators, or a bureaucratic maze for regular citizens, or something else? I’ll definitely add policing (a maze for legal enforcers in that getting ahead is almost unrelated to stopping crime).
Employment and social maze for both lawyers and legislators, and sometimes judges as well a police (as you mention). For lawyers perhaps there is some asymmetry between DA side and Defendant in criminal cases, not sure there. Examples for DA and police are the incentives to get convictions and so the image of making things safe and producing justice leading to some pretty bad cases of bad behavior that amounts to either framing and innocent in some cases, getting a conviction by illegal means (slippery slope situations) which then undermine the entire legal structure/culture of a rule of law society.
Similar for legislators—election results tend to drive behavior with lots of wiggle room for not actually working on delivering campaign promises or not ever actually having intended to push the advertised agenda. The incentives of being a tenured politician, particularly at the federal and state level, related to personal benefit seem poorly aligned with serving the electorate (representing constituents values rather than maximizing the politician’s income, status and the like.
There might be some dynamic related to rent-seeking results too. If rent-seeking is purely a structural reality (standard organizational costs facing special versus general interests) that should stabilize until the underlying costs structures change. But I’m not entirely sure that is the case, if we can make the cases that rent-seeking is growing then perhaps a moral maze would provide a better explanation of that growth where no shift in underlying cost are able to explain the growth.
For the regular citizen certainly the bureaucracy creates problems both for engaging with government as well as perhaps engaging with politics and elections—though I’m less sure this is a real maze problem.
My daily karma tracker showed that 3 comments got downvoted. Honestly, probably justified—they were pretty low-value. No worries, but it got me thinking:
Can I find out how I’m voting (how many positive and negative karma I’ve given) over time periods? Can I find it out for others? I’d love it if I could distinguish a downvote from someone who rarely downvotes from one from a serial downvoter.
I’ve been downvoting less, recently, having realized how little signal is there, and how discouraging even small amounts can be. Silence or comment are the better response, for anything better than pure drivel.
Worth noting that by default the karma notifier doesn’t show downvotes, for basically this reason.
I think it only makes sense to opt into seeing all downvotes if you’re confident you can take this as information without it harming your motivation.
Meanwhile, there are benefits to actually downvoting things that are wrong, or getting undue attention, or subtly making the site worse. It matters how things can sorted for attention allocation, and relative karma scores in a discussion are important, among other things, so that people don’t see bad arguments appearing highly endorsed and then feeling motivated to argue against them (when they wouldn’t ordinarily consider this a good use of their time)
So I think people should actually just downvote more, and not use the “show me all my downvotes” feature, which most people aren’t using and you probably shouldn’t worry about overmuch.
I opted in to seeing downvotes, and I think my curiosity (both about personal interactions on the site and about the general topic of mechanism design) will compel me to continue. I’m not worried about them—as I say, they were low-value comments, though not much different from others that get upvoted more (and IMO shouldn’t be). Mostly it shows that low-number-of-voters is mostly noise, and that’s fine.
My main point was wondering how I can monitor my own voting behavior? (ok, and a little complaining that variable-point voting is confusing.) I think I upvote pretty liberally and downvote very rarely, but I really wonder if that’s true, and I wonder things like how many votes I give (on multiple comments) on the same post.
I do agree that being able to see metrics on your own voting is pretty reasonable, although not something I expect to make it to the top of our priority queue for awhile.
Should I assign a significant chance that the US will actually default on some debts? Betting markets are at 6%, but they’re pretty bad at the tails, for various reasons. CDS spreads (people buying insurance for/against a default) are 172bp, implying a 1.72% chance with a fair bit of real money wagered. It’s also likely somewhat skewed, for regulatory and technical reasons, but less than the betting markets.
This wouldn’t be an existential threat, but it’s definitely a stability and lifestyle threat for a lot of current and future humans, and likely exacerbates other threats.
Not sure I can do much about it, but it’s on my mind.
Here’s a prediction I haven’t seen anywhere else: due to border closures from COVID-19 and Brexit, the value of dual-citizenship is going up (or at least being made visible). This will lead to an uptick in mixed-nationality marriages.
“One of the big lessons of market design is that participants have big strategy sets, so that many kinds of rules can be bent without being broken. That is, there are lots of unanticipated behaviors that may be undesirable, but it’s hard to write rules that cover all of them. (It’s not only contracts that are incomplete...) ”—Al Roth
I think this summarizes my concerns with some of the recent discussions of rules and norm enforcement. People are complicated and the decision space is much larger than usually envisioned when talking about any specific rule, or with rules in general. Almost all interactions have some amount of adversarial (or at least not-fully-aligned) beliefs or goals, which is why we need the rules in the first place.
I’m pretty sure https://en.wikipedia.org/wiki/Seeing_Like_a_State has bigger lessons than just government or large corporations.
A general problem with optimization and control (in AI and commerce and everything else) is that it limits breadth of activity in (at least) two ways. The obvious way is that it avoids things which oppose the stated goals. That’s arguably a good and intentional limit.
But it ALSO avoids things that are illegible or unexpected and hard to quantify. I suspect that’s a large majority of experiential value for a large majority of humans.
I don’t know if this can be made rigorous enough to generate a ‘Friendliness Impossibility Theorem’, but it’s probably the thing that keeps me up at night most often about the future of conscious experience in the universe.
The sketch of that argument is that for any given level of conscious compute power (agentic and/or moral patient), MOST of the calculations will be illegible to that entity, and to entities of similar power. The only way to have provable/fully-trustworthy agreement on values is to have full knowledge of all other entities.
It’s possible that there’s a corollary that things CAN be provably friendly only if a majority of total compute power in conscious entities is used for monitoring and understanding each other, rather than for understanding and optimizing the rest of the universe. If that’s the case, there’s a very unfortunate value loss (to me and many current humans) in illegibility and surprise experiences.
tl;dr I like (some amount of hard-to-identify) chaos and surprises, and I don’t know how to square that with the fact that I don’t trust people or future AIs.
Not losing yourself in the long term is a challenge even without others implanting their influence. Brains are not built to avoid value drift for millions of years, learning is change that has no safety or reflective stability guarantees. So getting some sort of intelligent substrate that understands you or a change of cognitive architecture that lets you understand yourself is a relatively urgent need.
But if everyone is protected from external influence in depth as a result (including superintelligent influence), there is no need to guarantee safety of behavior, let alone cognition. Visible problems can always be solved with backups, it’s value drift of global power that would be irreversible.
I’ve lost myself multiple times over, even in my insanely brief (well over half of human expectation, but still massively inadequate) memory. I give a high probability that my personal experiential sequence will end and be lost forever fairly soon. That’s not my main concern here.
My concern is in understanding what I {do|should} care about beyond my own direct experiences. It’s about extrapolating the things I value in my own life to future trillions (or more, eventually) of experiencing entities. Or to a small number of extremely-experiencing entities, maybe—I don’t know what elements are important to my decisions of how to influence the future. I seem to care about “interesting” experiences more than specifically pleasant or painful or difficult ones. And I don’t know how to reconcile that with the goals often stated or implied around here.
My point is that there is motivation to solve the problem of value drift and manipulation both for people and global power, on the defense side. The thing to preserve/anchor could be anything that shouldn’t be lost, not necessarily conventional referents of concern. The conservative choice is trying to preserve everything except ignorance. But even vague preference about experience of others is a direction of influence that could persevere, indifferent to its own form.
If the problem is solved, there is less motivation to keep behavior or cognition predictable to make externalities safe for unprotected others, as there are no unprotected others to worry about.
I absolutely understand that there’s motivation to “solve the problem of value drift and manipulation”. I suspect that the problem is literally unsolvable, and I should be more agnostic about distant values than I seem to be. I’m trying on the idea of just hoping that there are intelligent/experiencing/acting agents for a long long time, regardless of what form or preferences those agents have.
It’s worth remembering, in discussions about decision theory, deontology vs consequentialism, and related topics, that ALL of our actual examples of agents are MISTAKEN or DECEPTIVE about the causes of their actions.
There are no consistent VNM-rational people.
I give some probability space to being a Boltzmann-like simulation. It’s possible that I exist only for an instant, experience one quantum of input/output, and then am destroyed (presumably after the extra-universal simulators have measured something about the simulation).
This is the most minimal form of Solipsism that I have been configured to conceive. It’s also a fun variation of MWI (though not actually connected logically) if it’s the case that the simulators are running multiple parallel copies of any given instant, with slightly different configurations and inputs.
Georgism isn’t taxation, it’s public property ownership.
It finally clicked in my mind why I’ve reacted negatively to the discussion about Georgist land taxes. It’s the same reaction I have to ANY 100% tax rate on anything. That’s not what “ownership” means.
Honestly, I’d probably be OK with (or at least I’d be able to discuss it more reasonably) the idea that land is not privately owned, ever. Only rented/leased from the government, revocable for failure to pay the current rate (equivalent in all ways to the Georgist tax amount, and penalties for failure to pay). I suspect it’s unworkable for the same reasons—the public won’t stand for evicting sympathetic tenants just because they moved in when the rent/tax was lower.
Further, taxation of economic movement or realized profit is pretty easy to justify, and ALWAYS exists. Taxation of durable-ownership valuation is tricky, as there is disagreement over whether it’s “real”, and there’s no guarantee that the investment value, even if correctly assessed, is liquid enough to actually pay the tax.
Edit: this had occurred to me before, but I forgot to put it here. It also suffers from the socialist calculation problem. When self-interested humans aren’t making decisions to maximize value (to themselves) about a thing, the actual most valuable use is very difficult to determine. Holding someone responsible for the theoretical value of something when they don’t actually think it’s the best is really difficult to envision.
Further edit: thanks for the comments—it’s helped me understand my true objection, in addition to the auxiliary problems that often come to mind. I think there are two insurmountable problems, not necessarily in this order:
Taxing anything at 100% is disingenuous about the word “ownership”. It should be called “rent” or something else that is clear that it’s not owned, but merely usable for some purposes until you decline to pay a future arbitrary amount. It would also make clear that you’re also undertaking all of the political-economy influences that make government setting of rent amounts problematic.
Taxing non-realized value is always going to be problematic, both for calculation reasons (valuation is theoretical and debatable), and for the fragility of the value. I think this generalizes to “only collect taxes in-kind (or on liquid enough things that it can be easily converted at tax-collection time)”. Taxing a pile of wheat is easy—take some percentage of it. Taxing a monetary transaction likewise—the price increases and the tax is paid out of that. Taxing a future value is not (generally) feasible—either you’ve just taken someone’s seeds so they can’t plant the crop you’re trying to tax, or you’re demanding cash from someone whose investments will suffer from early liquidation. Deferral of payment can help with this, but if you’re doing that, you should just tax the realized gains at time of liquidity.
We would really need the technology where you can take a house and move it to a different place. That would also make Georgism easier—when people don’t pay, you simply move them to a cheaper location.
While I think that would be extremely valuable technology, exiling Grandma to Outer Nowhereville, hours away from the rest of the family, is probably not a lot more feasible.
Alternatively, what if the LVT only increased when the property was sold? Buy land, and you lock in the LVT at the point in time when your deal went through.
I guess people would never sell the real estate at good location, only lease it? Thus capturing the increased value of the land without ever paying the tax.
Maybe leases are subject to increased LVT, just not owning it for personal use? I suspect there’s a way to enact something like this, but this isn’t in my wheelhouse.
Rich people and businesses are more strongly motivated and able to find loopholes than government administrators are to design robust mechanisms. Modern fractional ownership and lease arrangements are complicated enough that most any scheme based on a change in contract participants is going to be arbitraged away very quickly.
Right, but I thought the point was to avoid people balking at a law that makes it easier to evict ordinary folks who’ve lived somewhere a long time and don’t want to move? Let the rich people find their loopholes—as long as people support the system and the LVT delivers most of its promise, that’s good enough.
My point is that whatever exclusions are in place to keep poor people in their newly-valuable homes will be used by rich people to avoid taxation. There is no way to avoid this without making the tax variable based on other factors than the land value (like the “owner’s” perceived financial situation).
Either the law is the same for all “pay or GTFO”, or it’s differentially enforced, so the rich and the politically-sympathetic pay less than the middle-class.
It sounds like you think there’s no practical way to introduce exceptions to make an LVT politically palatable without also breaking its ability to motivate efficient land use and generate tax revenue.
My first thought is sure, we have a lot of tax evasion now. But we also do generate lots of tax revenue, most tax revenue comes from the very rich. Why would an LVT be so much more prone to tax evasion than the current system?
Correct, I think there’s no practical or realpolitik-theoretical way to make it work. To the extent that you make exceptions in implementation, it’s no longer a land-value tax, it’s a land-value fig leaf over a “what we can get away with” tax.
This is true the same way that to the extent you dilute your cocoa with water, it’s no longer hot chocolate, it’s water :) I don’t know of almost any “pure” real world policies or taxes. I’d need some good evidence to believe that the LVT is that fragile.
I sometimes think about Neanderthal tribes, and what they thought about alignment with Homo Sapiens (in truth, they didn’t; there wasn’t that much interaction and the takeover was over a longer timeframe than any individual of either species could imagine. but I still think about it).
I wonder if we could learn anything from doing a post-mortem (literally) from their viewpoint, to identify anything they could have done differently to make coexistence or long-term dominance over the newer species more likely.
They gave us some their genes.
Al Roth interview discussing the historical academic path from simplistic game theory to market design, which covers interesting mixes of games with multiple dimensions of payoff. https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.33.3.118
I wonder if the general consensus that current LLMs aren’t conscious enough to be moral patients should change my assumption that most humans are.
We should probably generalize cryptocurrency mechanisms to “proof of waste”. Fundamentally, the concept of a blockchain is that real-world resources are used for artifically-complex calculations just to show that one doesn’t need those resources for something else.
This is only true for “proof of work” currencies and thus does not generalize to crypto currency that’s more efficient.
Interesting! What cryptocurrencies or blockchains are there which are based on something other than proof of {some resource use}?
There are certainly non-crypt-based currencies and centralized data authorities that simply use fiat and access bottlenecks. I’ve only come across distributed authority mechanisms based on usage of some expensive and limited resource.
Cardano and Polkadot both use proof of stake. They are more like proof of resource ownership instead of proof of resource use. Resource is however a bit misleading here given that there are no resources in the classical sense needed to create the coins that are used for the staking in the first place. The coins are created out of nothing. Saying that minting coins out of nothing is creating resources is like saying that the FED printing money creates more resources for the economy.
Besides reducing electricity consumption that has additional advantages such as increase the amount of transactions that can be processed by two orders of magnitude (and additional orders of magnitude if you count side-chains). It also makes the process of validating blocks faster.
Filecoin is also noteworthy in that it uses proof-of-spacetime where miners do valuable work of storing data instead of wasteful work.
I was aware of the proof of spacetime as an alternative to the normal proof-of-computation chains. Unused (for other purposes) disk space definitely falls into the category of “resource”, as much as unused (for other purposes) compute capacity.
A quick glance at polkadot and cardano didn’t jump out as to what limits their tokens and what a “stake” actually is. Thanks for pointing them out, though—I have more to learn :)
In the case of Filecoin it’s not space that’s unused for other purposes. It’s space that’s used to store files that users (that are verified clients) of filecoin want to store.
The basic idea of proof-of-stake is that when it comes to validating a new block, one of the users that stake coins gets randomly chosen weighted by the amount of staked coins. That user is then responsible for validating the next block in a way that’s similar to a miner in Bitcoin being responsible for validating the new block when the miner succeeds in solving the computational problem. If a user validates a block that’s invalid they will lose part of the coins that they staked which incentives them to only validate valid blocks.
In practice it’s a bit more complicated then that and for Polkadot explained at https://wiki.polkadot.network/docs/en/learn-staking
So, has the NYT had any reaction, response, or indication that they’re even considering the issue of publicizing private details of a pseudonymous author? Do we know when the article was planned for publication?
Unrelatedly, on another topic altogether, are there any new-and-upcoming blogs about topics of rationality, psychiatry, statistics, and general smart-ness, written by someone who has no prior reputation or ties to anyone, which I should be watching?
Incentives vs agency—is this an attribution fallacy (and if so, in what direction)?
Most of the time, when I see people discussing incentives about LW participation (karma, voting, comment quality and tone), we’re discussing average or other-person incentives, not our own. When we talk about our own reasons for participation, it’s usually more nuanced and tied to truth-seeking and cruxing, rather than point-scoring.
I don’t think you can create alignment or creative cooperation with incentives. You may be able to encourage it, and you can definitely encourage surface-cooperation, which is not valueless, but isn’t what you actually want. CF variants of Goodheart’s law—incentive design is _always_ misguided due to this, as visible incentives are _always_ a bad proxy for what you really want (deep and illegible cooperation).
There’s two sides of discussing incentives, wrt. X:
Incentivize X/Make tools that make it easier for people to do X [1].
Get rid of incentives that push people to not do X[2] /Remove obstacles to people doing X.
Even if alignment can’t be created with incentives, it can be made easier. I’m also curious about how the current incentives on LW are a bad proxy right now.
[1] There’s a moderation log somewhere (whatever that’s for?), GW is great for formatting things like bulleted lists, and we can make Sequences if we want.
[2] For example, someone made a post about “drive by criticism” a while back. I saw this post, and others, as being about “How can we make participating (on LW) easier (for people it’s hard for right now)?”
Comment throttling seems to catch people offguard somewhat often. I suspect this is because the criteria are imperfect, but also because it’s a VERY sharp change from “unlimited” to “quite small”.
Perhaps there should be a default throttle of 5 or 8 comments per day. And this throttle tightens with net downvotes. Maybe increase it with recent net upvotes, or maybe only by an explicit exception request with an explanation of why one wants to make more than that in a day.
edit: oh, better—do away with the daily granularity. Everyone has a leaky-bucket throttle, with say 4 comments max available, which replenish one every 2 hours. If your recent karma average is low, the replenishment slows down. If your recent karma is high, the replenishment speeds up. the bucket size could go up for posters with significant semi-recent upvotes.
Are human megastructures (capitalism, large corporations, governments, religious organizations) forms of AI?
They’re alien and difficult to understand. Hard to tell if they have their own motives, or if they’re reflecting some extracted set of beliefs from humans. They make leaps of action that individual humans can’t. They are at best semi-corrigible. Sounds very similar to LLMs to me.
edit: I know this isn’t a new thought, but it’s becoming more attractive to me as I think about more similarities. Groups do tend to hallucinate, both in policy that seems based on wrong assumptions, and in actual ludicrous statements.
Inspired by a thought I saw on Mastodon:
To the extent that this is a valid comparison, it shows a pretty large problem with the idea that a newer/bigger AI can control and undo the effects of a previous one.
It’s like asking whether virus is a form of life. Whether the answer is yes or no, the fact remains that it is different from the other forms of life.
We can decide to call corporations “AI”, but it will not make them any more similar to LLMs.
From https://gideons.substack.com/p/altruism-wrap, which I found through Zvi’s writeup. I hadn’t heard it put quite this way before, but the “shut up and multiply” mindset (at least with made-up coefficients) really does lead to prioritizing far-mode ideals and shaky projections over actual real living people.
It’s weird that EA lost credibility (with many, at least partly including me) when it drifted away from mosquito nets as the thing it was mocked for talking about.
Mastodon—is there any primer or description of the theory of federation behind it? What decisions about content or discovery are made at the server level? How do I simultaneously have access to everything federated from every server while getting the advantages of having picked a server with my preferred norms/enforcement?
specifically:
Should I set up accounts on multiple servers? Why or why not?
Should I care which server(s) I use at all? Why or why not?
It seems like routing and identity (in terms of who’s following and going to see your posts) are tied to server. There are procedures for migration, but presumably a dead or rogue server would just lose all your data. Is there a secure export available, that would let one seamlessly restore all follows, followers, and posts on a new server?
Yes, you probably should, primarily as backup options for when your current server goes down. Happened to me twice.
Probably a bit. You don’t want your server to be blocked by many other servers or be politically controversial (a mutual didn’t allow my follow request from my backup account because it was on a libertarian server). Finding interesting people on the fediverse is kind of difficult, starting with an interesting server can speed up that process. But most of the time you’ll probably be interacting with people who are not on your server.
There is a method for exporting, but it only allows for importing follows on a new server, but not followers (which I think is technically infeasible, because adversarial users could just declare that everyone is following them now) or posts (which looks technically feasible but hasn’t been done yet).
Hope this helps.
Hmm. I tend to frame deontololgical moral/decision frameworks as evolved heuristics from consequentialism, with lost purposes as to how they came about, and some loss of future-optimization flexibility from that, but also some serious advantages from that in that it reduces computability paralysis AND motivated cognition to justify worse behaviors. So, not “correct”, but “performs better than correct for many parts of normal life”.
The recent discussion of acausal human decisions (which I think is incorrect) has made me wonder—is deontology a form of acausal thinking? “Do this because it’s right”, as distinct from “do this because it makes the world better according to your values”, is a pretty clear denial of causality.
I think this works if you use the adjudicator framing of acausal coordination, where it’s shared ideas that coordinate people, but people don’t themselves coordinate each other. Deontological principles and norms are such shared ideas, though I think mere words/concepts also count.
This becomes a form of consequentialism when the shared ideas are themselves agents, doing consequentialist decision making. Here, asking “What is right?” becomes asking what the updateless policy of the Ideal of Right (as an agent that acts as an adjudicator between people) would say about responding to your circumstance, taking into account the willingness and ability of people in various circumstances to listen to what the Ideal of Right has to say, as they jointly channel its will.
https://marginalrevolution.com/marginalrevolution/2021/12/hunting-smaller-animals-is-this-also-a-theory-of-early-economic-growth.html?utm_source=rss&utm_medium=rss&utm_campaign=hunting-smaller-animals-is-this-also-a-theory-of-early-economic-growth may explain a shift from stag-hunting to rabbits. It’s not a loss of cooperation, we killed all the stags.
Best few paragraphs I’ve read recently:
-- Matt Levine in his Money Stuff newsletter. See also Raemon’s recommendation for this newsletter at https://www.lesswrong.com/posts/9Qwignbzu4ddXLTsT.
My prediction is that it’s not sustainable as a base-less store of value or transaction currency in the same way as bitcoin, because there’s SO MUCH overhead in trading. Predicting internet attention span is a mug’s game, but I think it’ll take a little longer than Levine does—I’ll be surprised if it lasts to the end of February, shocked if it goes through March, but won’t absolutely freak out for a year.
In 2030: “People who bought GameStop stock are now billionaires. It was even mentioned on LessWrong, but again we collectively failed at instrumental rationality. Maybe next time.” :D
I always enjoy convoluted Omega situations, but I don’t understand how these theoretical entities get to the point where their priors are as stated (and especially the meta-priors about how they should frame the decision problem).
Before the start of the game, Omega has some prior distribution of the Agent’s beliefs and update mechanisms. And the Agent has some distribution of beliefs about Omega’s predictive power over situations where the Agent “feels like” it has a choice. What experiences cause Omega to update sufficiently to even offer the problem (ok, this is easy: quantum brain scan or other Star-Trek technobabble)? But what lets the Agent update to believing that their qualia of free-will is such an illusion in this case? And how do they then NOT meta-update to understand the belief-action-payout matrix well enough to take the most-profitable action?
I guess I’ve discussed my perspective on the issue of unrealistic hypotheticals here, although you’ve already commented on that post. Beyond that, Scott Alexanders The Least Convenient Possible World is a great post, but I suspect you’ve seen it too.
One additional thing which I can add is that this seems related to Decoupling vs. Contextualising norms.
BTW, I created a wiki page for hypotheticals. I’ve summarised some arguments on why we should pay attention to unrealistic hypothetical, but it’d be useful to have some opposing arguments listed there as well.
Useful pointers. I do remember those conversations, of course, and I think the objections (and valid uses) remain—one can learn from unlikely or impossible hypotheticals, but it takes extra steps to specify why some parts of it would be applicable to real situations. I also remember the decoupling vs contextualizing discussion, and hadn’t connected it to this topic—I’m going to have to think more before I really understand whether Newcomb-like problems have clear enough paths to applicability that they can be decoupled by default or whether there’s a default context I can just apply to make sense of them.
Funny thought—I wonder if people were created (simulated/evolved/whatever) with a reward/utility function that prefers not to know their reward/utility function.
Is the common ugh field around quantifying our motivation (whether anti-economic sentiment, or just punishing those who explain the reasoning between unpleasant tradeoffs) a mechanism to keep us from goodhearting ourselves?
I’m torn between two models of recent technological change. In some sense, the internet from 1992 (hypertext is a cool organizing principle) to early ’00s was a transformative societal event, and it arguably destroyed civilization. It’s not clear to me whether AI is just the final visible piece of our self-destruction, or if it’s a brand new thing, hitting us after we were so weakened by the massive changes previously experienced.
I guess I could assign the breaking point still earlier—mass communication and easy air travel are what really broke through the isolated and understandable lives of most people.
Considering whether to set up (and when to use) an alt account. As LW starts to tolerate more controversial topics, and as the world is getting better at searching and holding people massively over-accountable for things that can be intentionally misconstrued, I find myself self-censoring some of my opinions on those topics.
Perhaps I should just ignore those topics, but I kind of like the interaction with smart people, even on topics I’d rather not be publicly discussing. Perhaps I should make a new throwaway each time (or every quarter, say). Perhaps I should get over myself, and stand behind everything I write.
The last one is kind of funny—I’m ALREADY using a pseudonym. But it’s one I’ve used since pre-internet days, and I use it enough that it wouldn’t take much motivation to connect it to my real-world existence.