In the late 80′s, I was homeschooled, and studied caligraphy (as well as cursive); but I considered that more of a hobby than preparation for entering the workforce of 1000 years ago.
I also learned a bit about DOS and BASIC, after being impressed with the fractal-generating program that the carpenter working on our house wrote, and demonstrated on our computer.
Your definition seems like it fits the Emperor of China example—by reputation, they had few competitors for being the most willing and able to pessimize another agent’s utility function; e.g. 9 Familial Exterminations.
And that seems to be a key to understanding this type of power, because if they were able to pessimize all other agents’ utility functions, that would just be an evil mirror of bargaining power. Being able to choose a sharply limited number of unfortunate agents, and punish them severely pour encourager les autres, seems like it might just stop working when the average agent is smart enough to implicitly coordinate around a shared understanding of payoff matrices.
So I think I might have arrived back to the “all dominance hierarchies will be populated solely by scheming viziers” conclusion.
Clarifying question: If A>B on the dominance hierarchy, that doesn’t seem to mean that A can always just take all B’s stuff, per the Emperor of China example. It also doesn’t mean that A can trust B to act faithfully as A’s agent, per the cowpox example.
If all dominance hierarchies control is who has to signal submission to whom, dominance seems only marginally useful for defense, law, taxes, and public expenditure; mostly as a way of reducing friction toward the outcome that would have happened anyway.
It seems like, with intelligence too cheap to meter, any dominance hierarchy that doesn’t line up well with the bargaining power hierarchy or the getting-what-you-want vector space is going to be populated with nothing but scheming viziers.
But that seems like a silly conclusion, so I think I’m missing something about dominance hierarchies.
Note also that there are several free parameters in this example. E.g., I just moved to Germany, and now have wimpy German burners on my stove. If I put on a large container with 6L or more of water, and I do not cover it, the water will never go beyond bubble formation into a light simmer, let alone a rolling boil. If I cover the container at this steady state, it reaches a rolling boil in about another 90s.
Is Patrick McKenzie (@patio11) another Matt Levine of fintech? Or is he something else? I know several people outside of the industry (including myself) who read pretty much everything he writes, which includes a lot of technical detail written very accessibly.
I think being a Catholic with no connection to living leaders makes more sense than being an EA who doesn’t have a leader they trust and respect
Catholic EA: You have a leader you trust and respect, and defer to their judgement.
Sola Fide EA: You read 80k hours and Givewell, but you keep your own spreadsheet of EV calculations.
I’d be interested to know what the numbers on UV in ductwork look like over the past 5 years. When I had to get a new A/C system installed in 2020, they asked whether I wanted a UVC light installed in the air handler. I had, before then, been using a 70w UVC corn light I bought on Amazon to sterilize the exterior of groceries (back when we thought fomites might be a major transmission vector), and in improvised ductwork with fans and cardboard boxes taped together.
Getting a proper bulb—an optimal wavelength source—seemed like a big upgrade. Hard to come up with quantitative efficacy numbers, but we did have a friend over for the day, who turned out to have been in the early stages of covid, without getting infected. Our first infection was years later, at a music event.
This is great! Everybody loves human intelligence augmentation, but I’ve never seen a taxonomy of it before, offering handholds for getting started.
I’d say “software exobrain” is less “weaksauce,” and more “80% of the peak benefits are already tapped out, for conscientious people who have heard of OneNote or Obsidian.” I also am still holding out for bird neurons with portia spider architectural efficiency and human cranial volume; but I recognize that may not be as practical as it is cool.
It’s very standard advice to notice when a sense of urgency is being created by a counterparty in some transaction; and to reduce your trust in that counterparty as well as pausing.
It feels like a valuable observation, to me, that the counterparty could be internal—some unendorsed part of your own values, perhaps.
(e.g. in the hypothetical ‘harbinger tax’ world, you actively want to sabotage the resale value of everything you own that you want to actually use).
“harberger tax,” for anyone trying to look that up.
If you can pay the claimed experts enough to submit to some testing, you could use Google’s new doubly-efficient debate protocol to make them either spend some time colluding, or spend a lot more time in their efforts at deception: https://www.lesswrong.com/posts/79BPxvSsjzBkiSyTq/agi-safety-and-alignment-at-google-deepmind-a-summary-of
This could exclude competent evaluators without other income—this isn’t Dath Ilan, where a bank could evaluate evaluators and front them money at interest rates that depended on their probability of finding important risks—and their shortage of liquidity could provide a lever for distortion of their incentives.
On Earth, if someone’s working for you, and you’re not giving them a salary commensurate with the task, there’s a good chance they are getting compensation in other ways (some of which might be contrary to your goals).
Thanks! Just what I was looking for.
Some cities have dedicated LW/ACX Discord servers, which is pretty neat. Many of the cities hosting meetups over the next month are too small to have much traffic to such a server, were it set up. A combined, LW meetup oriented Discord server for all the smaller cities in the world, with channels for each city and a few channels for common small-meetup concerns, seems like a $20 bill on the sidewalk. So I’m checking whether such a thing exists here, before I start it.
I think the cruxes here are whether Aldi forced out small retailers like Walmart did; and how significant the difference between Walmart and Aldi is, compared to the difference between Aldi and large, successful retail orgs in wentworthland or christiankiland.
(my experience in German shopping is that most grocery stores are one of a half-dozen chains, most hardware stores are Bauhaus or OBI, but there isn’t a dominant “everything” store like Walmart; Müller might be closest but its market dominance and scale is more like K-mart in the 90′s than Walmart today.)
An existing subgenre of this with several examples is the two-timer date. As I recall, it was popular in 90′s sitcoms. Don’t expect INT 18 tier scheming, but it does usually show the perspective of the people frantically trying to keep the deception running.
Here’s the intuition that’s making me doubt the utility of provably correct system design to avoiding bridge crashes:
I model the process leading up to a ship that doesn’t crash into a bridge as having many steps.
1. Marine engineers produce a design for a safe ship
2. Management signs off on the design without cutting essential safety features
3. Shipwrights build it to spec without cutting any essential corners
4. The ship operator understands and follows the operations and maintenance manuals, without cutting any essential corners
5. Nothing out-of-distribution happens over the lifetime of the ship.
And to have a world where no bridges are taken out by cargo ships, repeat that 60,000 times.
It seems to me that provably safe design can help with step 1--but it’s not clear to me that step 1 is where the fault happened with the Francis Scott Key bridge. Engineers can and do make bridge-destroying mistakes (I grew up less than 50 miles from the Tacoma Narrows bridge), but that feels rare to me compared to problems in the other steps: management does cut corners, builders do slip up, and O&M manuals do get ignored.
With verifiable probabilities of catastrophe, maybe a combination of regulation and insurance could incentivize makers and operators of ships to operate safely—but insurers already employ actuaries to estimate the probability of catastrophe, and it’s not clear to me that the premiums charged to the MV Dali were incorrect. As for the Francis Scott Key, I don’t know how insuring a bridge works, but I believe most of the same steps and problems apply.
(Addendum: The new Doubly-Efficient Debate paper on Google’s latest LW post might make all of these messy principal-agent human-corrigibility type problems much more tractable to proofs? Looks promising.)
https://space.mit.edu/home/tegmark/crazy.html Large universes put some subtleties into the meaning of “real” that aren’t present in its common usage.
Decision theory-wise, caring about versions of yourself that are inexorably about to dissolve into thermal noise doesn’t seem useful. As a more general principle, caring about the decisions you make seems useful to the extent that those decisions can predictably change things.
My dreams have none of the consistency that allowed smart people to figure out the laws of nature over the millenia. It might be possible for a superintelligence to figure out how to make decisions within a world working on dream rules which had predictable future effects, but I believe it to be far beyond my powers.
That example seems particularly hard to ameliorate with provable safety. To focus on just one part, how could we prove the ship would not lose power long enough to crash into something? If you try to model the problem at the level of basic physics, it’s obviously impossible. If you model it at the level of a circuit diagram, it’s trivial—power sources on circuit diagrams do not experience failures. There’s no obviously-correct model granularity; there are schelling points, but what if threats to the power supply do not respect our schelling points?
It seems to me that, at most, we could prove safety of a modeled power supply, against a modeled, enumerated range of threats. Intuitively, I’m not sure that compares favorably to standard engineering practices, optimizing for safety instead of for lowest-possible cost.
The story I read about why neighbor polling is supposed to correct for bias in specifically the last few presidential elections is that some people plan to vote for Trump, but are ashamed of this, and don’t want to admit it to people who aren’t verified Trump supporters. So if you ask them who they plan to vote for, they’ll dissemble. But if you ask them who their neighbors are voting for, that gives them permission to share their true opinion non-attributively.