quiet_NaN

Karma: 345

quiet_NaN Apr 3, 2025, 12:55 PM
2 points
0
in reply to: Big Tony’s comment on: How To Believe False Things

What other embarrassingly unequal parts of reality are being politely ignored, except by science-illiterate jerks?

First, I kind of object to the “science-illiterate” part (I know it is a quote from the article). The soft sciences have done many wrongs, but with regard to the average physical ability of men and women, I don’t think that the depiction of women as just as capable is based on published studies.

Second, this is kind of culture war bait, and as such might be better asked at the motte.

Despite that, here are some possible answers.

With regard to gender differences, one might look at Women in chess. WP is quick to tell us that

The low number of women to reach the top level of chess has created a lot of interest as to why women historically have not had more success. There is no evidence that women are innately disadvantaged at chess. It has been demonstrated statistically that the low numbers across all levels can largely account for the lack of women at or near the top. The general paucity of women in chess has contributed to women commonly being the subject of sexism, harassment, and sexual harassment, factors also thought to contribute to women achieving less or leaving chess early.

But also:

Judit Polgár, generally considered the strongest female player of all time,[4] was at one time the eighth highest rated player in the world, and remains the only woman to have ever been rated in the world’s top ten.

Personally, I think that part of the reason is that women are generally less likely to go uber-nerd on a very small topic such as chess or warhammer and dedicate their life to that topic—which is what is likely required to be a top ten chess player. But I am also ⁵⁰⁄₅₀ on there being an inherent male advantage to chess. I think I can entertain that notion without having to adjust the worth of women in general. Emmy Noether shows that we dismiss the intelligence of women at our own peril.

The other thing which comes to mind is HBD, which is it’s own can of worms. See Scott Alexander on the Ashkenazi intelligence hypothesis, for example.

quiet_NaN Mar 23, 2025, 4:14 PM
1 point
0
on: Privateers Reborn: Cyber Letters of Marque
I am not sure what you are suggesting. Is your call for sanctioning private “hack-backs” of foreign organizations and individuals suspected of ransomware attacks, or is it more generally sanctioning attacks against organizations residing in state entities which are suspected of fostering ransomware groups? The analogy to letters-of-marque would suggest the latter—it did not matter if you were a peaceful merchant, if you were flying the flag of the enemy you were fair game.

Either one seems to be a terrible idea.

First, attribution of IT attacks is notoriously hard. Sure, in many cases you can see which IP addresses the attacker used, but chances are that they did not use their personal DSL to attack you directly, but attacked you through another victim whose only crime is that it too had insufficient network security.

Second, I do not really see how hacking ransomware groups backed by nation state actors would lead to their arrests. Redirecting plane flights to your own sovereign soil so you can arrest someone is a tactic which was used by several states, but I do not think that it is either feasible or wise to try this purely through software exploits. Nor do I think it is wise to normalize this behavior.

Third, ransomware groups have a much smaller attack surface than their targets. Your Texan municipality likely ran a bunch of services on a shoestring budget, probably based on a Microsoft stack, possibly end-of-life. By contrast, a successful ransomware group will have a security budget per employee that is orders of magnitude higher. They will likely not use Outlook+Office+AD in their network, and might not even use email for internal communication. They will also not run a bunch of half-baked online services to get an appointment at the DMV or whatever. Sure, the NSA might get into their network, but they would also be reluctant to burn through their precious hoarded 0-days to do so.

--

My counter-proposal would be to just criminalize paying the ransom. At the end of the day, ransomware is a coordination problem. If nobody ever paid, it would not be a thing. But once you have been hit with it, it is generally cheaper to pay than to accept an extended outage (at least, if the attacker calculated their ransom correctly). By making the expected costs for paying the ransom higher (e.g. through corporate death penalty or excessive taxes if caught), one can easily adjust incentives for the negative externalities.

More fundamentally, there is the victim’s narrative that IT security is impossible in the face of ransomware groups. “We bought advanced endpoint snake oil 2025 enterprise edition and sprinkled it all over our AD, and still we got hacked. This proves that when facing an adversary with a lot of criminal energy, even state of the art cybersecurity is insufficient.” Bollocks. If you had spent the kind of money you suddenly had when it came to paying the ransom beforehand, you likely would not have been hacked.

quiet_NaN Mar 22, 2025, 5:47 PM
9 points
0
on: On the Rationality of Deterring ASI
I have just seen this in curated, but I had previously commented on Zvi’s reporting on it.

Obviously, any nation state aware of the escalation ladder who wanted to be the first to develop ASI would put their AI cluster deep underground and air-gap it. We must not allow a mine shaft gap and all that. Good luck to their peer superpowers to actually conquer the hardened bunker.

Also, to MAIM, you have to know that you are in imminent danger. But with ASI nobody is sure when the point of fast takeoff—if there is any—might start. Is that cluster in that mine still trying to catch up to ChatGPT, or has it reached the point where it can do useful AI research and find algorithmic gains far beyond what humans would have discovered in a millennium? Would be hard to tell from the outside.

Emphasize terrorist-proof security over superpower-proof security. Though there are benefits to state-proof security (SL5), this is a remarkably daunting task that is arguably much less crucial than reaching security against non-state actors and insider threats (SL3 or SL4).

This does not seem to have the least to do with super-intelligence. Daesh is not going to be the first group to build ASI, not in a world where US AI companies burn through billions to get there as soon as possible.

The Superintelligence Strategy paper mentions the 1995 Tokyo subway sarin attack, which killed 13 people. If anything, that attack highlights how utterly impractical nerve gas is for terrorist attacks. That particular group of crazies spent a lot of time on synthesizing a nerve gas (as well as a few other flashy plans) only for their their death toll being similar to a lone wolf school shooter or a someone driving a truck into a crowd. Even if their death toll was increased by an order of magnitude due to AI going “Sure, here are some easy ways to disperse Sarin in a subway carriage”, their attacks would still be pretty ineffective compared to more mundane attacks such as bombs or knives.

Basically, when DeepSeek released their weights (so terrorist groups can run it locally instead of foolishly relying on company hosted AI services where any question towards the productions of “WMD” would raise a giant red flag), I did not expect that this would be a significant boon for terrorists, and so far I have not seen anything convincing me of the opposite.

But then again, that paper seems to be clearly targeted towards the state security apparatus, and terrorists were the bogeyman of that apparatus since GWB, so it seems obvious to emphasize the dangers of AIs with “but what if terrorists use them” instead of talking about x-risks or the like.

quiet_NaN Mar 17, 2025, 1:59 PM
7 points
0
on: On MAIM and Superintelligence Strategy
The reason why MAD works (sometimes) is because
- ICBM launches are easy to detect
- It is hard to wipe out the second-strike capability of an enemy before they know what is going on.
By contrast, there is no fire alarm for ASI. Nobody knows how many nodes an neural net needs to start a self-improvement cascade which will end in the singularity, or if such a thing is even possible. Also, nobody knows if an ASI can jump by a few 100 IQ points just through algorithmic gains or if it will be required to design new chip fabs first.

--

Some more nitpicks about specifics:
- To protect against ‘cyber’ attacks, the obvious defense is to air-gap your cluster. Granted, there have been attacks on air-gapped systems such as Stuxnet. But that took years of careful planning and an offensive budget which was likely a few OOM higher than what the Iranians were spending on IT security, and it worked exactly once.
- Geolocation in chips: Data centers generally have poor GPS reception. You could add circuitry which requires the chip to connect to its manufacturer and measure the network delay, though. I will note that DRM, TPM, security enclaves and the like have been a wet dream of the content industry for as long as I have been alive, and that their success is debatable—more often than not, they are cracked rather sooner than later. If your adversary can open the chips and modify the circuitry (at a larger feature size—otherwise they would build their own AI chips), protecting against all possible attacks seems hard. Also, individual chips likely do not have the big picture context of what they are working on, e.g. if they are training a large LLM or a small one. To extend the nuclear weapon analogy: Sure, give your nukes anti-tampering devices, but the primary security should be that it is really hard to steal your nukes, not that removing the anti-tampering device is impossible.
- Drones and pre-existing military: An AI which works by running more effective agents in human-designed drones is not an ASI. An ASI would do something much more grand and deadly—grey goo, engineered plagues, hacking the enemies ICBMs or something else, perhaps something which no human has even thought of yet. Pretending that our current weapon systems will be anything more than sticks and stones in the face of ASI is pure pandering to the military industrial complex.
- Likewise, I do not think it is wise to “carefully integrate AI into military command and control”. As much as I distrust human decision making with regard to ICBMs, I will take them over ChatGPT any day of the week.
--

If we end up with MAIM, here is how I think it might work:
- Establishment of an International AI Agency in analogy to the IAEA.
- The IAIA will inspect the whole production chain—from EUV machines, state-of-the-art chip fabs to data centers which run AI-capable chips worldwide. It will limit the cluster sizes for AI training and/or control that the workloads are below a certain size.
- Any defection from the IAIA inspections is seen as an attempt to develop ASI. While withdrawal from the NPT merely results in diplomatic interventions and sanctions, an attempt at developing ASI is not seen as an illicit nuclear weapons program, but like a missile launch—it will immediately trigger a nuclear attack.
Of course, there are some problems with that approach:
- Obviously every superpower would have incentives to develop ASI in secret.
- A nation might decide that being the first to develop ASI is well worth getting nuked over.
- The IAIA has no idea at what model size the runaway to ASI starts.
- Politicians might be reluctant to end the world as we know it over a “mere” treaty withdrawal in a world where we do not even know that AI is feasible.

quiet_NaN Jan 25, 2025, 12:55 AM
1 point
2
in reply to: quetzal_rainbow’s comment on: Mechanisms too simple for humans to design
Unlike Word, the human genome is self-hosting. That means that it is paying fair and square for any complexity advantage it might have—if Microsoft found that the x86 was not expressive enough to code in a space-efficient manner, they could likewise implement more complex machinery to host it.

Of course, the core fact is that the DNA of eukaryotes looks memory efficient compared to the bloat of word.

There was a time when Word was shipped on floppy disks. From what I recall, it came on multiple floppies, but on the order of ten, not a thousand. With these modern CD-ROMs and DVDs, there is simply less incentive to optimize for size. People are not going to switch away from word to libreoffice if the latter was only a gigabyte.

quiet_NaN Jan 25, 2025, 12:40 AM
5 points
4
in reply to: NickH’s comment on: Mechanisms too simple for humans to design
I think formally, the Kolmogorov complexity would have to be stated as the length of a description of a Turing Machine (not that this gets completely rid of any wiggle room).

Of course, TMs do not offer a great gaming experience.

“The operating system and the hardware” is certainly an upper bound, but also quite certainly to be overkill.

Your floating point unit or your network stack are not going to be very busy while you play tetris.

If you cut it down to the essentials (getting rid of things like scores which have to displayed as characters, or background graphics or music), you have a 2d grid in which you need to toggle fields, which is isomorphic to a matrix display. I don’t think that having access to boost or JCL or the python ecosystem is going to help you much in terms of writing a shorter program than you would need for a bit serial processor. And these things can be crazy small—this one takes about 200 LUTs and FFs. If we can agree that an universal logic gate is a reasonable primitive which would be understandable to any technological civilization , then we are talking on the order of 1k or 2k logic gates here. Specifying that on a circuit diagram level is not going to set you back by more than 10kB.

So while you are technically correct that there is some overhead, I think directionally Malmesbury is correct in that the binary file makes for a reasonable estimate of the information content, while adding the size of the OS (sometimes multiple floppy disks, these days!) will lead to a much worse estimate.

quiet_NaN Jan 15, 2025, 11:32 PM
4 points
2
in reply to: TAG’s comment on: Chance is in the Map, not the Territory
Agreed.
If the authors claim that adding randomness in the territory in classical mechanics requires making it more complex, they should also notice that for quantum mechanics, removing the probability from the territory for QM (like Bohmian mechanics) tends to make the the theories more complex.
Also, QM is not a weird edge case to be discarded at leisure, it is to the best of our knowledge a fundamental aspect of what we call reality. Sidelining it is like arguing “any substance can be divided into arbitrary small portions”—sure, as far as everyday objects such as a bottle of water are concerned, this is true to some very good approximation, but it will not convince anyone.
Also, I am not sure that for the many world interpretation, the probability of observing spin-up when looking at a mixed state is something which firmly lies in the map. From what I can tell, what is happening in MWI is that the observer will become entangled with the mixed state. From the point of view of the observer, they find themselves either in the state where they observed spin-up and spin-down, but their world model before observation as “I will find myself either in spin-up-world or spin-down-world, and my uncertainty about which of these it will be is subjective” seems to grossly misrepresent their model. They would say “One copy of myself will find itself in spin-down-world, and one in spin-up-world, and if I were to repeat this experiment to establish a frequentist probability, I would find that that the probability of each outcome is given by the coefficient of that part of the wave function to the power of two”.
So, in my opinion,
- If a blackjack player wonders if a card placed face-down on the table is an ace, that is uncertainty in their map.
- If someone wonders how a deterministic but chaotic physical system will evolve over time, that is also uncertainty in the map.
- If someone wonders what outcome they are likely to measure in QM, that is (without adding extra epicycles) uncertainty in the territory.
- If someone wonders what the evolution of a large statistical ensemble which is influenced by QM at the microscopic level (such as a real gas) is, that might be mostly uncertainty in the map if looking at very short time scales (where position and momentum uncertainty and the statistical nature of scattering cross-sections would not constrain Laplace’s daemon too much), but exists in the territory for most useful time spans.

quiet_NaN Jan 13, 2025, 1:39 AM
5 points
0
on: A Novel Idea for Harnessing Magnetic Reconnection as an Energy Source
Okay. So from what I understand, you want to use a magnetic effect observed in plasma as a primary energy source.

Generally, a source of energy works by taking a fuel which contains energy and turning it into a less energetic waste product. For example, carbon and oxygen can be burned to form CO2. Or one can split some uranium nucleus into two fragments which are more stable and reap the energy difference as heat.

Likewise, a wind turbine will consume some of the kinetic energy of the air, and a solar panel will take energy from photons. For a fusion reactor, you gain energy because you turn lighter nuclei (hydrogen isotopes or helium-3) into helium-4, which is extraordinarily stable.

Thus, my simple question: where does the energy for your invention come from? “The plasma” is not a sufficient answer, because on Earth we generally encounter plasma rarely for us to exploit, in fusion reactor designs, it is generated painstakingly at a huge energy cost.

Something goes into your reactor, and something comes out of it. If it is the same, then it can hardly have expended energy in your reaction.

quiet_NaN Jan 5, 2025, 6:25 PM
1 point
0
in reply to: Three-Monkey Mind’s comment on: The Online Sports Gambling Experiment Has Failed
Seconded. Also, in the second picture, that line is missing, so it seems that it is just Zvi complaining about the “win probability”?
My guess is that the numbers (sans the weird negative sign) might indicate the returns in percent for betting on either team. Then, if the odds were really 50:50 and the bookmaker was not taking a cut, they should be 200 each? So 160 would be fair if the first team had a win probability of 0.625, while 125 would be fair if the other team had a win probability of 0.8. Of course, these add up to more than one, which is to be expected, the bookmaker wants to make money. If they were adding up to 1.1, that would (from my gut feeling) be a ten percent cut for the bookie. Here, it looks like the bookie is taking almost a third of the money? Why would anyone play at those odds? I find it hard to imagine that anyone can outperform the wisdom of the crowds by a third. The only reason to bet here would be if you knew the outcome beforehand because you had rigged the game.
This is all hypothetical, for all I know the odds in sports betting are stated customarily as the returns on a 70$ bet or whatever.
Edit: seems I was not very correct in my guess.

quiet_NaN Jan 2, 2025, 8:36 AM
2 points
−1
on: Practicing Bayesian Epistemology with “Two Boys” Probability Puzzles
There is also a quantum version of that puzzle.
I have two identical particles of non-zero spin in identical states (except possibly for the spin direction). One of them is spin up. What is the probability that both of them are spin up.
For fermions, that probability is zero, of course. Pauli exclusion principle.
For bosons, …
… the key insight is that you can not distinguish them. The possible wave functions are either (spin-up, spin-up) or (spin-up,spin-down)=(spin-down,spin-up). Hence, you get p=1/2. (From this, we can conclude that boys (p=1/3) are made up from ²⁄₃ bosons and ¹⁄₃ fermions.)

quiet_NaN Dec 29, 2024, 7:08 PM
1 point
0
on: If all trade is voluntary, then what is “exploitation?”
Let us assume that the utility of personal wealth is logarithmic, which is intuitive enough: 10k$ matter a lot more to you if you are broke than if your net worth is 1M$.
Then by your definition of exploitation, every transaction where a poor person pays a rich person and enhances their personal wealth in the process is exploitive. The worker surely needs the rent money more than the landlord, so the landlord should cut the rent to the point where he does not make a profit. Likewise the physician providing aid to the poor, or the CEO selling smartphones to the middle class.
Classifying most of capitalism as “exploitive” (per se) is of course not helpful. You can add epicycles to your definition by considering money rather than its subjective utilitarian value, but under that definition all financial transactions would be ‘fair’: the poor person values 1k$ more than his kidney, while the rich person values the kidney more than 1k$, so they trade and everyone is better off (even though we would likely consider this trade exploitative).
More generally, to have a definition of fairness, we need some unit of inter-personal utility. Consider the Ultimatum game. If both participants are of similar wealth and with similar needs and have worked a similar amount for the gains, then a 50:50 split seems fair. But if we don’t have a common frame of utility, perhaps because one of them is a peasant and the other is a feudal king, then objectively determining what is a fair split seems impossible.

quiet_NaN Dec 29, 2024, 6:27 PM
6 points
0
on: By default, capital will matter more than ever after AGI
Some comments.
[...] We will quickly hit superintelligence, and, assuming the superintelligence is aligned, live in a post-scarcity technological wonderland where everything is possible.
Note, firstly, that money will continue being a thing, at least unless we have one single AI system doing all economic planning. Prices are largely about communicating information. If there are many actors and they trade with each other, the strong assumption should be that there are prices (even if humans do not see them or interact with them). Remember too that however sharp the singularity, abundance will still be finite, and must therefore be allocated.
Personally, I am reluctant to tell superintelligences how they should coordinate. It feels like some ants looking at the moon and thinking “surely if some animal is going to make it to the moon, it will be a winged ant.”Just because market economies have absolutely dominated the period of human development we might call ‘civilization’, it is not clear that ASIs will not come up with something better.
The era of human achievement in hard sciences will probably end within a few years because of the rate of AI progress in anything with crisp reward signals.
As an experimental physicist, I have opinions about that statement. Doing stuff in the physical world is hard. The business case for AI systems which can drive motor vehicles on the road is obvious to anyone, and yet autonomous vehicles remain the exception rather than the rule. (Yes, regulations are part of that story, but not all of it.) By contrast, the business case for an AI system which can cable up a particle detector is basically non-existent. I can see an AI using either a generic mobile robot developed for other purposes for plugging in all the BNC cables, or I can see it using a minimum wage worker with a head up display as a bio-drone—but more likely in two decades than a few years.
Of course, experimental physics these days is very much a team effort—the low-hanging fruits have mostly been picked, nobody is going to discover radium or fission again, at the most they will be a small cog in a large machine which discovers the Higgs boson or gravity waves.^[1] So you might argue that experimental physics today is already not a place for peak human excellence (a la the Humanists in Terra Ignota).
More broadly, I agree that if ASI happens, most unaugmented humans are unlikely to stay at the helm of our collective destiny (to the limited degree they ever were). Even if some billionaire manages to align the first ASI to maximize his personal wealth, if he is clever he will obey the ASI just like all the peasants. His agency is reduced to not following the advice of his AI on some trivial matters. (“I have calculated that you should wear a blue shirt today for optimal outcomes.”—“I am willing to take a slight hit in happiness and success by making the suboptimal choice to wear a green shirt, though.”)
Relevant fiction: Scott Alexander, The Whispering Earring.
Also, if we fail to align the first ASI, human inequality will drop to zero.
1. ^
  Of course, being a small cog in some large machine, I will say that.

quiet_NaN Dec 28, 2024, 5:00 PM
16 points
4
on: Review: Planecrash
Some additional context.
This fantasy world is copied from a role-playing game setting—a fact I discovered when Planecrash literally linked to a Wiki article to explain part of the in-universe setting.
The world of Golarian is a (or the?) setting of the Pathfinder role playing game, which is a fork of the D&D 3.5 rules^[1] (but notably different from Forgotten Realms, which is owned by WotC/Hasbro). The core setting is defined in some twenty-odd books which cover everything from the political landscape in dozens of polities to detailed rule for how magic and divine spells work. From what I can tell, the Planecrash authors mostly take what given and fill in the blanks in a way which makes the world logically coherent, in the same way Eliezer did with the world of Harry Potter in HPMOR.
Also, Planecrash (aka Project Lawful) is a form of collaborative writing (or free form roleplaying?) called glowfic. In this case, EY writes Keltham and the setting of dath ilan (which exists only in Keltham’s mind, as far as the plot is concerned), plus a few other minor characters. Lintamande writes Carissa and most of the world—it is clear that she is an expert in the Pathfinder setting—as well as most of the other characters. At some point, other writers (including Alicorn) join and write minor characters. If you have read through planecrash and want to read more from Eliezer in that format, glowfic.com has you covered. For example, here is a story which sheds some light on the shrouded past of dath ilan (plus more BDSM, of course).
1. ^
  D&D 3 was my first exposure to the D&D (via Bioware’s Neverwinter Nights), so it is objectively the best edition. Later editions are mostly WotC milking the franchise with a complete overhaul every few years. AD&D 2 is also fine, if a bit less streamlined—THAC0, armor classes going negative and all that.

quiet_NaN Dec 28, 2024, 4:01 PM
20 points
2
on: Review: Planecrash
One big aspect of Yudkowskian decision theory is how to respond to threats. Following causal decision theory means you can neither make credible threats nor commit to deterrence to counter threats. Yudkowsky endorses not responding to threats to avoid incentivising them, while also having deterrence commitments to maintain good equilibria. He also implies this is a consequence of using a sensible functional decision theory. But there’s a tension here: your deterrence commitment could be interpreted as a threat by someone else, or visa versa.
I have also noted this tension. Intuitively, one might think that it depends on the morality of the action—the robber who threatens to blow up a bank unless he gets his money might be seen as a threat, while a policy of blowing up your own banks in case of robberies might be seen as a deterrence commitment.
However, this can not be it, because decision theory works with any utility functions.
The other idea is that that ‘to make threats’ is one of these irregular verbs. I make credible deterrence commitments, you show a willingness to escalate, they try to blackmail me with irrational threats. This is of course just as silly in the context of game theory.
One axis of difference might be if you physically restrict your own options to prevent you from not following through on your threat (like a Chicken player removing their steering wheel, or that doomsday machine from Dr. Strangelove). But this merely makes a difference for people who are known to follow causal decision theory where they would try to maximize the utility in whatever branch of reality they find themselves in. From my understanding, the adherents of functional decision theory do not need to physically constrain their options—they would be happy to burn the world in one branch of reality if that was the dominant strategy before their opponent had made their choice.
Consider the ultimatum game (which gets convered in Planecrash, naturally) where one party makes a proposal on how to split 10$ and the other party can either accept (gaining their share) or reject it (in which case neither party gains anything). In planecrash, the dominant strategy is presented as rejecting unfair allocations with some probability so that the expected value of the proposing party is lower than if they had proposed a fair split. However, this hinges on the concept of fairness. If each dollar has the same utility to every participant, then a 50-50 split seems fair. But in a more general case, the utilities of both parties might be utterly incomparable, or the effort of both players might be very different—an isomorphic situation is a silk merchant encountering the first of possibly multiple highwaymen, and having to agree on a split, with both parties having the option to burn all the silk if they don’t agree. Agreeing to a 50-50 split each time could easily make the business model of the silk merchant impossible.
“This is my strategy, and I will not change it no matter what, so you better adapt your strategy if you want to avoid fatal outcomes” is an attitude likely to lead to a lot of fatal outcomes.

quiet_NaN Nov 21, 2024, 12:01 PM
6 points
0
on: “It’s a 10% chance which I did 10 times, so it should be 100%”
Relatedly, if you perform an experiment n times, and the probability of success is p, and the expected number of total successes kp is much smaller than one, then kp is a reasonable measure of getting at least once success, because the probability of getting more than one success can be neglected.
For example, if Bob plays the lottery for ten days, and each days has a 1:1000,000 chance of winning, then overall he will have a chance of 100,000 of winning once.
This is also why micromorts are roughly additive: if travelling by railway has a mortality of one micomort per 10Mm, then travelling for 50Mm will set you back 5 micomort. Only if you leave what I would call the ‘Newtonian regime of probability’, e.g. by somehow managing to travel 1Tm with the railway, you are required to do proper probability math, because naive addition would tell you that you will surely have a fatal accident (1 mort) in that distance, which is clearly wrong.

quiet_NaN Oct 27, 2024, 1:28 PM
6 points
1
in reply to: Gunnar_Zarncke’s comment on: The Case For Bullying
Getting down-voted to −27 is an achievement. Most things judged ‘bad AI takes’ only go to −11 or so, even that recent P=NP proof only got to −25. Of course, if the author is right, then downvoting further is providing helpful incentives to him.
I think that bullying is quite distinct from status hierarchies. The latter are unavoidable. There will always be some clique of cool kids in the class who will not invite the non-cool kids to their parties. This is ok. Sometimes, status is correlated with behaviors which are pro-social (kids not smoking; donating to EA), sometimes it is correlated with behaviors which are net-negative (kids smoking; serving in the SS). I was not part of the cool kids circle, and I was fine with that. Live and let live and all that.
‘Bullying’ has a distinct negative connotation. The central example is someone who is targeted for sport for being different from the others. The bullies don’t want the victims to change their ways, they just like to make their life miserable for thrills. I am sure that sometimes it unintentionally helps their victim, if you push enough people, at some point you are bound to push someone out of the path of a car or bullet. In the grand scheme of things, the bullies are however net negative for their victims and society overall.

quiet_NaN Oct 27, 2024, 12:52 PM
5 points
3
on: Arithmetic Models: Better Than You Think
I see this as less of an endorsement of linear models and more of a scathing review of expert performance.
- When an arithmetic model is calibrated, it is specifically by including feedback from the real-world effects of its predictions. Experts do not, as a rule, seek out any feedback on their calibration.
This. Basically, if your job is to do predictions, and the accuracy of your predictions is not measured, then (at least the prediction part of) your job is bullshit.
I think that if you compare simple linear models in domains where people actually care about their predictions, the outcome would be different. For example, if simple models predicted stock performance better than experts at investment banks, anyone with a spreadsheet could quickly become rich. There are few if any cases of ‘I started with Excel and 1000$, and now I am a billionaire’. Likewise, I would be highly surprised to see a simple linear model outperform Nate Silver or the weather forecast.
Even predicting chess outcomes from mid-game board configurations is something where I would expect human experts to outperform simple statistical models working on easily quantifiable data (e.g. number of pieces remaining, number of possible moves, being in check, etc).
Neural networks contained in animal brains (which includes human brains) are quite capable of implementing linear models, and such should at least perform equally well when they are properly trained. A wolf pack deciding to chase or not chase some prey has direct evolutionary skin in the game of making their prediction of success as accurate as possible which the average school counselor predicting academic success simply does not have.
--
You touch this a bit in ‘In defense of explainatory modeling’, but I want to emphasize that uncovering causal relationships and pathways is central to world modelling. Often, we don’t want just predictions, we want predictions conditional on interventions. If you don’t have that, you will end up trying to cure chickenpox with makeup, as ‘visible blisters’ is negatively correlated with outcomes.
Likewise, if we know the causal pathway, we have a much better basis to judge if some finding can be applied to out-of-distribution data. No matter how many anvils you have seen falling, without a causal understanding (e.g. Newtonian mechanics), you will not be able to reliably apply your findings to falling apples or pianos.

quiet_NaN Aug 24, 2024, 11:01 PM
38 points
11
on: How I started believing religion might actually matter for rationality and moral philosophy
What I don’t understand is why there should be a link between trapped priors and an moral philosophy.
I mean, if moral realism was correct, i.e. if moral tenets such as “don’t eat pork”, “don’t have sex with your sister”, or “avoid killing sentient beings” had an universal truth value for all beings capable of moral behavior, then one might argue that the reason why people’s ethics differ is that they have trapped priors which prevent them from recognizing these universal truths.
This might be my trapped priors talking, but I am a non-cognitivist. I simply believe that assigning truth values to moral sentences such as “killing is wrong” is pointless, and they are better parsed as prescriptive sentences such as “don’t kill” or “boo on killing”.
In my view, moral codes are intrinsically subjective. There is no factual disagreement between Harry and Professor Quirrell which they could hope to overcome through empiricism, they simply have different utility functions.
--
My second point is that if moral realism was true, and one of the key roles of religion was to free people from trapped priors so they could recognize these universal moral truths, then at least during the founding of religions, we should see some evidence of higher moral standards before they invariably mutate into institutions devoid of moral truths. I would argue that either, our commonly accepted humanitarian moral values are all wrong or this mutation process happened almost instantly:
- Whatever Jesus thought about gender equality when he achieved moral enlightenment, Paul had his own ideas a few decades later.
- Mohammed was clearly not opposed to offensive warfare.
- Martin Luther evidently believed that serfs should not rebel against their lords.
On the other hand, for instances where religions did advocate for tenets compatible with humanitarianism, such as in Christian abolitionism, do not seem to correspond to strong spiritualism. Was Pope Benedict XIV condemning the slave trade because he was more spiritual (and thus in touch with the universal moral truth) than his predecessors who had endorsed it?
--
My last point is that especially with regard to relational conflicts, our map not corresponding to the territory might often not be a bug, but a feature. Per Hanson, we deceive ourselves so that we can better deceive others. Evolution has not shaped our brains to be objective cognitive engines. In some cases, objective cognition is what it advantageous—if you are alone hunting a rabbit, no amount of self-deception will fill your stomach—but in any social situation, expect evolution to put the hand on the scales of your impartial judgement. Arguing that your son should become the new chieftain because he is the best hunter and strongest warrior is much more effective than arguing for that simply because he is your son—and the best way to argue that is to believe it, no matter if it is objectively true.
The adulterer, the slave owner and the wartime rapist all have solid evolutionary reasons to engage in behaviors most of us might find immoral. I think their moral blind spots are likely not caused by trapped priors, like an exaggerated fear of dogs is. Also, I have no reason to believe that I don’t have similar moral blind spots hard-wired into my brain by evolution.
I would bet that most of the serious roadblocks to a true moral theory (if such a thing existed) are of that kind, instead of being maladaptive trapped priors. Thus, even if religion and spirituality are effective at overcoming maladaptive trapped priors, I don’t see how they would us bring closer to moral cognition.

quiet_NaN Jul 29, 2024, 4:12 AM
2 points
0
on: Universal Basic Income and Poverty
Note: there is an AI audio version of this text over here: https://askwhocastsai.substack.com/p/eliezer-yudkowsky-tweet-jul-21-2024
I find the AI narrations offered by askwho generally ok, worse than what a skilled narrator (or team) could do but much better than what I could accomplish.

quiet_NaN Jul 29, 2024, 4:08 AM
5 points
3
on: Universal Basic Income and Poverty
[...] somehow humanity’s 100-fold productivity increase (since the days of agriculture) didn’t eliminate poverty.
That feels to me about as convincing as saying: “Chemical fertilizers have not eliminated hunger, just the other weekend I was stuck on a campus with a broken vending machine.”
I mean, sure, both the broken vending machine and actual starvation can be called hunger, just as both working 60h/week to make ends meet or sending your surviving kids into the mines or prostituting them could be called poverty, but the implication that either scourge of humankind has not lost most of its terror seems clearly false.
Sure, being poor in the US sucks, but I would rather spend a year living the life of someone in the bottom 10% income bracket in the 2024 US than spending a month living the life of a poor person during the English industrial revolution.
I am also not convinced that 60h/week is what it actually takes to survive in the US. I can totally believe that this amount of unskilled labor might be required to rent accommodations in cities, though.