Foyle

Karma: 144

Foyle 30 Jan 2025 9:01 UTC
3 points
2
on: Fertility Will Never Recover
Not worth worrying about given context of imminent ASI.
But assuming a Butlerian jihad occurs to make it an issue of importance again then most topics surrounding it are gone into at depth by radical pro-natalists Simone and Malcom Gladwell, who have employed genetic screening of their embryos to attempt to have more high-achievers, on their near-daily podcast https://www.youtube.com/user/simoneharuko . While quite odd in their outlook they delve into all sorts of sociopolitical issues from the pronatalist worldview. Largely rationalist and very interesting and informative, though well outside of Overton window on a lot of subjects.

Foyle 20 Jan 2025 7:26 UTC
1 point
0
on: It is (probably) time for a Buterlian Jihad
The phone seems to be off the hook for most of public on AI danger, perhaps a symptom of burnout from numerous other scientific Millenialist scares—people have been hearing of imminent dangers of catastrophe for decades that have failed to impact the lives of 95%+ of population in any significant way and now just write it all off as more of the same.
I am sure that most LW readers find little in the way of positive reception for our concerns amongst less technologically engaged family members and acquaintances. There are just too many comforting techno-utopian narratives that we are still having to compete with informed by the superficially positive representations in movies and TV, and and most people bias towards optimism in relatively good/comfortable times like these. We are dealing with emotional reactions and the sheeply ‘vibe’ of population rather than thoughtfully considered positions.
But I am pretty sure that will all change as soon as we see significant AI competition/undercutting for white collar professions. Those affected will quickly start baying for blood, and the electorally dominant empathic response to those sad stories of the economically impacted will rapidly swing the more emotively governed wider population against AI. OECD Democratic govts will inevitably then move to ban AI taking jobs of particularly politically protected classes of people (might still leave some niches vulnerable—like medicine where there is always a shortage of service supply causing ridiculously high prices, and perhaps tech for vengeful reasons). It will be Butlerian Jihad lite, aimed at symptoms rather than causes, and will likely buy us a few years of relative normalcy as more dangerous ASI is developed in govt approved labs and by despotic regimes.
I doubt it will save us in 50 year time frame, but will perhaps make the economic disruption less severe for 5-10 years.
The way to have a bigger impact in the shorter term would be to buy AI-danger editorial support from influencers, particularly those with young female audiences that form the core of environmental and other popular protest movements. They are by temperament the easiest to bring on board, and have outsized poltical influence.

Foyle 15 Jan 2025 10:24 UTC
3 points
0
on: Why do futurists care about the culture war?
Agree that most sociological, economic and environmental problems that loom large in current context will radically shift in importance in next decade or two, to the point that they are probably no longer worth devoting any significant resources to in the present. Impacts of AI are only issue worth worrying about. But even assuming utopian outcomes; who gets possession of the Malibu beach houses in post scarcity world?
Once significant white-collar job losses start to mount in a year or two I think it inevitable that a powerful and electorally dominant anti-AI movement will grow, at least in erstwhile democracies, and likely ban most AGI applications outside of a few fields where fewer workers would stand to lose jobs (health—with near endless demand, perhaps cutting edge tech where payoff to human net welfare is highest). Butlerian Jihad-lite.
It won’t save us, and has substantial risk of ushering in repressive authoritarianism in the political ruckus caused but will likely delay our demise or (at best) delivery into powerless pet status by perhaps a decade or two.

Foyle 28 Dec 2024 4:15 UTC
3 points
0
in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Perlman’s Shortform
This is depressing, but not surprising. We know the approximate processing power of brains (O(1e16-1e17flops) and how long it takes to train them, and should expect that over the next few years the tricks and structures needed to replicate or exceed that efficiency in ML will be uncovered in an accelerating rush towards the cliff as computational resources needed to attain commercially useful performance continue to fall. AI Industry can afford to run thousands of experiments at this cost scale.
Within a few years this will likely see AGI implementations on Nvidia B200 level GPUS (~1e16flop). We have not yet seen hardware application of the various power-reducing computational ‘cheats’ for mimicking multiplication with reduced gate counts that are likely to see a 2-5x performance gain at same chip size and power draw.
Humans are so screwed.

Foyle 21 Dec 2024 7:41 UTC
9 points
2
in reply to: Thane Ruthenis’s comment on: o3
A very large amount of human problem solving/innovation in challenging areas is creating and evaluating potential solutions, it is a stochastic rather than deterministic process. My understanding is that our brains are highly parallelized in evaluating ideas in thousands of ‘cortical columns’ a few mm across (Jeff Hawkin’s 1000 brains formulation) with an attention mechanism that promotes the filtered best outputs of those myriad processes forming our ‘consciousness’.
So generating and discarding large numbers of solutions within simpler ‘sub brains’, via iterative, or parallelized operation is very much how I would expect to see AGI and SI develop.

Foyle 21 Dec 2024 7:27 UTC
5 points
1
in reply to: Noosphere89’s comment on: o3
I think Elon will bring strong concern about AI to fore in current executive—he was an early voice for AI safety though he seems too have updated to a more optimistic view (and is pushing development through x-AI) he still generally states P(doom) ~10-20%. His antipathy towards Altman and Google founders is likely of benefit for AI regulation too—though no answer for the China et al AGI development problem.

Foyle 5 Dec 2024 3:37 UTC
1 point
0
in reply to: sunwillrise’s comment on: dirk’s Shortform
The era of AGI means humans can no longer afford to live in a world of militarily competing nations. Whatever slim hope there might be for alignment and AI not-kill-everyone is sunk by militaries trying to out-compete each other in development of creatively malevolent and at least somewhat unaligned martial AI. At minimum we can’t afford non-democratic or theocratically ruled nations, or even nations with unaccountable power-unto-themselves military, intelligence or science bureaucracies to control nukes, pathogen building biolabs or AGI. It will be necessary to enforce this even at the cost of war.

Foyle 14 Nov 2024 6:46 UTC
3 points
0
in reply to: AnthonyC’s comment on: Why would ASI share any resources with us?
Humans as social animals have a strong instinctual bias towards trust of con-specifics in prosperous times. Which makes sense from a game theoretic strengthen-the-tribe perspective. But I think that leaves us, as a collectively dumb mob of naked apes, entirely lacking a sensible level of paranoia in the building ASI that has no existential need for pro-social behavior.
The one salve I have for hopelessness is that perhaps the Universe will be boringly deterministic and ‘samey’ enough that ASI will find it entertaining to have agentic humans wandering around doing their mildly unpredictable thing. Although maybe it will prefer to manufacture higher levels of drama (not good for our happiness)

Foyle 14 Nov 2024 6:35 UTC
7 points
0
on: Thoughts after the Wolfram and Yudkowsky discussion
It was a very frustrating conversation to listen to, because Wolfram really hasn’t engaged his curiosity and done the reading on AI-kill-everyoneism. So we just got a torturous number of unnecessary and oblique diversions from Wolfram who didn’t provide any substantive foil to Eliezer
I’d really like to find Yudkowsky debates with better prepared AI optimists prepared to try and counter his points. Do any exist?

Foyle 7 Nov 2024 5:15 UTC
2 points
0
on: An alternative approach to superbabies
It seems unlikely to me that there is potential to make large brain based intelligence advancements beyond the current best humans using human evolved biology. There will be distance scaling limitations linked to neural signal speeds.
Then there is Jeff Hawkins ‘thousand brains’ theory of human intelligence that our brains are made up of thousands of parallel processing cortical columns of a few mm cross section and a few mm thick with cross communication and recursion etc, but that fundamental processing core probably isn’t scalable in complexity, only in total number—your brain could perhaps be expanded to handle thinking about more things in parallel at once, but not at much higher levels of sophistication without paying a large coordination speed price (and evolution places a premium on reaction speed for animals that encounter violence)
I look at whales and other mammals with much much larger than human brains and wonder why they are not smarter—some combination of no evolutionary driver and perhaps a lot of their neurons are dedicated to delay-line processing needed for processing sonar and controlling large bodies with long signaling delays.
Regardless, if AI is a dominant part of our future then it seems likely to me that regardless of whether the future is human utopia or dystopia, non-transhuman humans will not exist in significant numbers in a few hundred years. Neural biology and perhaps all biology is going to be superseded as maladapted to the technological future.

Foyle 7 Nov 2024 4:47 UTC
0 points
3
on: Should CA, TX, OK, and LA merge into a giant swing state, just for elections?
Are any of the socio-economic-political-demographic problems of the world actually fixable or improvable in the time before the imminent singularity renders them all moot anyway? It all feels like bread-and-circuses to me.
The pressing political issues of today are unlikely to even be in the top-10 in a decade.

Foyle 22 Oct 2024 12:38 UTC
3 points
0
on: Sleeping on Stage
Fantastic life skill to be able to sleep in a noise environment on a hard floor. Most Chinese can do it so easily, and I would frequently less kids anywhere up to 4-5 years old being carried sleeping down the road by guardians.
I think super valuable when it comes to adulthood and sharing a bed—one less potential source of difficulties if adaption to noisy environment when sleeping makes snoring a non-issue.

Foyle 22 Oct 2024 12:31 UTC
4 points
0
in reply to: AnthonyC’s comment on: What’s a good book for a technically-minded 11-year old?
It is the literary, TV and movie references, a lot of stuff also tied to technology and social developments of the 80′s-00′s (particularly Ank-Morpork situated stories) and a lot of classical and allusions. ‘Education’ used to lean on common knowledge of a relatively narrow corpus of literature and history Shakespeare, chivalry, European history, classics etc for the social advantage those common references gave and was thus fed to boomers and gen-x, y but I think it’s now rapidly slipping into obscurity as few younger people read and schools shift away from teaching it in face of all that’s new in the world. I guess there are a lot of jokes that pre-teens will get, but so many that they will miss. Seems a waste of such delightful prose.

Foyle 22 Oct 2024 12:17 UTC
4 points
0
in reply to: Martin Sustrik’s comment on: What’s a good book for a technically-minded 11-year old?
Yeah, powering through it. I’ve tried adult Fiction and Sci-Fi but he’s not interested in it yet—not grokking adult motivations, attitudes and behaviors yet, so feeding him stuff that he enjoys to foster habit of reading.

Foyle 20 Oct 2024 9:35 UTC
5 points
0
on: What’s a good book for a technically-minded 11-year old?
I’ve just started my 11yr old tech minded son reading the Worm web serial by John Macrae (free and online, longer than Harry potter series). It’s a bit grim/dark and violent, but an amazing and compelling sci-fi meditation on superheroes and personal struggles. A more brutal and sophisticated world build along lines of popular ‘my hero academia’ anime that my boys watched compulsively. 1000′s of fanfics too.
Stories from Larry Niven’s “known space” universe. Lots of fun overcoming-challenges short stories and novellas that revolve around interesting physics or problems or ideas. And the follow up Man-Kzin War series by various invited authors have some really great stories too with a strong martial bent that will likely appeal to most boys.
At that age I read and loved Dune, The stars my destination (aka Tiger Tiger, a sci fi riff on Comte de Monte Christo), Enders Game. I think Terry Pratchett humor needs a more sophisticated adult knowledge base, with culture references that are dating badly.
My 11yr old loved the Expanse TV series, though I haven’t given them the books to read yet and I can’t recommend the transhumanism anime Pantheon on Amazon highly enough—its one of best sci fi series of all time.
All good to introduce more adult problems and thinking to kids in an exciting context.

Foyle 18 Oct 2024 3:23 UTC
4 points
0
on: You’re Playing a Rough Game
We definitely want our kids involved in at times painful activities as a means of increasing confidence, fortitude and resilience against future periods of discomfort to steel them against the trials of later life. A lot of boys will seek it out as a matter of course in hobby pursuits including martial arts.
I think there is also value in mostly not interceding in conflicts unless there is an established or establishing pattern of physical abuse. Kids learn greater social skills and develop greater emotional strength when they have to deal with the knocks and unfairness themselves, and rewarding tattle-tailing type behavior with the exercise of parental power (or even attention) over the reported perpetrator creates some probably not-good crutch-like dynamics in children’s play stunting their learning of social skills.
I think it’s generally not good for kids to have power over others even if that power is borrowed, as it often enables maliciousness in kids that are (let’s face it) frequently little sociopaths trying to figure out how to gain power over others until they start developing more empathy in their teens. Their play interactions should be negotiated between them, not imposed by outside agents. Feign disinterest in their conflicts unless you see toxic dynamics forming. They should sort things out amongst themselves as much as possible.
For my boys (9,11) I’ll only intercede if they are getting to the point of physical harm or danger, or if there is a violent response to an accidental harm (must learn to control violent/vengeful impulses). But they frequently wrestle with each other in play. It is a challenge to balance with my 7 daughter though as lacking physical strength of her older brothers she works much harder to use parents as proxies to fight her conflicts.
Less cotton wool and helicopter parenting is mostly good.

Foyle 17 Oct 2024 4:51 UTC
3 points
−2
on: Against empathy-by-default
“In many cases, however, evolution actually reduces our native empathic capacity—for instance, we can contextualize our natural empathy to exclude outgroup members and rivals.”
Exactly as it should be.
Empathy is valuable in close community settings, a ‘safety net’ adaption to make the community stronger with people we keep track of to ensure we are not being exploited by people not making concomitant effort to help themselves. But it seems to me that it is destructive at wider social scales enabled by social media where we don’t or can’t have effective reputation tracking to ensure that we are not being ‘played’ for the purpose of resource extraction by people making dishonest or exaggerated representations.
In essence at larger scales the instinct towards empathy rewards dishonest, exploitative, sociopathic and narcissistic behavior in individuals, and is perhaps responsible for a lot of the deleterious aspects of social media amongst particularly more naturally or generally empathic-by-default women. Eg ‘influencers’ (and before them exploitative televangelists) cashing in on follower empathy. It also rewards misrepresentations of victimhood/suffering for attention and approval—again in absence of more in depth knowledge of the person that would exist in a smaller community—that may be a source of rapid increase in ‘social contagion’ mental health pathologies amongst particularly young women instinctually desirous of attention most easily attained by inventing of exaggerating issues in absence of other attributes that might garner attention.
In short the empathic charitable instinct that works so well in families and small groups is socially destructive and dysfunctional at scales beyond community level.

Foyle 9 Oct 2024 0:23 UTC
2 points
−6
on: Overview of strong human intelligence amplification methods
I read some years ago that average IQ of kids is approximately 0.25*(Mom IQ + Dad IQ + 2x population mean IQ). So simplest and cheapest means to lift population average IQ by 1 standard deviation is just use +4 sd sperm (around 1 in 30000), and high IQ ova if you can convince enough genius women to donate (or clone, given recent demonstration of male and female gamete production from stem cells). +4sd mom+dad = +2sd kids on average. This is the reality that allows ultra-wealthy dynasties to maintain ~1.3sd IQ average advantage over general population by selecting (attractive/exciting) +4sd mates.
Probably the simplest and cheapest thing you can do to lift population IQ over long term is to explain this IQ-heritability reality to every female under the age of 40, make it common knowledge and a lot of them will choose genius sperm for themselves.
Beyond that intervention that can happen immediately there is little point in trying to do anything. In 20 years when ASIs stradle the earth like colossuses, and we are all their slaves or pets, they will (likely in even best case scenarios) be dictating our breeding and culling—or casually ignoring/exterminating us. In optimistic event of Banksian Culture like post-singularity utopia magic ASI tech will be developed to near-universally optimize human neural function via nootropics or genetic editing to reach a peak baseline (or domesticate us into compliant meat-bots). I think even a well aligned ASI is likely to push this on us.

Foyle 5 Oct 2024 9:51 UTC
7 points
6
on: If I have some money, whom should I donate it to in order to reduce expected P(doom) the most?
I think there is far too much focus on technical approaches, when what is needed is a more socio-political focus. Raising money, convincing deep pockets of risks to leverage smaller sums, buying politicians, influencers and perhaps other groups that can be coopted and convinced of existential risk to put a halt to Ai dev.
It amazes me that there are huge, well financed and well coordinated campaigns for climate, social and environmental concerns, trivial issues next to AI risk, and yet AI risk remains strictly academic/fringe. What is on paper a very smart community embedded in perhaps the richest metropolitan area the world has ever seen, has not been able to create the political movement needed to slow things up. I think precisely because they pitching to the wrong crowd.
Dumb it down. Identify large easily influenceable demographics with a strong tendency to anxiety that can be most readily converted—most obviously teenagers, particularly girls and focus on convincing them of the dangers, perhaps also teachers as a community—with their huge influence. But maybe also the elederly—the other stalwart group we see so heavily involved in environmental causes. It would have orders of magnitude more impact than current cerebral elite focus, and history is replete with revolutions borne out of targeting conversion of teenagers to drive them.

Foyle 11 Sep 2024 8:55 UTC
3 points
0
in reply to: Daniel Kokotajlo’s comment on: Self-driving car bets
They cannot just add an OOM of parameters, much less three.
How about 2 OOM’s?
HW2.5 21Tflops HW3 72x2 = 72 Tflops (redundant), HW4 3x72=216Tflops (not sure about redundancy) and Elon said in June that next gen AI5 chip for fsd would be about 10x faster say ~2Pflops
By rough approximation to brain processing power you get about 0.1Pflop per gram of brain so HW2.5 might have been a 0.2g baby mouse brain, HW3 a 1g baby rat brain HW4 perhaps adult rat, and upcoming HW5 a 20g small cat brain.
As a real world analogue cat to dog (25-100g brain) seems to me the minimum necessary range of complexity based on behavioral capabilities to do a decent job of driving—need some ability to anticipate and predict motivations and behavior of other road users and something beyond dumb reactive handling (ie somewhat predictive) to understand anomalous objects that exist on and around roads.
Nvidia Blackwell B200 can do up to about 10pflops of FP8, which is getting into large dog/wolf brain processing range, and wouldn’t be unreasonable to package in a self driving car once down closer to manufacturing cost in a few years at around 1kW peak power consumption.
I don’t think the rat brain HW4 is going to cut it, and I suspect that internal to Tesla they know it too, but it’s going to be crazy expensive to own up to it, better to keep kicking the can down the road with promises until they can deliver the real thing. AI5 might just do it, but wouldn’t be surprising to need a further oom to Nvidia Blackwell equivalent and maybe $10k extra cost to get there.