weverka

Karma: 105

weverka 22 Nov 2023 1:04 UTC
2 points
0
in reply to: EJT’s comment on: The Shutdown Problem: Three Theorems
thank you.

weverka 20 Nov 2023 13:45 UTC
−1 points
0
in reply to: weverka’s comment on: The Shutdown Problem: Three Theorems
Why down votes and a statement that I am wrong because I misunderstood.
This is a mean spirited reaction when I lead with admission that I could not follow the argument. I offered a concrete example and stated that I could not follow the original thesis as applied to the concrete example. No one took me up on this.
Are you too advanced to stoop to my level of understanding and help me figure out how this abstract reasoning applies to a particular example? Is the shut down mechanism suggested by Yudkowsky too simple?

weverka 23 Oct 2023 18:11 UTC
−13 points
−6
on: The Shutdown Problem: Three Theorems
I tried to follow with a particular shut down mechanism in mind and this whole argument is just too abstract to see how it applies.
Yudkowsky gave us a shut down mechanism in his Time Magazine article. He said we could bomb* the data centers. Can you show how these theorems cast doubt on this shut down proposal?
*”destroy a rogue datacenter by airstrike.”—https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

weverka 9 Jun 2023 20:34 UTC
3 points
0
on: Transformative AGI by 2043 is <1% likely
Compute is not the limiting factor for mammalian intelligence. Mammalian brains are organized to maximize communication. The gray matter, where most compute is done, is mostly on the surface and the white matter which dominate long range communication, fills the interior, communicating in the third dimension.
If you plot volume of white matter vs. gray matter across the various mammal brains, you find that the volume of white matter grows super linearly with volume of gray matter. https://www.pnas.org/doi/10.1073/pnas.1716956116
As brains get larger, you need a higher ratio of communication/compute.
Your calculations, and Cotras as well, focus on FLOPs but the intelligence is created by communication.

weverka 28 May 2023 13:43 UTC
1 point
on: Hard Takeoff
dy/dt = f(y) = m*y whose solution is the compound interest exponential, y = e^(m*t).
Why not estimate m?
An exactly right law of diminishing returns that lets the system fly through the soft takeoff keyhole is unlikely.
This blog post contains a false dichotomy. In the equation, m can take any value and there is no special keyhole value, and there is no line between fast and slow.
The description in the subsequent discussion is a distraction. The posted equation is meaningful only if we have an estimate of the growth rate.

weverka 1 Apr 2023 14:55 UTC
1 point
0
in reply to: ZankerH’s comment on: All AGI Safety questions welcome (especially basic ones) [~monthly thread]
We don’t have to tell it about the off switch!

weverka 8 Feb 2023 13:59 UTC
0 points
0
in reply to: WilliamKiely’s comment on: A BOTEC estimating the effects of an AI capabilities project on AI timelines, unaligned AI, and human extinction
You said nothing about positive contributions. When you throw away the positives, everything is negative.

weverka 6 Feb 2023 14:13 UTC
3 points
2
on: A BOTEC estimating the effects of an AI capabilities project on AI timelines, unaligned AI, and human extinction
Why didn’t you also compute the expectation this project contributes towards human flourishing?
If you only count the negative contributions, you will find that the expectation value of everything is negative.

weverka 5 Feb 2023 14:30 UTC
11 points
8
on: A BOTEC estimating the effects of an AI capabilities project on AI timelines, unaligned AI, and human extinction
The ML engineer is developing an automation technology for coding and is aware of AI risks . The engineers polite acknowledgment of the concerns is met with your long derivation of how many current and future people she will kill with this.
Automating an aspect of coding is part of a long history of using computers to help design better computers, starting with Carver Mead’s realization that you don’t need humans to cut rubylith film to form each transistor.
You haven’t shown an argument that this project will accelerate the scenario you describe. Perhaps the engineer is brushing you off because your reasoning is broad enough to apply to all improvements in computing technology. You will get more traction if you can show more specifically how this project is “bad for the world”.

weverka 29 Jan 2023 17:09 UTC
6 points
on: The AI Timelines Scam
The missile gap was a lie by the US Air Force to justify building more nukes, by falsely claiming that the Soviet Union had more nukes than the US
This statement is not supported by the link used as a reference. Was it a lie? The reference speaks to failed intelligence and political manipulation using the perceived gap. The phrasing above suggests conspiracy.

weverka 26 Jan 2023 22:28 UTC
1 point
0
on: All AGI Safety questions welcome (especially basic ones) [~monthly thread]
Why doesn’t an “off switch” protect us?

weverka 24 Jan 2023 19:13 UTC
1 point
0
on: Trends in GPU price-performance
You have more than an order of magnitude scatter in your plot, but you write 3 significant figures to your calculated doubling period. Is this precision of value?
Also, your black data appears to have something different going on prior to 2008. It would be worthwhile doing a separate fit to post 2008 data. Eyeballing it, it is longer than 4 year doubling time.

weverka 20 Jan 2023 19:59 UTC
0 points
1
on: We don’t trade with ants
AI is dependent on humans. It gets power and data from humans and it cannot go on without humans. We don’t trade with it, we dictate terms.
Do we fear a world where we have turned over mining, production and powering everything to the AI. Getting there would take a lot more than self amplifying feedback loop of a machine rewriting its own code.

weverka 11 Jan 2023 20:06 UTC
6 points
0
on: Running With a Backpack
When I was doing runs in the dozens of miles, I found it better to cache water ahead of time at the ten mile points. On a hot day, you need more water than you can comfortably carry.

weverka 10 Jan 2023 14:06 UTC
3 points
in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Median Future)
Ok, I could be that someone. here goes. You and the paper author suggest a heat engine. That needs a cold side and a hot side. We build a heat engine where the hot side is kept hot by the incoming energy as described in this paper. The cold side is a surface we have in radiative communication with the 3 degrees Kelvin temperature of deep space. In order to keep the cold side from melting, we need to keep it below a few thousand degrees, so we have to make it really large so that it can still radiate the energy.
From here, we can use Stefan–Boltzmann law, to show that we need to build a radiator much bigger than a billion times the surface area of Mercury. It goes as the fourth power of the ratio of temperatures in our heat engine.
The paper’s contribution is the suggestion of a self replicating factory with exponential growth. That is cool. But the problem with all exponentials is that, in real life, they fail to grow indefinitely. Extrapolating an exponential a dozen orders of magnitude, without entertaining such limits, is just silly.

weverka 9 Jan 2023 13:25 UTC
1 point
in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Median Future)
A billion times the energy flux from the surface of the sun, over any extended area is a lot to deal with. It is hard to take this proposal seriously.

weverka 8 Jan 2023 19:14 UTC
11 points
0
in reply to: gwern’s comment on: What 2026 looks like (Daniel’s Median Future)
For the record, I find that scientists make such errors routinely. In public conferences when optical scientists propose systems that violate the constant radiance theorem, I have no trouble standing up and saying so. It happens often enough that when I see a scientist propose such a system, It does not diminish my opinion of that scientist. I have fallen into this trap myself at times. Making this error should not be a source of embarrassment.
either way, you are claiming Sandberg, a physicist who works with thermodynamic stuff all the time, made a trivial error of physics;
I did not expect this to revert to credentialism. If you were to find out that my credentials exceed this other guy’s, would you change your position? If not, why appeal to credentials in your argument?

weverka 3 Jan 2023 17:30 UTC
3 points
in reply to: gwern’s comment on: [Preprint] The Computational Limits of Deep Learning
I stand corrected. Please forgive me.

weverka 2 Jan 2023 21:50 UTC
8 points
3
on: Why don’t Rationalists use bidets?
Last summer I went on a week long backpacking trip where we had to carry out all our used toilet paper.
This year, I got this bidet for Christmas: https://www.garagegrowngear.com/products/portable-bidet-by-culoclean
You could carry one with you so you are no longer reliant on having them provided for you.

weverka 25 Dec 2022 15:50 UTC
4 points
0
in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Median Future)
How much extra energy external energy is required to get an energy flux on Mercury of a billion times that leaving the sun? I have an idea, but my statmech is rusty. (the fourth root of a billion?)
And do we have to receive the energy and convert it to useful work with 99.999999999% efficiency to avoid melting the apparatus on Mercury?