Bugs could potentially result in a new sentient species many millions of years down the line. With super-AI that happens to be non-sentient, there is no such hope.
Nnotm
Karma: 111
Thank you for this, I had listened to the lesswrong audio of the last one just before seeing your comment about making your version, and now waited before listening to this one hoping you would post one
Missed opportunity to replace “if the box contains a diamond” with the more thematically appropriate “if the chest contains a treasure”, though
FWIW the AI audio seems to not take that into account
Thanks, I’ve found this pretty insightful. In particular, I hadn’t considered that even fully understanding static GPT doesn’t necessarily bring you close to understanding dynamic GPT—this makes me update towards mechinterp being slightly less promising than I was thinking.
Quick note:
> a page-state can be entirely specified by 9628 digits or a 31 kB file.
I think it’s a 31 kb file, but a 4 kB file?
I think an important difference between humans and these Go AIs is memory: If we find a strategy that reliably beats human experts, they will either remember losing to it or hear about it and it won’t work the next time someone tries it. If we find a strategy that reliably beats an AI, that will keep happening until it’s retrained in some way.
Are you familiar with Aubrey de Grey’s thinking on this?
To summarize, from memory, cancers can be broadly divided into two classes:
about 85% of cancers rely on lengthening telomeres via telomerase
the other 15% of cancers rely on some alternative lengthening of telomeres mechanism (“ALT”)
The first, big class, can be solved if we can prevent cancers from using telomerase. In his 2007 book “Ending Aging”, de Grey and his co-author Michael Rae wrote about “Whole-body interdiction of lengthening of telomeres” (WILT), which was about using gene therapy to remove telomerase genes in all somatic cells. Then, stem cell therapy could be used to produce new telomeres for those cases where the body usually legitimately uses telomerase.
But research has moved on since then, and this might not be necessary. One promising approach is Maia Biotech’s THIO, a small molecule drug which is incorporated into the telomeres of cancer cells, compromises the telomeres’ structure, and results in rapid cell death. They are currently preparing for phase 2 for a few different clinical trials.
For the other 15% of cancers, the ALT mechanism is as far as I know not as well understood, and I haven’t heard of a promising general approach to cure it. But it seems plausible that it could be a similar affair in the end.
Thanks, I will read that! Though just after you commented I found this in my history, which is the post I meant: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth
I think there was a post/short-story on lesswrong a few months ago about a future language model becoming an ASI because someone asked it to pretend it was an ASI agent and it correctly predicted the next tokens, or something like that. Anyone know what that post was?
Second try: When looking at scatterplots of any 3 out of 5 of those dimensions and interpreting each 5-tuple of numbers as one point, you can see the same structures that are visible in the 2d plot, the parabola and a line—though the line becomes a plane if viewed from a different angle, and the parabola disappears if viewed from a different angle
.
Looking at scatterplots of any 3 out of 5 of those dimensions, it looks pretty random, much less structure than in the 2d plot.
Edit: Oh, wait, I’ve been using chunks of 419 numbers as the dimensions but should be interleaving them
The double line I was talking about is actually a triple line, at indices 366, 677, and 1244. The lines before come from fairly different places, and they diverge pretty quickly afterwards:
However, just above it, there’s another duplicate line, at indices 1038 and 1901:
These start out closer together and also take a little bit longer to diverge.
This might be indicative of a larger pattern that points that are close together and have similar histories tend to have their next steps close to each other as well.
For what it’s worth, colored by how soon in the sequence they appear (blue is early, red is late) (Also note I interpreted it as 2094 points, with each number first used in the x-dimension and then in the y-dimension):
Note that one line near the top appears to be drawn twice, confirming if nothing else that it’s not a rule that it’s not a succession rule that only depends on the previous value, since the paths diverge afterwards.
Still, comparing those two sections could be interesting.
Interpreting the data as unsigned 8-bit integers and plotting it as an image with width 8 results in this (only the first few rows shown):
The rest of the image looks pretty similar. There is a almost continuous high-intensity column (yellow, the second-to-last column), and the values in the first 6 columns repeat exactly in the next row pretty often, but not always.
Very cool, thanks!
The original DALL-E was capable of having almost the same image with slight variations in one generation, so I’d be interested to see something like “A photograph of a village in 1900 on the top, and the same photo colorized on the bottom”.
I haven’t read the luminosity sequence, but I just spent some time looking at the list of all articles seeing if I can spot a title that sounds like it could be it, and I found it: Which Parts are “Me”? - I suppose the title I had in mind was reasonably close.
Is there a post as part of the sequences that’s roughly about how your personality is made up of different aspects, and some of them you consider to be essentially part of who you are, and others (say, for example, maybe the mechanisms responsible for akrasia) you wouldn’t mind dropping without considering that an important difference to who you are?
For years I was thinking Truly Part Of You was about that, but it turns out, it’s about something completely different.
Now I’m wondering if I had just imagined that post existing or just mentally linked the wrong title to it.
One question was whether it’s worth working on anything other than AGI given that AGI will likely be able to solve these problems; he agreed, saying he used to work with 1000 companies at YC but now only does a handful of things, partially just to get a break from thinking about AGI.
In principle, I prefer sentient AI over non-sentient bugs. But the concern that is if non-sentient superintelligent AI is developed, it’s an attractor state, that is hard or impossible to get out of. Bugs certainly aren’t bound to evolve into sentient species, but at least there’s a chance.