The issue arises specifically in the situation of recursive self-improvement: You can’t prove self-consistency in mathematical frameworks of “sufficient complexity” (that is, containing the rules of arithmetic in a provable manner).
What this cashes out to is that, considering AI as a mathematical framework, and the next generation of AI (designed by the first) as a secondary mathematical framework—you can’t actually prove that there are no contradictions in an umbrella mathematical framework that comprises both of them, if they are of “sufficient complexity”. Which means an AI cannot -prove- that a successor AI has not experienced value drift—that is, that the combined mathematical framework does not contain contradictions—if they are of sufficient complexity.
To illustrate the issue, suppose the existence of a powerful creator AI, designing its successor; the successor, presumably, is more powerful than the creator AI in some fashion, and so there are areas of the combinatorially large space that the successor AI can explore (in a reasonable timeframe), but that the creator AI cannot. If the creator can prove there are no contradictions in the combined mathematical framework—then, supposing its values are embedded in that framework in a provable manner, it can be assured that the successor has not experienced value drift.
Mind, I don’t particularly think the above scenario is terribly likely; I have strong doubts about basically everything in there, in particular the idea of provable values. I created the post for the five or six people who might still be interested in ideas I haven’t seen kicked around on Less Wrong for over a decade.
Another crackpot physics thing:
My crackpot physics just got about 10% less crackpot. As it transpires, one of the -really weird- things in my physics, which I thought of as a negative dimension, already exists in mathematics—it’s a Riemann Sphere. (Thank you, Pato!)
This “really weird” thing is kind of the underlying topology of the universe in my crackpot physics—I analogized the interaction between this topology and mass once to an infinite series of Matryoshka dolls, where every other doll is “inside out and backwards”. Don’t ask me to explain that; that entire avenue of “attempting to communicate this idea” was a complete and total failure, and it was only after drawing a picture of the topology I had in mind that someone (Pato) observed that I had just drawn a somewhat inaccurate picture of a Riemann Sphere. (I drew it as a disk in which the entire boundary was the same point, 0, with dual infinities coinciding at the origin. I guess, in retrospect, a sphere was a more obvious way of describing that.)
If we consider that the points are not evenly allocated over the surface of the sphere—they’re concentrated at the poles (each of which is simultaneously 0 and infinity, the mapping is ambiguous), if we drew a line such that it crosses the same number of points with each revolution, we get—something like a logarithmic spiral. (Well, it’s a logarithmic spiral with the “disk” interpretation; it’s a spherical spiral whose name I don’t know in the spherical interpretation.)
If we consider the bundle of lines connecting the poles, and use this constant-measure-per-revolution spiral to describe their path, I think that’s about … 20% of the way to actually converting the insanity in my head into real mathematics. Each of these lines is “distance” (or, alternatively, “time”—it depends on which of the two charts you employ). The bundle of these spirals provides one dimension of rotation; there’s a mathematical way of extracting a second dimension of rotation, to get a three-dimensional space, but I don’t understand it at an intuitive level yet.
A particle’s perspective is “constantly falling into an infinity”; because of the hyperbolic nature of the space, I think a particle always “thinks” it is at the equator—it never actually gets any closer. Because the lines describe a spiral, the particle is “spinning”. Because of the nature of the geometry of the sphere, this spin expresses itself as a spinor, or at least something analogous to one.
Also, apparently, Riemann Spheres are already used in both relativistic vacuum field equations and quantum mechanics. Which, uh, really annoys me, because I’m increasingly certain there is “something” here, and increasingly annoyed that nobody else has apparently just sat down and tried to unify the fields in what, to me, is the most obvious bloody way to unify them; just assume they’re all curvature, that the curvature varies like a decaying sine wave (like “sin(ln(x))/x”, which exhibits exactly the kind of decay I have in mind). Logarithmic decay of frequency over distance ensures that there is a scalar symmetry, as does a linear decay of amplitude over distance.
Yes, I’m aware of the intuitive geometry involved in an inverse-square law; I swear that the linear decay makes geometric sense too, given the topology in my head. Rotation of a logarithmic spiral gives rise to a linear rescaling relative to the arclength of that rotation. Yes, I’m aware that the inverse-square law also has lots of evidence—but it also has lots of evidence against it, which we’ve attempted to patch by assuming unobserved mass that precisely accounts for the observed anomalies. I posit that the sinusoidal wave in question has ranges wherein the amplitude is decaying approximately linearly, which creates the apparent inverse-square behavior for certain ranges of distances—and because these regions of space are where matter tends to accumulate, having the most stable configurations, they’re disproportionately where all of our observations are made. It’s kind of literally the edge cases, where the inverse-square relationship begins to break down (whether it really does, or apparently does), and the configurations become less stable, that we begin to observe deviations.
I’m still working on mapping my “sin(ln(x))/x” equation (this is not the correct equation, I don’t think, it’s just an equation that kind of looks right for what’s in my head, and it gave me hints about where to start looking) to this structure; there are a few options, but none stand out yet as obviously correct. The spherical logarithmic spiral is a likely candidate, but figuring out the definition of the spiral that maintains constant “measure” with each rotation requires some additional understanding on my part.