Well I suppose starting with the assumption that my superintelligent AI is merely turing complete, I think that we can only say our AI has “hypothesis about the world” if it has a computable model of the world. Even if the world weren’t computable, any non-computable model would be useless to our AI, and the best it could do is a computable approximation. Stable time loops seem computable through enumeration as you show in the post.
Now, if you claim that my assumption that the AI is computable is flawed, well then I give up. I truly have no idea how to program an AI more powerful than turing complete.
Even if the world weren’t computable, any non-computable model would be useless to our AI, and the best it could do is a computable approximation.
Again, what distinguishes a “turing oracle” from a finite oracle with a bound well above the realizable size of a computer in the universe? They are indistinguishable hypotheses. Giving a turing complete AI a turing oracle doesn’t make it capable of understanding anything more than turing complete models. The turing-transcendant part must be an integral part of the AI for it to have non-turing-complete hypotheses about the universe, and I have no idea what a turing-transcendant language looks like and even less of an idea of how to program in it.
I don’t see how this changes the possible sense-data our AI could expect. Again, what’s the difference between infinitely many computations being performed in finite time and only the computations numbered up to a point too large for the AI to query being calculated?
If you can give me an example of a universe for which the closest turing machine model will not give indistinguishable sense-data to the AI, then perhaps this conversation can progress.
Well, for starters, an AI living in a universe where infinitely many computations can be performed in finite time can verify the responses a Turing oracle gives it. So it can determine that it lives in a universe with Turing oracles (in fact it can itself be a Turing oracle), which is not what an AI living in this universe would determine (as far as I know).
As mentioned below, we you’d need to make infinitely many queries to the Turing oracle. But even if you could, that wouldn’t make a difference.
Again, even if there was a module to do infinitely many computations, the code I wrote still couldn’t tell the difference between that being the case, and this module being a really good computable approximation of one. Again, it all comes back to the fact that I am programming my AI on a turing complete computer. Unless I somehow (personally) develop the skills to program trans-turing-complete computers, then whatever I program is only able to comprehend something that is turing complete. I am sitting down to write the AI right now, and so regardless of what I discover in the future, I can’t program my turing complete AI to understand anything beyond that. I’d have to program a trans-turing complete computer now, if I ever hoped for it to understand anything beyond turing completeness in the future.
Ah, I see. I think we were answering different questions. (I had this feeling earlier but couldn’t pin down why.) I read the original question as being something like “what kind of hypotheses should a hypothetical AI hypothetically entertain” whereas I think you read the original question as being more like “what kind of hypotheses can you currently program an AI to entertain.” Does this sound right?
I was reading a lesswrong post and I found this paragraph which lines up with what I was trying to say
Some boxes you really can’t think outside. If our universe really is Turing computable, we will never be able to concretely envision anything that isn’t Turing-computable—no matter how many levels of halting oracle hierarchy our mathematicians can talk about, we won’t be able to predict what a halting oracle would actually say, in such fashion as to experimentally discriminate it from merely computable reasoning.
I don’t think that’s different, unless it can also make infinitely many queries of the Turing oracle in finite time. Or make one query of a program of infinite length. In any case, I think it needs to perform infinite communication with the oracle.
I’ll grant that it seems likely that a universe with infinite computation capability will also have infinite communication capability using the same primitives, but I don’t think it’s a logical requirement.
Well I suppose starting with the assumption that my superintelligent AI is merely turing complete, I think that we can only say our AI has “hypothesis about the world” if it has a computable model of the world. Even if the world weren’t computable, any non-computable model would be useless to our AI, and the best it could do is a computable approximation. Stable time loops seem computable through enumeration as you show in the post.
Now, if you claim that my assumption that the AI is computable is flawed, well then I give up. I truly have no idea how to program an AI more powerful than turing complete.
Suppose the AI lives in a universe with Turing oracles. Give it one.
Again, what distinguishes a “turing oracle” from a finite oracle with a bound well above the realizable size of a computer in the universe? They are indistinguishable hypotheses. Giving a turing complete AI a turing oracle doesn’t make it capable of understanding anything more than turing complete models. The turing-transcendant part must be an integral part of the AI for it to have non-turing-complete hypotheses about the universe, and I have no idea what a turing-transcendant language looks like and even less of an idea of how to program in it.
Suppose the AI lives in a universe where infinitely many computations can be performed in finite time...
(I’m being mildly facetious here, but in the interest of casting the “coherently-thinkable” net widely.)
I don’t see how this changes the possible sense-data our AI could expect. Again, what’s the difference between infinitely many computations being performed in finite time and only the computations numbered up to a point too large for the AI to query being calculated?
If you can give me an example of a universe for which the closest turing machine model will not give indistinguishable sense-data to the AI, then perhaps this conversation can progress.
Well, for starters, an AI living in a universe where infinitely many computations can be performed in finite time can verify the responses a Turing oracle gives it. So it can determine that it lives in a universe with Turing oracles (in fact it can itself be a Turing oracle), which is not what an AI living in this universe would determine (as far as I know).
As mentioned below, we you’d need to make infinitely many queries to the Turing oracle. But even if you could, that wouldn’t make a difference.
Again, even if there was a module to do infinitely many computations, the code I wrote still couldn’t tell the difference between that being the case, and this module being a really good computable approximation of one. Again, it all comes back to the fact that I am programming my AI on a turing complete computer. Unless I somehow (personally) develop the skills to program trans-turing-complete computers, then whatever I program is only able to comprehend something that is turing complete. I am sitting down to write the AI right now, and so regardless of what I discover in the future, I can’t program my turing complete AI to understand anything beyond that. I’d have to program a trans-turing complete computer now, if I ever hoped for it to understand anything beyond turing completeness in the future.
Ah, I see. I think we were answering different questions. (I had this feeling earlier but couldn’t pin down why.) I read the original question as being something like “what kind of hypotheses should a hypothetical AI hypothetically entertain” whereas I think you read the original question as being more like “what kind of hypotheses can you currently program an AI to entertain.” Does this sound right?
Yes, I agree. I can imagine some reasoning being concieving of things that are trans-turing complete, but I don’t see how I could make an AI do so.
I was reading a lesswrong post and I found this paragraph which lines up with what I was trying to say
I don’t think that’s different, unless it can also make infinitely many queries of the Turing oracle in finite time. Or make one query of a program of infinite length. In any case, I think it needs to perform infinite communication with the oracle.
I’ll grant that it seems likely that a universe with infinite computation capability will also have infinite communication capability using the same primitives, but I don’t think it’s a logical requirement.
Yes, let’s replace “computations” with “actions,” I guess.