The real counting argument that Evan believes in is just a repackaging of Paul’s argument for the malignity of the Solomonoff prior, and not anything novel.
Evan admits that Solomonoff is a very poor guide to neural network inductive biases.
At this point, I’m not sure why you’re privileging the hypothesis of scheming at all.
you want to substitute it out for whatever the prior is that you think is closest to deep learning that you can still reason about theoretically.
I mean, the neural network Gaussian process is literally this, and you can make it more realistic by using the neural tangent kernel to simulate training dynamics, perhaps with some finite width corrections. There is real literature on this.
The real counting argument that Evan believes in is just a repackaging of Paul’s argument for the malignity of the Solomonoff prior, and not anything novel.
I’m going to stop responding to you now, because it seems that you are just not reading anything that I am saying. For the last time, my criticism has absolutely nothing to do with Solomonoff induction in particular, as I have now tried to explain to you here and here and here etc.
I mean, the neural network Gaussian process is literally this, and you can make it more realistic by using the neural tangent kernel to simulate training dynamics, perhaps with some finite width corrections. There is real literature on this.
Yes—that’s exactly the sort of counting argument that I like! Though note that it can be very hard to reason properly about counting arguments once you’re using a prior like that; it gets quite tricky to connect those sorts of low-level properties to high-level properties about stuff like deception.
I know that you think your criticism isn’t dependent on Solomonoff induction in particular, because you also claim that a counting argument goes through under circuit prior. It still seems like you view the Solomonoff case as the central one, because you keep talking about “bitstrings.” And I’ve repeatedly said that I don’t think the circuit prior works either, and why I think that.
At no point in this discussion have you provided any reason for thinking that in fact, the Solomonoff prior and/or circuit prior do provide non-negligible evidence about neural network inductive biases, despite the very obvious mechanistic disanalogies.
Yes—that’s exactly the sort of counting argument that I like!
Then make an NNGP counting argument! I have not seen such an argument anywhere. You seem to be alluding to unpublished, or at least little-known, arguments that did not make their way into Joe’s scheming report.
So today we’ve learned that:
The real counting argument that Evan believes in is just a repackaging of Paul’s argument for the malignity of the Solomonoff prior, and not anything novel.
Evan admits that Solomonoff is a very poor guide to neural network inductive biases.
At this point, I’m not sure why you’re privileging the hypothesis of scheming at all.
I mean, the neural network Gaussian process is literally this, and you can make it more realistic by using the neural tangent kernel to simulate training dynamics, perhaps with some finite width corrections. There is real literature on this.
I’m going to stop responding to you now, because it seems that you are just not reading anything that I am saying. For the last time, my criticism has absolutely nothing to do with Solomonoff induction in particular, as I have now tried to explain to you here and here and here etc.
Yes—that’s exactly the sort of counting argument that I like! Though note that it can be very hard to reason properly about counting arguments once you’re using a prior like that; it gets quite tricky to connect those sorts of low-level properties to high-level properties about stuff like deception.
I’ve read every word of all of your comments.
I know that you think your criticism isn’t dependent on Solomonoff induction in particular, because you also claim that a counting argument goes through under circuit prior. It still seems like you view the Solomonoff case as the central one, because you keep talking about “bitstrings.” And I’ve repeatedly said that I don’t think the circuit prior works either, and why I think that.
At no point in this discussion have you provided any reason for thinking that in fact, the Solomonoff prior and/or circuit prior do provide non-negligible evidence about neural network inductive biases, despite the very obvious mechanistic disanalogies.
Then make an NNGP counting argument! I have not seen such an argument anywhere. You seem to be alluding to unpublished, or at least little-known, arguments that did not make their way into Joe’s scheming report.