Blake is not reliable here. I work with LaMDA, and while I can’t say much specific other than what’s published (which directly refutes him), he is not accurate here.
Interesting! I do wish you were able to talk more openly about this (I think a lot of confusion is coming from lack of public information about how LaMDA works), but that’s useful to know at least. Is there any truth to the claim that there’s any real-time updating going on, or is that false as well?
I do wish you were able to talk more openly about this (I think a lot of confusion is coming from lack of public information about how LaMDA works)
Just chiming to say that I’m always happy to hear about companies sharing less information about their ML ideas publicly (regardless of the reason!), and I think it would be very prosocial to publish vastly less.
This is an excellent point actually, though I’m not sure I fully agree (sometimes lack of information could end up being far worse, especially if people think we’re further along than we really are, and try to get into an “arms race” of sorts)
Blake is not reliable here. I work with LaMDA, and while I can’t say much specific other than what’s published (which directly refutes him), he is not accurate here.
Three questions here; obviously you can only answer what you can answer, but I figure it doesn’t hurt to ask.
Does LaMDA contain any kind of cycle in its computation graph, such as a recurrence or iterative process?
Does LaMDA have any form of ‘medium-term’ memory (shorter-term than weights but longer-term than activations)?
Does LaMDA (or, to what extent does LaMDA) make use of non-NN tools such as internet/database searches, calculators, etc?
No, no, yes according to the LaMDA paper. :)
See Gwern’s comment below for more detail on #3.
Interesting! I do wish you were able to talk more openly about this (I think a lot of confusion is coming from lack of public information about how LaMDA works), but that’s useful to know at least. Is there any truth to the claim that there’s any real-time updating going on, or is that false as well?
Just chiming to say that I’m always happy to hear about companies sharing less information about their ML ideas publicly (regardless of the reason!), and I think it would be very prosocial to publish vastly less.
This is an excellent point actually, though I’m not sure I fully agree (sometimes lack of information could end up being far worse, especially if people think we’re further along than we really are, and try to get into an “arms race” of sorts)