Is that in doubt? Note that I don’t say it models the base objective in the post, I just say that it has a complex world model. This seemed unquestionable to me since it demonstrably knows lots of things. Or are you drawing a distinction between “a lot of facts about stuff” and ” a world model?” I haven’t draw that; “model” seems very general and “complex” trivially true. It may not be a smart model.
Part of me thinks: I was trying to push on whether it has a world model or rather has just memorised loads of stuff on the internet and learned a bunch of heuristics for how to produce compelling internet-like text. For me, “world model” evokes some object that has a map-territory relationship with the world. It’s not clear to me that GPT-3 has that.
Another part of me thinks: I’m confused. It seems just as reasonable to claim that it obviously has a world model that’s just not very smart. I’m probably using bad concepts and should think about this more.
It looks good to me!
Idk, maybe...?
Is that in doubt? Note that I don’t say it models the base objective in the post, I just say that it has a complex world model. This seemed unquestionable to me since it demonstrably knows lots of things. Or are you drawing a distinction between “a lot of facts about stuff” and ” a world model?” I haven’t draw that; “model” seems very general and “complex” trivially true. It may not be a smart model.
Part of me thinks: I was trying to push on whether it has a world model or rather has just memorised loads of stuff on the internet and learned a bunch of heuristics for how to produce compelling internet-like text. For me, “world model” evokes some object that has a map-territory relationship with the world. It’s not clear to me that GPT-3 has that.
Another part of me thinks: I’m confused. It seems just as reasonable to claim that it obviously has a world model that’s just not very smart. I’m probably using bad concepts and should think about this more.