One class is procedural / functional stuff like it should be able to use “consciousness” correctly when talking about things other than itself. I don’t see much point in asking it if it’s “token X,” where it’s never seen token X before. Another caveat would be that it should give good faith answers when we ask it hard or confusing questions about itself, but it should also often say “I don’t know,” and overall have a low positivity bias and low tendency to fall back on answers copied from humans.
The second class of caveats are about consciousness being a “suitcase word”, and language often being a bit treacherous. Consciousness isn’t an all or nothing proposition, humans have a bunch of different properties that we bundle together as “consciousness.” Modeling itself and the world, and using the word “conscious” to describe itself the same way it’d use it about you or me, are very important properties, but they’re only a small chunk of the properties we care about (like sensing the world in a human-understandable way, feeling emotions, feeling pleasure and pain, living and growing, etc.) that also often get lumped into “consciousness.”
I kind of agree, with two classes of caveats:
One class is procedural / functional stuff like it should be able to use “consciousness” correctly when talking about things other than itself. I don’t see much point in asking it if it’s “token X,” where it’s never seen token X before. Another caveat would be that it should give good faith answers when we ask it hard or confusing questions about itself, but it should also often say “I don’t know,” and overall have a low positivity bias and low tendency to fall back on answers copied from humans.
The second class of caveats are about consciousness being a “suitcase word”, and language often being a bit treacherous. Consciousness isn’t an all or nothing proposition, humans have a bunch of different properties that we bundle together as “consciousness.” Modeling itself and the world, and using the word “conscious” to describe itself the same way it’d use it about you or me, are very important properties, but they’re only a small chunk of the properties we care about (like sensing the world in a human-understandable way, feeling emotions, feeling pleasure and pain, living and growing, etc.) that also often get lumped into “consciousness.”