Other fun fact: When giving Sonnet 3.5 or Opus or GPT-4o the BIG-BENCH GUID, they are not able to identify it as the canary string, even with a lot of prodding. In humans this would be the other way around, where someone can easily recognize a particular string, even if they can’t memorize it verbatim.
Other fun fact: When giving Sonnet 3.5 or Opus or GPT-4o the BIG-BENCH GUID, they are not able to identify it as the canary string, even with a lot of prodding. In humans this would be the other way around, where someone can easily recognize a particular string, even if they can’t memorize it verbatim.