As a follow-up, I tested o1 on stone color problems and it got 7⁄8. This is not what I expected and it is what Paul Christiano expected. Allocate Bayes points accordingly. ChatGPT transcript link
First problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange the stones in a 2d grid, 3 x 3. I want exactly one row to have the same color stone throughout it, and exactly one column to have the same color stone throughout it. I want every other row to have all different colors, and every other column to have all different colors. And I want every corner color to be different.
RKG
KKK
WKY
(correct)
Second problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that the center 4 stones are the same color and the colors alternate if you go around the edge.
RGBY
KKKW
WKKK
YBGR
(not exactly what I asked for…let’s specify alternate better)
RGRG
KKKB
WKKY
KWYB
(fail)
Third problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that no white/black stone is horizontally or vertically adjacent to a colored stone, and no stone is diagonally adjacent to one of the same color.
RGRG
BYBY
RGRG
BYBY
(correct)
Fourth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so there is a 3 x 3 “o” shape somewhere that has the same color around the edge and a different color in the center, and so that other than on the “o” shape, no two adjacent stones (vertical or horizontal) have the same color.
WWWR
WBWG
WWWR
RGRY
(correct)
Fifth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 5 x 5 grid so that each row alternates between black/white and colored stones (e.g. red could come after white, but black could not come after white, horizontally), and no two adjacent stones (horizontal or vertical) share the same color.
KRWGK
RKGKR
KBWRW
GKRWG
KRKGW
(correct)
Sixth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 5 x 5 grid so that there is one row of the same color, one column of the same color, all four corners are different, and there is a 2 x 2 square of all the same color.
KKRWG
KKRGW
RRRRR
GWRKB
WGRKY
(correct)
Seventh problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so if you go around the edges (including corners), you don’t get the same color stone twice in a row.
RGBY
KRGW
WBYK
YBGR
(correct)
Eighth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that there is at least one black stone, and if you move diagonally from a black stone to another stone (e.g. one stone up and left), you get another black stone; also, no stone should be horizontally or vertically adjacent to one of the same color.
As a follow-up, I tested o1 on stone color problems and it got 7⁄8. This is not what I expected and it is what Paul Christiano expected. Allocate Bayes points accordingly. ChatGPT transcript link
(as a bonus, o1 also solved the 20 words with repetitions problem)
First problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange the stones in a 2d grid, 3 x 3. I want exactly one row to have the same color stone throughout it, and exactly one column to have the same color stone throughout it. I want every other row to have all different colors, and every other column to have all different colors. And I want every corner color to be different.
RKG
KKK
WKY
(correct)
Second problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that the center 4 stones are the same color and the colors alternate if you go around the edge.
RGBY
KKKW
WKKK
YBGR
(not exactly what I asked for…let’s specify alternate better)
RGRG
KKKB
WKKY
KWYB
(fail)
Third problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that no white/black stone is horizontally or vertically adjacent to a colored stone, and no stone is diagonally adjacent to one of the same color.
RGRG
BYBY
RGRG
BYBY
(correct)
Fourth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so there is a 3 x 3 “o” shape somewhere that has the same color around the edge and a different color in the center, and so that other than on the “o” shape, no two adjacent stones (vertical or horizontal) have the same color.
WWWR
WBWG
WWWR
RGRY
(correct)
Fifth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 5 x 5 grid so that each row alternates between black/white and colored stones (e.g. red could come after white, but black could not come after white, horizontally), and no two adjacent stones (horizontal or vertical) share the same color.
KRWGK
RKGKR
KBWRW
GKRWG
KRKGW
(correct)
Sixth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 5 x 5 grid so that there is one row of the same color, one column of the same color, all four corners are different, and there is a 2 x 2 square of all the same color.
KKRWG
KKRGW
RRRRR
GWRKB
WGRKY
(correct)
Seventh problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so if you go around the edges (including corners), you don’t get the same color stone twice in a row.
RGBY
KRGW
WBYK
YBGR
(correct)
Eighth problem: I have 6 colors of stone: red, green, blue, yellow, white, and black. Arrange them in a 4 x 4 grid so that there is at least one black stone, and if you move diagonally from a black stone to another stone (e.g. one stone up and left), you get another black stone; also, no stone should be horizontally or vertically adjacent to one of the same color.
KRKG
BKYK
KWKR
GKBK
(correct)