This has less of the fingernails-along-a-blackboard feeling given off by every sentence of the original story
Below I’ve collected excerpts from the works of several less-known but talented human writers. Or maybe they were written by GPT4. Can you guess which ones are human-made?
Sample 1:
“Nottingham has enough pubs and clubs”, say the local police. If you wanted to get around every last one of them it would be a year at a brisk trot before you were starting to visit establishments more than one mile from the centre of the city. Pick a Friday or a Saturday, any Friday or Saturday of the year: the establishments will be rammed and jumping and the streets bustling with people in their most tightly-wound and elaborately crafted drinking costumes. It’s almost Christmas but the cold season has not added much to the average number of layers.
Sample 2:
The book was better than the movie. For one thing, there was a lot more in it. And some of the pictures were awfully different from the movie. But in both, Pinocchio—a life-sized wooden boy who magically is roused to life—wore a kind of halter, and there seemed to be dowels in his joints. When Geppetto is just finishing the construction of Pinocchio, he turns his back on the puppet and is promptly sent flying by a well-placed kick. At that instant the carpenter’s friend arrives and asks him what he is doing sprawled on the floor.
Sample 3:
He glanced outside at the buildings casting long shadows in the fading sunlight, the city frozen in the grasp of time. He took a deep breath. And then, he jumped.
He closed his eyes, feeling the rush of displaced seconds, the vertigo of time stretching, condensing, then snapping back into place. When he opened them, he found the world stilled. The shadows were now statues, the sun paused in its descent, and a bird hung motionless in the sky. This was Finn’s minute—his extra minute.
Sample 4:
Once upon a Martian sunrise...
Yeah, I know, sounds like the beginning of a fairy tale or a bedtime story, right? But I promise you, on my physicist-turned-astronaut honor, this isn’t fiction. It’s the raw, unadulterated truth. My truth.
Red soil underfoot, as fine as confectioner’s sugar. Low-grav shuffle making every step a dance move. Peaks and valleys sprawled across the horizon like a mythological beast sleeping off a hard night.
I’m Corporal Thea Kolinski, an astrophysicist by trade, astronaut by accident, and currently the number one recipient of the “Most Unlikely to Succeed” superlative in our six-person crew. A crew assembled to survive on Mars. A first in human history.
Sample 5:
Then, as abruptly as it began, the light faded. Billy blinked, expecting to see a spaceman, green and with eyes as big as dinner plates, but instead, there was a rock. A simple, gleaming, alien rock, sat innocently in the middle of his vegetable patch.
The spaceship blinked out of existence just as quickly as it had appeared, leaving Billy alone in his garden, a lump of extraterrestrial mineral his only proof of what he’d witnessed.
Sample 6:
For in the silence of their failure, a sound, inaudible to human ears but felt in the marrow of their bones, resonated from the vessel, wrapping the men in a shroud of madness. One by one, they fell, their minds invaded by images of cosmic horror, their sanity shredded by the unintelligible secrets whispered by the alien ship.
Standish was the last to succumb, his face a rictus of terror as he stared at the vessel. As the sun dipped below the horizon, plunging the valley into an unfathomable darkness, Standish’s final cry echoed through the hills of Belseth, marking the tragic end of their misguided endeavour.
Sample 7:
“Jump!” he shouted to Mandel. The crawler shuddered, throwing up clouds of sand and dust, and started to turn stern up. Novago switched off the engine and scrambled out of the crawler. He landed on all fours, and, without standing up, scurried off to one side. The sand slid and sank underneath him, but Novago managed to reach firm ground. He sat down, tucking his legs under him.
He saw Mandel, who was kneeling at the opposite edge of the crater, and the stern of the crawler, shrouded in steam and sticking up out of the sand on the bottom of the newly formed crater. Theoretically it was impossible for something like this to happen to a Lizard model. Here on Mars, at least. A Lizard was a light, fast machine—a five-seat open platform mounted on four autonomous caterpillar-tracked chassis. But here it was, slowly slipping
Sample 8:
No, I ain’t pullin’ your leg, sonny, it’s the God’s honest truth. It happened in our little town of Lonesome Hollow, right there in the foothills of the Appalachians. It’s a tale that puts the ‘odd’ in odds and ends, I tell you.
The lass in question was Amelia, known to most as plain ol’ Millie. Millie was as normal as the day is long, a pretty thing with a head full of chestnut curls and the sweetest smile you’d ever see. She ran the general store, knew the names of every critter in town, and baked the best apple pie this side of the Mississippi.
Firstly, there is one clear standout from these: Sample 6. This must be GPT4. It could come straight out of the original story. None of the others give off that smell so intensely. While they could all be written by humans, I am very suspicious of some of them.
Here are my detailed criticisms, unassisted by any consultation of the Internet. I’m writing based on the assumption that the human-written extracts (if any) are from commercially published (and not self-published) works.
Sample 1: Probably human, on account of the coherence and lack of slack writing. There is one red flag, the “tightly-wound and elaborately crafted drinking costumes” that are apparently worn every weekend, not just on some special occasion. This ethnographic detail is just weird, given that nothing else in the passage suggests that this is fantastic fiction. However, the passage as a whole has better coherence than GPT4 usually manages, and the successful exact parallelism in “Pick a Friday or a Saturday, any Friday or Saturday of the year” is something I have not seen GPT4 do.
Sample 2: Probably GPT4. Each sentence on its own is innocuous enough, but on the larger scale it seems to veer from talking about the book and the movie to telling the Pinocchio story. What is this “halter”? Why mention the dowels? Or course there would be dowels in a puppet’s joints, and they wouldn’t just “seem” to be there.
Sample 3: Doubtful, but I lean towards GPT. “Frozen in the grasp of time” is cliched. If the story has already established this anomalous city, there’s no reason to describe it here with that phrase. It is also inconsistent with the motion of time implied by “fading sunlight”. When Finn’s fall is stilled, the shadows of buildings are described as statues. The shadows of buildings do not look like statues. The movement of shadows is not ordinarily perceptible anyway.
Sample 4: Human. I’m not aware that GPT4 can spontaneously use italics. Leaving that aside, the italicized paragraph is not something that GPT4 would write. The first and last paragraphs have something of its thump, thump repetitiveness, but human writers do sometimes do that, especially at the start of a story to establish the scene.
Sample 5: I’m about 60-40 on this one, more likely GPT than not. It could be from near the start of a Ray Bradbury story, but the writing just isn’t tight enough. For example, “blinked out of existence just as quickly as it had appeared” is redundant. A blink is as fast as anything could appear or disappear.
Sample 6: As I said, this instantly stands out as GPT4. It’s junk. Even if every other sample is human, this one is not.
Sample 7: GPT4. Land vehicles don’t have sterns. The scene describes the vehicle sinking into a sudden subsidence, but that is a hazard that any vehicle would be susceptible to, given a big enough subsidence. The surprise should not be that the vehicle sank, but that the subsidence happened at all. “Steam”, escaping from a Mars vehicle? No. The extract ends in the middle of a sentence, but that might just be a copy-paste error.
Sample 8: Likely human. It’s the sort of writing that single-mindedly is exactly what it is, but it has a ring to it without repetitions, that I don’t find in GPT4′s bells of lead.
I’ve marked all of these quite sternly, but if I’ve wrongly accused any real authors of writing like GPT4, I stand by the criticisms I’ve made of what they’ve written.
Of course, real writers, even stars of the literary firmament, do nod, and lesser-known published writers are often lesser-known for a good reason. David Langford’s science fiction newsletter Ansible has a regular item called Thog’s Masterclass, exhibiting examples of “differently good” actually published writing. Dare the Thog-o-Matic to see some random examples. ETA: or look at any Perry Rhodan novel.
You got 4 of 8 right. In two cases you failed to recognize humans, and in another two—GPT4.
It was a weakly adversarial test:
I took a few less-known but obviously talented writers from the top of my head, and copied the excerpts from the first pages.
For GPT4, I’ve used several prompts from the competition, and then selected the parts for their stylistic diversity.
I suspect that a test with longer excerpts would be much easier for you, as the vanilla GPT4 is indeed often easy to detect due to its repetitiveness etc (I haven’t tried the APIs yet).
If GPT4 already can fool some of us science fiction junkies, I can’t wait to read the fiction by GPT5.
David Langford’s science fiction newsletter Ansible has a regular item called Thog’s Masterclass, exhibiting examples of “differently good” actually published writing. Dare the Thog-o-Matic to see some random examples. ETA: or look at any Perry Rhodan novel.
Thank you!
BTW, have you read “Appleseed” by John Clute? I have a feeling you may be one of the few people on Earth who can enjoy it. A representative sample:
They passed the iron-grey portcullis that sealed off the inferno of drive country. A dozen ceremonial masks, mourning the hardened goblin eidolons of KathKirtt that died hourly inside drive country, hung within their tile embrasure above the frowning portal. The masks were simplified versions of the flyte gorgon. Their single eyes shut in unison at the death of one of the goblin eidolons, who spent their brief spans liaising with the quasi-sentient engine brother that drove the ship through the demonic rapturous ftl maze of wormholes. Even for eidolons with hardened carapaces, to liaise was to burn and die. When Tile Dance plunged through the ashen caltraps of ftl at full thrust, the engine brother howling out something like anguish or joy all the while, its entirely imaginary ‘feet’ pounding the turns of the maze, goblins lived no longer than mayflies.
Very interesting. My accuracy was the same as Richard’s: 4/8*. I think you probably used my prompt for one of the ones I got right, which is probably why I got it right (the tone and structure are very familiar to me after so much experimentation).
To those who think the current crop of AIs aren’t capable of writing great novellas (18-40k words): Do you think your opinion will change in the next 5 years?
* I originally reported a score of 1⁄8 by mistake.
I think you probably used my prompt for the one I got right, which is probably why I got it right (the tone and structure are very familiar to me after so much experimentation).
Nope, this one. But their prompt does incorporate some ideas from your prompt.
Below I’ve collected excerpts from the works of several less-known but talented human writers. Or maybe they were written by GPT4. Can you guess which ones are human-made?
Sample 1:
“Nottingham has enough pubs and clubs”, say the local police. If you wanted to get around every last one of them it would be a year at a brisk trot before you were starting to visit establishments more than one mile from the centre of the city. Pick a Friday or a Saturday, any Friday or Saturday of the year: the establishments will be rammed and jumping and the streets bustling with people in their most tightly-wound and elaborately crafted drinking costumes. It’s almost Christmas but the cold season has not added much to the average number of layers.
Sample 2:
The book was better than the movie. For one thing, there was a lot more in it. And some of the pictures were awfully different from the movie. But in both, Pinocchio—a life-sized wooden boy who magically is roused to life—wore a kind of halter, and there seemed to be dowels in his joints. When Geppetto is just finishing the construction of Pinocchio, he turns his back on the puppet and is promptly sent flying by a well-placed kick. At that instant the carpenter’s friend arrives and asks him what he is doing sprawled on the floor.
Sample 3:
He glanced outside at the buildings casting long shadows in the fading sunlight, the city frozen in the grasp of time. He took a deep breath. And then, he jumped.
He closed his eyes, feeling the rush of displaced seconds, the vertigo of time stretching, condensing, then snapping back into place. When he opened them, he found the world stilled. The shadows were now statues, the sun paused in its descent, and a bird hung motionless in the sky. This was Finn’s minute—his extra minute.
Sample 4:
Once upon a Martian sunrise...
Yeah, I know, sounds like the beginning of a fairy tale or a bedtime story, right? But I promise you, on my physicist-turned-astronaut honor, this isn’t fiction. It’s the raw, unadulterated truth. My truth.
Red soil underfoot, as fine as confectioner’s sugar. Low-grav shuffle making every step a dance move. Peaks and valleys sprawled across the horizon like a mythological beast sleeping off a hard night.
I’m Corporal Thea Kolinski, an astrophysicist by trade, astronaut by accident, and currently the number one recipient of the “Most Unlikely to Succeed” superlative in our six-person crew. A crew assembled to survive on Mars. A first in human history.
Sample 5:
Then, as abruptly as it began, the light faded. Billy blinked, expecting to see a spaceman, green and with eyes as big as dinner plates, but instead, there was a rock. A simple, gleaming, alien rock, sat innocently in the middle of his vegetable patch.
The spaceship blinked out of existence just as quickly as it had appeared, leaving Billy alone in his garden, a lump of extraterrestrial mineral his only proof of what he’d witnessed.
Sample 6:
For in the silence of their failure, a sound, inaudible to human ears but felt in the marrow of their bones, resonated from the vessel, wrapping the men in a shroud of madness. One by one, they fell, their minds invaded by images of cosmic horror, their sanity shredded by the unintelligible secrets whispered by the alien ship.
Standish was the last to succumb, his face a rictus of terror as he stared at the vessel. As the sun dipped below the horizon, plunging the valley into an unfathomable darkness, Standish’s final cry echoed through the hills of Belseth, marking the tragic end of their misguided endeavour.
Sample 7:
“Jump!” he shouted to Mandel. The crawler shuddered, throwing up clouds of sand and dust, and started to turn stern up. Novago switched off the engine and scrambled out of the crawler. He landed on all fours, and, without standing up, scurried off to one side. The sand slid and sank underneath him, but Novago managed to reach firm ground. He sat down, tucking his legs under him.
He saw Mandel, who was kneeling at the opposite edge of the crater, and the stern of the crawler, shrouded in steam and sticking up out of the sand on the bottom of the newly formed crater. Theoretically it was impossible for something like this to happen to a Lizard model. Here on Mars, at least. A Lizard was a light, fast machine—a five-seat open platform mounted on four autonomous caterpillar-tracked chassis. But here it was, slowly slipping
Sample 8:
No, I ain’t pullin’ your leg, sonny, it’s the God’s honest truth. It happened in our little town of Lonesome Hollow, right there in the foothills of the Appalachians. It’s a tale that puts the ‘odd’ in odds and ends, I tell you.
The lass in question was Amelia, known to most as plain ol’ Millie. Millie was as normal as the day is long, a pretty thing with a head full of chestnut curls and the sweetest smile you’d ever see. She ran the general store, knew the names of every critter in town, and baked the best apple pie this side of the Mississippi.
Firstly, there is one clear standout from these: Sample 6. This must be GPT4. It could come straight out of the original story. None of the others give off that smell so intensely. While they could all be written by humans, I am very suspicious of some of them.
Here are my detailed criticisms, unassisted by any consultation of the Internet. I’m writing based on the assumption that the human-written extracts (if any) are from commercially published (and not self-published) works.
Sample 1: Probably human, on account of the coherence and lack of slack writing. There is one red flag, the “tightly-wound and elaborately crafted drinking costumes” that are apparently worn every weekend, not just on some special occasion. This ethnographic detail is just weird, given that nothing else in the passage suggests that this is fantastic fiction. However, the passage as a whole has better coherence than GPT4 usually manages, and the successful exact parallelism in “Pick a Friday or a Saturday, any Friday or Saturday of the year” is something I have not seen GPT4 do.
Sample 2: Probably GPT4. Each sentence on its own is innocuous enough, but on the larger scale it seems to veer from talking about the book and the movie to telling the Pinocchio story. What is this “halter”? Why mention the dowels? Or course there would be dowels in a puppet’s joints, and they wouldn’t just “seem” to be there.
Sample 3: Doubtful, but I lean towards GPT. “Frozen in the grasp of time” is cliched. If the story has already established this anomalous city, there’s no reason to describe it here with that phrase. It is also inconsistent with the motion of time implied by “fading sunlight”. When Finn’s fall is stilled, the shadows of buildings are described as statues. The shadows of buildings do not look like statues. The movement of shadows is not ordinarily perceptible anyway.
Sample 4: Human. I’m not aware that GPT4 can spontaneously use italics. Leaving that aside, the italicized paragraph is not something that GPT4 would write. The first and last paragraphs have something of its thump, thump repetitiveness, but human writers do sometimes do that, especially at the start of a story to establish the scene.
Sample 5: I’m about 60-40 on this one, more likely GPT than not. It could be from near the start of a Ray Bradbury story, but the writing just isn’t tight enough. For example, “blinked out of existence just as quickly as it had appeared” is redundant. A blink is as fast as anything could appear or disappear.
Sample 6: As I said, this instantly stands out as GPT4. It’s junk. Even if every other sample is human, this one is not.
Sample 7: GPT4. Land vehicles don’t have sterns. The scene describes the vehicle sinking into a sudden subsidence, but that is a hazard that any vehicle would be susceptible to, given a big enough subsidence. The surprise should not be that the vehicle sank, but that the subsidence happened at all. “Steam”, escaping from a Mars vehicle? No. The extract ends in the middle of a sentence, but that might just be a copy-paste error.
Sample 8: Likely human. It’s the sort of writing that single-mindedly is exactly what it is, but it has a ring to it without repetitions, that I don’t find in GPT4′s bells of lead.
I’ve marked all of these quite sternly, but if I’ve wrongly accused any real authors of writing like GPT4, I stand by the criticisms I’ve made of what they’ve written.
Of course, real writers, even stars of the literary firmament, do nod, and lesser-known published writers are often lesser-known for a good reason. David Langford’s science fiction newsletter Ansible has a regular item called Thog’s Masterclass, exhibiting examples of “differently good” actually published writing. Dare the Thog-o-Matic to see some random examples. ETA: or look at any Perry Rhodan novel.
The key[1].
You got 4 of 8 right. In two cases you failed to recognize humans, and in another two—GPT4.
It was a weakly adversarial test:
I took a few less-known but obviously talented writers from the top of my head, and copied the excerpts from the first pages.
For GPT4, I’ve used several prompts from the competition, and then selected the parts for their stylistic diversity.
I suspect that a test with longer excerpts would be much easier for you, as the vanilla GPT4 is indeed often easy to detect due to its repetitiveness etc (I haven’t tried the APIs yet).
If GPT4 already can fool some of us science fiction junkies, I can’t wait to read the fiction by GPT5.
Thank you!
BTW, have you read “Appleseed” by John Clute? I have a feeling you may be one of the few people on Earth who can enjoy it. A representative sample:
Only one, two, and seven are human: from “Ra” by qntm, “Contact” by Sagan, “Noon: 22nd Century” by Strugatsky. The rest is GPT4Very interesting. My accuracy was the same as Richard’s: 4/8*. I think you probably used my prompt for one of the ones I got right, which is probably why I got it right (the tone and structure are very familiar to me after so much experimentation).
To those who think the current crop of AIs aren’t capable of writing great novellas (18-40k words): Do you think your opinion will change in the next 5 years?
* I originally reported a score of 1⁄8 by mistake.
Nope, this one. But their prompt does incorporate some ideas from your prompt.