Yeah, I think people asking AIs to write essays for them is currently pretty doomed. The RLHF also makes the writing style of basically all AI systems into a weird kind of polite-markety-slop that tends to waffle in a way that I find deeply grating.
That said, I sure use AI systems as part of a lot of my writing and thinking, and I also desperately wish I could somehow get them to help me with editing. Many times has it happened that I changed some small part of an article, and where I wished I could just ask my AI assistant to propagate the obvious consequences of that change throughout the rest of the article (for example, let’s say I start my article with two examples, I remove the second one because I no longer like it, now I would like to rewrite the rest to no longer reference the second example).
I think the real lesson to take away from this is that, at least right now, chat models are bad writers and bad writing assistants. I think things like base models or autocomplete models have more of a chance of producing writing that isn’t terribly grating, and in almost no circumstance would I recommend that someone directly copies more than maybe 1 paragraph of LLM output into their essay without clearly attributing it to AI systems.
On the other hand, I would personally be in favor of people embedding more references to LLM conversations into their essays. It’s an easy way to be transparent about your sources, and I often find it interesting to understand why an LLM got something wrong.
Right now, I would consider it to be a mod-warneable offense to publish an obviously LLM-written essay under your own name, without telling people an LLM wrote it, or most of it. And at least for now it’s pretty easy to tell.
I wonder, could this be solved by a good prompt? I mean, most humans are bad writers, and LLMs are taught on their texts, so they should be bad writers by default.
But sometimes a few words in the prompt change a lot, even if from human perspective that doesn’t make much sense. For me, this is all black magic, but I would not be surprised if starting the prompt with “you are a great writer; you write concisely and make your points clear” had a dramatic impact on the outcome.
(Kinda like generated images are sometimes better if you include “realistic” and “correct number of fingers” in the prompt. At least I think so; this is what I have seen other people do in prompts, but I didn’t do an A/B test to verify that it really improves the outcome.)
Even more, maybe we could put (a drastically condensed version of) the Sequences in the prompt, to remind the LLM to avoid specific biases, to reason step by step rather than make the conclusion first, etc. Yeah, if you tell me this wouldn’t work, I will trust your experience, but I see no a-priori reasons why not.
EDIT:
I see other people already experimented with prompts, and it improved the results, but not sufficiently.
Yeah, I think people asking AIs to write essays for them is currently pretty doomed. The RLHF also makes the writing style of basically all AI systems into a weird kind of polite-markety-slop that tends to waffle in a way that I find deeply grating.
That said, I sure use AI systems as part of a lot of my writing and thinking, and I also desperately wish I could somehow get them to help me with editing. Many times has it happened that I changed some small part of an article, and where I wished I could just ask my AI assistant to propagate the obvious consequences of that change throughout the rest of the article (for example, let’s say I start my article with two examples, I remove the second one because I no longer like it, now I would like to rewrite the rest to no longer reference the second example).
I think the real lesson to take away from this is that, at least right now, chat models are bad writers and bad writing assistants. I think things like base models or autocomplete models have more of a chance of producing writing that isn’t terribly grating, and in almost no circumstance would I recommend that someone directly copies more than maybe 1 paragraph of LLM output into their essay without clearly attributing it to AI systems.
On the other hand, I would personally be in favor of people embedding more references to LLM conversations into their essays. It’s an easy way to be transparent about your sources, and I often find it interesting to understand why an LLM got something wrong.
Right now, I would consider it to be a mod-warneable offense to publish an obviously LLM-written essay under your own name, without telling people an LLM wrote it, or most of it. And at least for now it’s pretty easy to tell.
I wonder, could this be solved by a good prompt? I mean, most humans are bad writers, and LLMs are taught on their texts, so they should be bad writers by default.
But sometimes a few words in the prompt change a lot, even if from human perspective that doesn’t make much sense. For me, this is all black magic, but I would not be surprised if starting the prompt with “you are a great writer; you write concisely and make your points clear” had a dramatic impact on the outcome.
(Kinda like generated images are sometimes better if you include “realistic” and “correct number of fingers” in the prompt. At least I think so; this is what I have seen other people do in prompts, but I didn’t do an A/B test to verify that it really improves the outcome.)
Even more, maybe we could put (a drastically condensed version of) the Sequences in the prompt, to remind the LLM to avoid specific biases, to reason step by step rather than make the conclusion first, etc. Yeah, if you tell me this wouldn’t work, I will trust your experience, but I see no a-priori reasons why not.
EDIT:
I see other people already experimented with prompts, and it improved the results, but not sufficiently.
I think with the right prompting techniques you can indeed do better. I might post something later today.