I guess we’re talking about the Shy Girl thing

Book publisher Hachette has pulled out of its deal with author Mia Ballard over accusations that her novel, Shy Girl, was written with the assistance of a large language model.

I say “accusations” because the actual evidence here is pretty thin. The arguments presented are all either “the writing looks like AI” or “we ran bits of the novel through an AI detector and it told us the text was AI-generated.”

First off, human beings are quite bad at telling the difference between machine-generated and human-generated art. The logical inconsistencies in Shy Girl's plot and prose could be the result of a chatbot overrunning its context window, or just sloppy writing, or (considering the subject matter) a deliberate attempt at surrealism. Most of the “obvious tells” of LLM-generated text (em dashes, rule of three, etc.) are actually features of a formal African English education, because LLMs were largely trained by criminally underpaid African workers. And guess who most often gets accused of “sounding like a chatbot”?

Then we get into AI detection tools, which The New York Times claims to have used. Many “AI detectors” also use machine learning to some degree — analyzing and comparing two sets of text is something LLMs are built to do — but these tools tend to produce a lot of false results. The differences between human-generated text and machine-generated text are going to be largely invisible to an LLM designed to mechanically produce text that could plausibly pass as something a human wrote.

And that’s not even getting into the “AI detectors” that simply paste the submitted text into ChatGPT and ask it, “hey, did you write this?” To quote a friend of mine with approximate knowledge of many things, “anyone who tells you they have a tool to accurately detect AI is probably a liar, and liars love to use AI, so it’s probably just feeding the text into ChatGPT and asking.”

Three years ago, I noted that because “it looks like it was made by AI” is an accusation that can’t be proven right or wrong, it would inevitably be used as an ideological bludgeon against any art an accuser personally didn’t like very much. And now it looks as though a lot of people really didn’t like Shy Girl, didn’t think anyone else should like Shy Girl, and found the perfect way to bully it off the market.

(Does it feel good to be right all the time? No, it’s awful.)

Trying to suss out whether a book was written using an LLM or not is, in my opinion, pointless. A book should be criticized on the basis of whether it sucks — and, so far, provably LLM-written books have universally sucked. When an LLM manages to write a novel that’s actually good, we can revisit this topic.

And I’m fine with making it a rule that nobody can publish a novel they didn’t personally write, but in that case someone should have a word with Tom Clancy’s corpse.

( Read more... )

-K