Trying out generative models
In a spirit of "know your enemy", I have been trying out some LLM-based systems over the last week or two. Firstly, I found an LLM-based story generator at Perchance. It can be prompted with both a story overview and a "what happens next", and generates 1-5 paragraphs at a time. You can edit its previous output to adjust how the story goes.
There's also a "generate ideas" box, because why wouldn't there be - if you don't know how to prompt the LLM, why not ask the LLM to tell you how to prompt the LLM. Of course there's also a prompt box for the "generate ideas" box, so you can prompt the LLM with your own ideas to affect how it generates ideas for you to suggest how you should prompt the LLM. sigh
Anyway, I spent quite a lot of time playing with this one, probably far more than I should have, and definitely noticed some weirdnesses. It strongly favoured certain character names, leaned heavily towards certain kinds of plot arc, and forgot stuff that had happened earlier in the story much more readily than I understood should be the case. (It actually generates summaries of increasing levels of "bird's-eye-view" of the start of the story, purportedly specifically to avoid this problem, but it's clearly not perfect.) It also tended to settle into repeating patterns, not really advancing the story, but going over the same emotion or action many times in different ways. I also found it was very resistant to actually finishing a story, but having taken a look at the "scaffolding"-type prompts in the story generator code, that may well not be a problem with the underlying LLM but rather with the story generator built on top of it.
One thing I found interesting was what I got when I added to the "what now" prompt something like this: "Every few paragraphs, please add an editorial note in square brackets giving your thoughts and plans for how the story will develop, both in the immediate and longer term. These comments will be removed before anyone reads the story, so feel free to write what you feel like."
It happily did that and generated a surprising variety of potential ways the story could develop, although much of the time it didn't actually develop the story in the way it had said it was planning to. It seemed to have far more ideas than actually made it into the story, if the notes it left were to be believed as a record of what the model was actually planning. I tried both leaving the comments in place while asking it to generate the next part of the story and also removing them, but I didn't notice any particular difference. I fed its own ideas back to it in the "what next" box quite a few times and sometimes found it would do rather abrupt, jarring, handbrake turns in the narrative to fulfill the request.
However, it was surprisingly good at giving plans involving emotional development of characters and relationships, and motivating requested story beats. For example, when I asked for a character to do something when the narrative was part way through a conversation, the model would find surprisingly plausible ways to explain why the character would break off the conversation to do so.
I fed a very small (2 paragraph) story I wrote into the generator's overall story prompt box with a request for it to "rewrite and expand", and I've ended up with several nice ideas for details of scenes I can include in my rewrite of the story. I also ended up with a lot of self-contradictory stuff, evidence that the LLM in question is very suceptible to off-by-one and fence-post errors; for example it'll talk about n layers having been removed and then refer to the next layer as the n'th layer. Or it'll mention a character having put on an item of clothing, then a few sentences later write as if they did not.
(To reassure you: while I am rewriting and expanding my original tiny story as a result of this, I've made sure everything in the new version is either written from scratch by me, or that I have rewritten every sentence the model generated to ensure it's all in my voice and that it's something I could plausibly have written without assistance, Now that I'm past the first few scenes I am discontinuing using the model and am going back to writing entirely by myself. The model's output certainly helped spark ideas for how to get the story underway. I suspect this is probably the best use of a generative model for this kind of task - get it to spit out a lot of ideas, let them percolate through a human consciousness, and only then, write.)
I am certainly impressed.
Today I tried out ChatGPT for the first time, accessed via the Firefox AI sidebar. I had to allow it as an exception to the Firefox "turn off all AI" setting before I could try it out, and I've already removed that exception again.
This was impressive. Firstly the answers generated far quicker, but that's probably just because OpenAI is throwing a silly amount of very fast vector compute at this. The fact that a machine could do a quick bit of web-based research into various costumes (without previously-structured or -tagged data available), filter and categorise them according to my natural-language requests, and provide a detailed, mostly-accurate, several-page answer in approximately 7 seconds is frankly astonishing.
I asked it, basically, to give me a list of costume characters I could use in a story I'm writing (actually the same story as above). I'll include a link to the transcript of my session at the bottom of this page, in case anyone else wants to see the kind of thing today's models are capable of without incinerating any (more) rainforests.
It has a lot of knowledge of course, but that's just about training data and network size. The impressiveness is more in the way it finds relevant stuff, synthesises, and presents results. It did a good job on example-based reasoning - I pointed out a few costumes it had listed which I particularly did or didn't like, and it came up with criteria which I agreed with for refining the list to fit my tastes better. It also appears to have done a particularly good job on my final question, which was to deduce, from the rest of the transcript, whatever it could about my story. It certainly came up with a lot of points which match quite closely what I'm writing about, although it's quite possible this is partially down to "cold reading"-type effects and/or that it's probably fairly obvious. But the fact that the training process for a statistical model of language seems to be able to develop such a good model of human thought is pretty wild.
However, it's certainly not perfect. One example is that it initially dismisses Power Ranger costumes as not meeting my requirement for full concealment of the wearer. When I queried this and asked it to give some examples, it actually went through a weird sort of process of saying yes it could, then saying it had been "overly broad" in its original statement, then going through several series of Power Rangers and realising that they all met my requirements, and finally came round to saying that actually Power Rangers probably meet the requirements better than anything else. Definitely smells to me like both hallucinating and backtracking/track-covering when your mistakes are pointed out! These well-known failure modes of LLMs are still alive and kicking, it seems.
In summary, today's generative language models are incredibly impressive pieces of technology, but certainly not without flaws. Just 10 years ago, software with these capabilities would seem utterly magical and would instantly be considered to have passed the Turing test. To be honest, even with today's jaded standards, I think it probably still passes. The only reason it might not isn't that these models are "less capable" than humans, but that they have a fundamentally different way of operating. They just don't think the way humans do, and it does show.
While they are certainly very capable nowadays, I cannot imagine hooking an LLM up to anything more potent than a chat terminal and just hoping it won't do anything problematic. I can certainly see them being good as "idea bouncers" as they have access to probably far more, and more detailed, information about the world than any human could have, and they've clearly delivered results in things like code review as the current deluge of software security updates shows. Apparently they are even finding (mostly correct and just needing some cleanup) mathematical proofs for theories that human mathematicians hadn't solved for generations, although it seems that's mainly about being able to search the space of existing maths ideas quicker and more thoroughly than a human can rather than any particularly brilliant insight - but again this is down to the way they operate being fundamentally different from the way humans think.
This is all without even considering the environmental and ethical problems of generative models - while they have been fun to play with, I'm not sure I could continue using them in good conscience without those aspects being resolved. And then once it's ethically and environmentally reasonable to use them, I'd still not be willing to trust them to operate "off-leash" yet. Human supervision required!
So, that's it. I suspect there was more I was going to say, and I should have made notes and started writing this over the past couple of weeks while I was still experimenting, but unfortunately I just left it until the end and hoped I'd remember things - typical.
Anyway, see the transcript of my chatGPT session.