GPT dreams

The Times a few days ago offered an unusually illuminating account of how the "large language learning" of programs like ChatGPT is able to replicate the style of different writers and genres. They demonstrate how a program, starting from scratch with only the collected works of Jane Austen or Shakespeare, or particular works like the Federalist Papers, the Harry Potter books, Moby Dick or the TV show "Star Trek: The Next Generation," can arrive with remarkable speed at success.

The program begins at the level of letters, generating a random array and then measuring it against its target database, noting which, if any, juxtapositions appear there and with what frequency. (This is the Austen simulator.) Within 250 rounds, it learns to distinguish letters from numbers and punctuation marks, and the approximate length of words - there were no spaces when it started. From a distance it looks like English, although every word is gibberish.

After another 250 rounds, the small words are English words, although the rest remain monstrosities of randomness, like bits of polished glass among the stones on a seashore. It takes ten times longer for the process to get some longer words right, but soon it's looking like this. 

The words are jumbled and monster words remain but even they aren't random any more, but assembled like real English words. And after another 25,000 rounds we arrived at what look like sentences. 

There are still some non-words, but even these look like English. Punctuation and the rhythm of words of different lengths makes it look like natural syntax unfolding. It's confounding to realize that these words, while English, still have nothing to do with each other - confounding because it looks right. As is clearest when you're reading a text aloud, we never read a text word by word (at least in languages we're fluent in) but first take in the Gestalt of sentence and paragraph in order to know where to lay the stress, when to pause. Freakishly, the Gestalt is here. I feel like I could read this aloud. 

I also feel like something is emerging that was always there, struggling to articulate itself. Wrong, wrong, wrong, of course! Kindly, the article leaves it at this. I don't know how many more rounds it would take for "BabyGPT" to be able to give me the grammatically correct and syntactically seamless texts spun by ChatGPT...

When I first read the article I was reminded of the babbling of children, and of a young nephew's expertly mimicking the playing of a piano with dramatically satisfying phrasing as he pounded his little hands on the keyboard, pausing only to turn the page of the score. They get the Gestalt without - before - the meaning, too. As I write this I recall also a fun acting exercise from "Religion and Theater," ages ago, in which student improvised scenes without words but "babbling" - and understood each other well enough to make for compelling interactions!

But last night, perhaps freed by the Adirondack mountain air, my machine learning came home to roost. I had a sequence of dreams, as I sometimes do, taking place in different times and with different people I've known, some silly some less so. One was a gathering at the faculty club of a university I used to teach at, where everyone was unrecognizable in silhouette and I worried with whom I'd sit. But when I woke, the dreams still on the edge of awareness, I saw them in a whole new way. Not garbled memories, not something from my unconscious, struggling to articulate itself, not some privileged glimpse into the matrix. They don't mean anything, I thought. They're Gestalt only, meaningful-seeming arrays of what are really just random snibbets of content.

In this season of AI excitement pundits have wondered how the newest generation of programs will change the way we understand each other and even ourselves. Am I too early an adapter?