Meet the Solutionists, with Mark Scott

We always thought that when the robots arrived, they’d be cold and calculating like The Terminator... But they’re mostly charming and inaccurate. We also thought they’d do away with the drudgery of our most boring work... But it’s writing songs and sonnets instead. 

Or is it?

At a Sydney Ideas live talk hosted at the Opera House, Sophie Gee, Kai Riemer and Sandra Peter tell a combined history of writing and AI, exploring the uniquely human pains and processes that have always been needed to create masterpieces of art. They detail how AI ‘writes,’ as well as what happens mentally and emotionally when we employ AI in our own writing. 

Sophie Gee is an author, a Professor in the English department at Princeton University, and the co-host of The Secret Life of Books podcast. 

Sandra Peter is Director of Sydney Executive Plus and Associate Professor at the University of Sydney Business School.

Kai is Professor of Information Technology and Organisation at the University of Sydney and Director of Sydney Executive Plus.

You can watch the full live talk here.

Click the names to listen to full Solutionists interviews with Sophie Gee and Sandra Peter.

Mark Scott  00:00

This podcast is recorded at the University of Sydney's Camperdown campus, on the land of the Gadigal people of the Eora Nation. They've been discovering and sharing knowledge here for 10s of 1000s of years. I pay my respects to elders, past and present, and extend that respect to all Aboriginal and Torres Strait Islander people. Artificial intelligence has long been a staple of science fiction. But now that just about everyone is using artificial intelligence in some form every day, it's clear we've ended up with something very different to what we expected.

Sandra Peter  00:41

Literature, pop culture, Hollywood have gotten us all used to the very idea that if artificial intelligence was ever going to come, it would be accurate and objective and rational. The idea is that we were promised these really cold machines that would be accurate and objective, but would struggle with humanity, and what we ended up with is something completely different.

Mark Scott  01:08

That's Dr. Sandra Peter. She's an Associate Professor at the University of Sydney Business School and co-director of Sydney Executive Plus. Sandra's at the Opera House alongside her co-director at Sydney Executive Plus, Kai Riemer, Professor of Information Technology and Organisation at the University of Sydney Business School. They've come together to discuss how AI is challenging the very idea of what it means to be human, and to help illuminate that very human side of things they’ve brought along Professor Sophie Gee, a novelist, journalist, and scholar, and a professor in the English department at Princeton University.

Sophie Gee  01:46

Well, it's a little disappointing that, that this great art form that took Austen and countless others so long to master it can be replicated in a matter of mere seconds. When I first heard about ChatGPT being capable of writing, I thought to myself, "Well, that's insane”. And then when I saw it do it, I thought, "Whoa, that's a game changer. That's insane.”

Mark Scott  02:11

I'm Mark Scott, and this is The Solutionists. This recording was made at the Sydney Opera House at a live public talks event presented by Sydney Ideas, the University of Sydney's flagship public talks program. Sophie and Sandra tell a combined history of human writing and AI, pointing out just how ancient some of our current concerns about technology and humanity really are.

Sophie Gee  02:53

You think your pain and your heartbreak are unprecedented in the history of the world, but then you read. It was Dostoevsky and Dickens who taught me that the things that tormented me the most were the very things that connected me with all the people who were alive or who ever had been alive. That's James Baldwin, one of the greatest writers, social activists, and problem solvers of the 20th century, and the words that he gave in that interview, they hook you, don't they? They pull you straight in, as is the tune that Titus was playing, Over the Rainbow, which was one of James Baldwin's favourites. He used to play it every morning when he was getting his creative juices flowing before he sat down to write. So, Baldwin's alone with his heartbreak. He's isolated. He's trapped, and then he realises that he can face it in connection with others through the power of reading and encountering art, and it sets him free into action and activism. So the story that I'm going to be telling over the next few minutes is the story of why we need to feel our heartbreak in order to have the joy and the excitement of being in community with other people. The humanities, the history of the humanities, is the story of how humans have realised over and over again that we have to be able to enter other human minds through art and the imagination, if we're going to survive, and if we're going to get ourselves out of our current ghastly predicament. So, let's go back in time. We're in the late Roman Republic, and we're going to meet the hero of this section of our talk, Marcus Tullius Cicero, who many of you, well, of course, know already, the great rhetorician and lawyer of the Roman Republic. So it was Cicero who invented the phrase studia humanitatis, the study of being human, and he came up with it when he had to defend his old tutor and teacher, who was going to be kicked out of the Roman state for sedition. Cicero knew that he was really up against it with this case, because his tutor had some extremely powerful political enemies, and Cicero knew that if he fell back on the old tricks of rhetoric or legal defence, he wasn't going to stand a chance. So he went wildly off script. He made a very bold and audacious claim, and he made the argument that you cannot have a great state, you can't have powerful politics. You can't have virtuous citizens. You can't have happy people unless you have poets. And by poets he meant all writers and artists. Poets are the people who speak truth to power. Poets are the people who know how to use language in surprising, unexpected, arresting ways, and the chaos, the surprise, the emotional truth of human art, is what lets people truly see the world differently. What Cicero perceived in this speech so long ago was that the studia humanitatis were going to become the key to human transformation. Cicero was watching the rise of authoritarianism and imperial power, and he knew that the weapons against authoritarianism are human feelings transformed by an artist's creativity. He pointed out that the word humanitatis was the opposite of the word acerbitas, which means cruelty. When we embrace and study our humanity, we are going to stand together against cruelty and violence. So Cicero won his case, but he was right. He knew that the Roman Empire would not stand forever, and Western Rome fell under the weight of its own ambition and authoritarianism. For many centuries, Europe entered a polycrisis, escalating problems that were characterised by unpredictability, volatility, complex causes, and uncertain outcomes. We used to call this period the Dark Ages, but I'm rebranding it to a big, bright wake up call to the challenges that we're facing right now in the 21st century. 1818, the young Mary Shelley goes on a wet walking holiday with her husband Percy Bysshe Shelley and Lord Byron, and on one especially boozy drug-filled night, they sit up late, they tell stories, and Mary Shelley writes Frankenstein. Victor Frankenstein is a scientist who makes an artificial human. He trains it, but he doesn't train it right. Frankenstein's creature kills all the people Victor cares about. But Frankenstein isn't a story about how artificial intelligence destroys the humans who have created it. It's about what happens when humans don't respect and understand, and properly train the nature of their artificial creations. So we have to be really clear in our minds about what makes us human.  Heartbreak, chaos, uncertainty, and the ability to reflect on them together, with art. These aren't glitches in the human system, these are the greatest assets that we have.

Sandra Peter  08:49

But what if, what if every aspect of knowledge, every feature of our intelligence, could be so precisely described that a machine could be made to simulate it? We've been fascinated by building intelligent machines ever since the dawn of the humanities, and this is John McCarthy back in the day, who tackles on a most ambitious project. He gets together the greatest minds of his generation, computer scientists, mathematicians, cognitive psychologists, philosophers, and they get together to build intelligent machines. Ladies and gentlemen, we're now in 1956. A rock and roll revolution is happening back in 1956. I see what you're doing there. I see what you're doing there. Elvis, Heartbreak Hotel was topping the charts, and somewhere another revolution was brewing. It was all heartbreak, by the way, back in the day. There was heartbreak with James Baldwin, and there was heartbreak with the tech bros of 1956. At the very famous now Dartmouth conference, which, ladies and gentlemen, was not a conference, it was more like a two month long summer camp where 47 blokes got together, 11 of them showed up to the actual meetings. No women were harmed in the making of AI, ladies and gentlemen. 47 blokes got together to basically set up the science of building and the engineering of building intelligent machines, and their idea was that we could encode everything that we know into machines. The rules to playing chess, how to organise a birthday party, how to bake a brownie, or even what a cat is. The idea was that experts would encode, this is where coding comes from into machines. Every aspect of our knowledge, of our learning, our intelligence. So, if it was a cat, it would be an animal. It could be black. It could be white. It would have four legs and two ears and two eyes and a tail. And this worked. This worked extraordinarily well. Money poured into universities, defence money, government money. We built the first chatbots like Eliza that could fool people into thinking they were talking to an intelligent human on the other end. But then, very quickly, some cracks started to appear in our great plan because a three-legged cat from the pound, ladies and gentlemen, still a cat, so it has could have three cats. And Picasso's cat with the eyes all messed up, that's still a cat. And the bald cats they give people with allergies, still a cat. It was cats all the way down. This was the common sense problem. We would have to explain everything to the machine. So I would have to say that yes, Kai has come to the stage, and so have his on brand shoes, and Sophie's come to the stage, and her on brand necklace has come to the stage with her, and my on brand socks have also come to the stage. We would have to explain everything, and everything turns out is a lot, so the money kind of dried up, and people lost interest in artificial intelligence for quite a while. Nothing happens, except for the nerds in universities, like the three of us, who continue to try to think and come up with a new way of doing artificial intelligence, and by the late 70s, early 80s, we figure out a new way of doing cats. Seriously? Left that one right open. Thank you. A new way of doing cats and also AI. You will remember this because this was the last time we did cats well, ladies and gentlemen, back in the 1980s, and it's also the last time we had a brilliant idea about how to do AI differently. This carries us all the way to today to ChatGPT, because that brilliant idea was, let's not explain the world to the computer, let's give it data about the world, pictures of cats, and let it figure out patterns in those pictures. Heartbreaking moment, by the way. We realise we don't need experts, we just need brute force, lots of data and computing power. But over the years, we got exceptionally good at it because we got more and more data. Because you put your grumpy cat on Instagram and said my grumpy cat on this couch on the internet, and we took that data and we figured out how to encode patterns in this data and create a new wave of artificial intelligence. One more thing had to happen to tell my story about AI, and that thing happened just before COVID, the new BCE. Se what I'm doing there, Sophie? New BCE. We, we figured out that if you could find patterns in data, we could just flip those algorithms around and create data and patterns. Patterns like the ones we've seen before. And first we did that with faces because we just had a lot, a lot of data, a lot of pictures of faces because you take a lot of passport pictures and mugshots and other things. By the way, these are all pre-COVID fake faces created by the New York Times, who have a lot more time than us researchers at the University of Sydney, and probably a lot more money than us at the University of Sydney to sit around and create fake faces. We were really good at creating these patterns before COVID, and then during COVID, we had an enormous amount of time to think about, what else could we do that with? And we figured out well, technically you could also do that with text, so we did that with the internet. That is the New York Times website and the University of Sydney website and Wikipedia and Reddit and links out of Reddit. The good, the bad of the humanity. We put it in there and we figured out simply what words go with what other words in context. We got so good at it that by 2022, something else started to spread, a bit like a virus, and we all got it. By 2022, we had figured out a new way of doing patterns in data, we were generating something very, very different to what we expected in the first place that we would back in 1956. Because we always thought, well, we would teach the computer what a cat is, and the way we would encode it is with the letter C A T. We ended up with a machine that encodes numerical patterns in data. I'm going to end my bit on some absolutely gratuitous pictures of cats, just because we can. Thought you were just gonna keep going there. Very nice.

Sophie Gee  16:38

Oh, so relaxing. It really puts me in the mood for some classic literature.

Sandra Peter  16:43

Well, Act One should be writing.

Sophie Gee  16:45

Yeah,

Sandra Peter  16:45

That's why we are here. It is the humanities, so we are going to start with writing. And I thought, since it is writing, we should bring a couple of pieces of writing. There will be some audience participation, and at least one of these passages is written, at least one is written by Jane Austen. Is it the first one? I’ll let you read it out. I feel you are, you know?

Sophie Gee  17:11

I inhabit the role, don’t I?

Sandra Peter  17:13

Yes.

Sophie Gee  17:14

Yeah. Yeah.

Sandra Peter  17:14

Professor of English, so I'm somewhere.

Sophie Gee  17:16

I cannot fix on the hour or the spot or the look or the words which laid the foundation. It is too long ago. I was in the middle before I knew that I had begun. Is that Jane Austen?

Sandra Peter  17:30

Or is it the second one?

Sophie Gee  17:34

With every civility he offered and each glance of quiet regard, she felt her once resolute heart surrender, though she scarcely dared confess the depth of her regard. Or is that Jane Austen?

Sandra Peter  17:50

That's some beautiful reading. Let's hear some applause if you thought it was the first one that is actual Jane Austen.

Sophie Gee  18:00

And the second, you can keep applauding. Thank you very much. All will be revealed.

Sandra Peter  18:08

All will be revealed. Also, now we have applause for the end of the recording, in case that doesn't work out.

Sophie Gee  18:15

Sandra's always thinking ahead, and so was Jane Austen. So here she is. It took the real Jane Austen 10 years to write that sentence. She started writing Pride and Prejudice in the 1790s, and she finished writing it sometime in the early 1800s. And in that interlude, she suffered heartbreak, devastating loss, and having to completely remake her life because her father died, she was thrown out of her childhood home. She and her mother wandered somewhat aimlessly from relative's house to relative's house until eventually they ended up in a cottage of their own, where Jane Austen would practice the piano every morning at 6 o'clock, before she would sit down and start to write her books. Now, the fascinating thing about the way that Jane Austen did her writing is that she didn't like listening to kind of highbrow, highfalutin stuff of the sort that we would think Austen would like to listen to. She basically liked to listen to the kind of Taylor Swift of her day, the ballads and the airs, and this is what she practised every morning before she got going. So you can kind of see why it took her 10 years to write it.

Sandra Peter  19:38

Now, Kai, I want to bring you in here to see how the other passage got written, because if that's what Jane Austen's experience was.

Kai Riemer  19:48

So it took Jane Austen 10 years to write her bit. It probably took ChatGPT about five seconds to write its bit. But the question is how, how can it do that? So, how does a language model learn? We always, we do a lot of this in AI when we use terms like learn because it mathematically encodes the style of Jane Austenness. So we can have a look at what this looks like. So we're bringing you a baby GPT.

Sandra Peter  20:20

Big sizes, no?

Kai Riemer  20:20

It's a it's a small language model. Yeah, we're not being sizest, but it's only seen not the internet and Reddit, only seen the work of Jane Austen. Okay, so Baby GPT, we're training it to learn Jane Austen. So we're looking at a mathematical model, right? There's just numbers in there. There's, by the way, you know, big revelation. There is not a single word of text inside ChatGPT, right? It's all numbers. It's a mathematical model that encodes petals. So let's have a look, what happens when we train this model? Before we train it, it's just random numbers in the model. So we create a prompt, and I can't read.

Sandra Peter  20:58

“You must decide for yourself, said Elizabeth” is the prompt.

Kai Riemer  21:01

Thank you, is the prompt. So what happens now is we start training it, or the the training algorithm starts training it. It's an automatic brute force problem. That's the heartbreak moment. So it works in rounds. Every round we create an output. We do a mathematical distance comparison between the numbers that represent real Jane Austen text and whatever the output is. We change the model numbers, do another, have another go, and every time we increase the the probability that we get closer. Okay, so we do this,

Sandra Peter  21:32

250.

Kai Riemer  21:33

250 rounds.

Sandra Peter  21:33

Same prompt. You must decide for yourself, said Elizabeth.

Kai Riemer  21:37

No more hashtags, no more German umlauts. We're getting closer to English letters.

Sandra Peter  21:41

We try 500 rounds.

Kai Riemer  21:43

Yeah, small words appear. He, his, him. There's lots of that in Jane Austen time, apparently. We do, we do a bit more.

Sandra Peter  21:51

We're at 5,000.

Kai Riemer  21:53

5,000 rounds. We get close to sentences, bigger words. Takes about 30,000 rounds for the thing to sound like Jane Austen. Now, this is an entirely computational mathematical process by which the model learns to impersonate Jane Austen.

Sandra Peter  22:13

The same prompt. You must decide for yourself, said Elizabeth, rather repeatedly. This is a very agreeable displeasure that they will ever be a very lively. Makes absolutely no sense,

Kai Riemer  22:24

But sounds,

Sandra Peter  22:24

But sounds like Jane Austen now. It's learned Jane Austenness.

Kai Riemer  22:29

So there you go. This is how ChatGPT does it.

Sophie Gee  22:34

No piano practice at all.

Kai Riemer  22:34

No, the majority. Okay. So just pointing out that the applause was louder for ChatGPT.

Sandra Peter  22:41

Than the,

Kai Riemer  22:41

Yeah.

Sandra Peter  22:42

Than the real? So I have a provocation for both of you, and Kai I’ll probably go back to you because you know these two things looked very similar, but the process is clearly very, very different. So my first provocation to the two of you is, can AI write classics?

Kai Riemer  22:59

What we know is, we know is that we can encode the voice of writers, and we've just seen this. And we also know that humans are really not very good at telling the difference, right? So we've just seen this here, and there's lots of research. So Nature's published a study where people couldn't basically tell the difference between AI-generated poems and real poems. In fact, they actually preferred the AI-generated poems for rhythm and beauty, whatever that means.

Sandra Peter  23:28

So you're in good company. No one can tell.

Kai Riemer  23:30

So we've tested this with essays where people couldn't tell the difference. It's basically just random whether they can, can see it or not. However, interesting fact, and that's a Princeton study. When we tell people they look at the same text, we tell them it's written by a human, or it's written by AI. They clearly prefer the human-written text, even though they're looking at the same one by about 14%. By the way, you do that with ChatGPT. ChatGPT is even more biassed, prefers the human-written text in 35% of the cases, even though there's no difference. So clearly, there's something going on there.

Sandra Peter  24:09

Yeah, is that surprising, Sophie?

Sophie Gee  24:12

Well, it's a little disappointing that, that this great art form that took Austen and countless others so long to master it can be replicated in a matter of mere seconds. The question to me from all of this is sort of why does it matter? Now I think when I first heard about ChatGPT being capable of writing, I thought to myself, "Well, that's insane”. And then when I saw it do it, I thought, "Whoa, that's a game changer. That's insane. We're all going to be out of a job”. So, what I sort of want us to think about is, is, what, what is human writing? You know, what what actually takes place when we write as humans, and in a kind of correlated question, what actually happens cognitively when we read, because that those feel like the really crucial questions here. So, I think that there's a certain kind of fear surrounding AI, right? Which is, if this machine can do it, then do I need to learn how to make the machine do it? Do I need to stop, stop writing? Do I need to kind of abandon this craft. If I'm, if I or my children are interested in the art of writing, the art of reading, in humanities subjects, should I be throwing those things away? Because suddenly we see that we have these very powerful machines that can do it for us. And you know, as you've probably figured out, I'm going to say no. We shouldn't, we shouldn't stop writing. So here are some things that happen to us when we write. We have this default modal network in our brain, and it's kind of whirring away all the time. And it's the side of our brain that is our creative side, but it's also our very incoherent side. It's the side where the sort of slosh thoughts are kind of running through it, and it can be very free flowing, but it also can be intensely ruminative, and it's not particularly good at making sense or solving problems. Now, what happens when we write, when we either physically write or do computer writing, is that we activate neural pathways that integrate our prefrontal cortex and the emotional centres in our brain, and they help all of this activity in the default modal network to kind of settle down and to become structured. And the impact of that is to give us coherence, to give us a capacity for critical thought, to give us a capacity for emotional regulation. And none of these things do we have unless we are physically and cognitively engaging in the act of writing and correlatively the act of reading. So in other words, you might be able to generate writing from chat, and sometimes it might give us some pleasure. But what's really important is not the fact of having written something, it's not the generating of writing. It's this process that takes place when we as humans write.

Sandra Peter  27:07

We've got five more minutes for one last provocation, one last act that I want to bring in here. The last one is the most important provocation of them all because this is going to be all about thinking because on the one hand we do know that some of these technologies are actually really, really good at passing tests, and there's a 2025 study that I'm sure one of you will bring up where OpenAI aced most law school final exams, and we know that already in 2023 it was passing the bar exam. So this thing seems to be very good at taking exams. Makes you wonder where learning is, is heading. But on the other hand, I remember you said stuff about Cicero being a lawyer. So you know,

Sophie Gee  27:56

So could he have used ChatGPT to win that speech for him?

Sandra Peter  27:58

Yeah.

Sophie Gee  27:59

Well, I'm going no on that one. And I want to kind of go back to the explanation Cicero gave for why he went completely off script when he gave that famous defence speech, where the stakes were really incredibly high, and an actual human's fortunes were involved. He knew that he couldn't just go to kind of boilerplate or predictable arguments from legal history or from his bar exams. He said, in the presence of a crowded audience of citizens, my speech should be made in a style out of keeping, not nearly with the conventions of the bar, but also with forensic language. And so, what Cicero is picking up on there, obviously, is that for real transformative thought to happen, we have to be not predictive, not referring back to human patterns of thought or cognition or conclusion that we have already reached. But we have to be able to leap forward into new frontiers that can't be predicted and can't be foreseen. And this, I think, is where AI starts to become more of a problem.

Sandra Peter  29:06

That is my provocation, though. We can’t unfortunately, all speak like Cicero, so some of us are still using AI, and the idea of getting it to pass tests is really tempting. So my last provocation, and I'll hear brief arguments from both parties is, do we still need to think now that we've got AI?

Sophie Gee  29:27

Yes.

Kai Riemer  29:32

Yeah. You know. Well, first of all, I want to say AI isn't actually thinking, right? These model developers use a whole lot of human language to give the illusion that it's thinking, and even the models that do the reasoning or the thinking mode, and they sort of show you their thinking traces. Researchers at Arizona State did a study to try and see whether these intermediate sentences, these intermediate tokens, had any role to play in whether the actual outcome was better. And spoiler alert, no. They're basically generated to make you feel better and to pass the time that it takes to generate an output, a complex output, right? So these reasoning traces are nothing like reasoning traces. They're just probably just for show. However, still very useful technology to work to help us work through complex problems, right? And I think the key here is to not outsource your thinking. There's been interesting studies by Anthropic that have shown that programmers who work a lot with these tools to create their code had a severe drop of incomprehension in coding skills and things like that. So, you know, we might be able to produce outcomes, but we might actually erode our own skill and expertise. Which, ironically, because the people who have this expertise are much better using the tools, is the necessary ingredient to be effective with the tools. So the question is, how do we elevate and still keep our expertise that makes us effective in working with the tools, given context, and not erode that because we offload our thinking to the tools? And I think that's the thing that we need to to talk about. This thing helps us thinking, maybe, but we still have to think because we're the only one in this relationship that does the thinking.

Sandra Peter  31:30

Can you guess what that means for you? Have a think starts with a B? Balance. Again, treat this as an assistant, as an advisor, as a sparring partner, but do not let it take the wheel, right? Steve Jobs calls these things bicycle for the mind. Computers, you don't want the thing that's in Wall-E where the people just wobble around, right? Do not let AI take the wheel. AI can't think, and it's more important than ever now that we do that thinking.

Mark Scott  32:06

That's Dr. Sandra Peter, co-director of Sydney Executive Plus. You've also heard from her co-director, Professor Kai Riemer, and Professor Sophie Gee. For more great ideas from the University of Sydney, and to hear the full hour long version of this public conversation, visit the Sydney Ideas website. We've put a link in the show notes. You can also hear more from both Sophie and Sandra in our conversations from previous episodes of The Solutionists. Both of these episodes are linked in the show notes.

The Solutionists is a podcast from the University of Sydney, produced by Deadset Studios. Keep up to date with The Solutionists by following @sydney_uni on Facebook and Instagram.

This conversation is originally from a live public talks event, presented by Sydney Ideas – the University of Sydney’s flagship public talks program. You can hear the full hour-long version of this public conversation here.

This podcast was recorded on the land of the Gadigal people of the Eora nation. For thousands of years, across innumerable generations, knowledge has been taught, shared and exchanged here. We pay respect to elders past and present and extend that respect to all Aboriginal and Torres Strait Islander people.