Meet the Solutionists transcript and episode notes - season 2, episode 2

Mark Scott 00:01

This podcast is recorded at the University of Sydney's Camperdown campus on the land of the Gadigal people of the Eora nation. They've been discovering and sharing knowledge here for 10s of 1000s of years. I pay my respects to Elders past and present and extend that respect to all Aboriginal and Torres Strait Islander people.

Matthew Esterman 00:33

So, all I'm going to do is just test its basic knowledge about what a historian is. And I'm going to ask ‘What do you think a historian is?’

Mark Scott 00:43

History teacher, Matt Esterman is with his year 12 students at Our Lady of Mercy College in Parramatta.

Matthew Esterman 00:49

And let's see what it comes up with. So, it's thinking, and it's doing a little bit of web searching.

Mark Scott 00:55

He's teaching them how to use the artificial intelligence platform Copilot.

Matthew Esterman 01:00

And it's come up with ‘A historian is a person who studies and writes about the past delving into historical records events and developments.’ What I like about Copilot is it does give you where it's getting its information from, knowing that first one's Wikipedia, so we've got to be aware of that.

Student Ariana 01:12

I think that it reflects, like, an approximate perspective, which is, of course, what AI typically gathers. So, it's a pretty surface level, uncontroversial perspective.

Mark Scott 01:25

They asked the AI to answer again. But this time from the perspective of the venerable Bede, an historian from the early Middle Ages.

Matthew Esterman 01:37

Certainly, I love how positive it is at the start, by the way. Certainly, let's delve into the historical perspective of the venerable Bede. So, it's giving us a bit of an analysis of Bede.

Student Sophie 01:48

I think things can often get misconstrued with the way you ask questions. I know, sometimes if I'm entering something, you have to change your phrasing a couple of times before it delivers what you're particularly looking for.

Mark Scott 02:03

The students are learning how to finesse their question, so that Copilot can give them better information. This time they tell it to take on Bede’s voice.

Matthew Esterman 02:16

Oh, here we go. Certainly my dear reader. Allowed me to don the scholarly robes of the venerable Bede and illuminate the path that befits a historian's noble calling.

Mark Scott 02:35

I'm old enough to remember the rise of the pocket calculator. What a thrilling device it was plus and minus, divide and multiply. And for a couple of 100 bucks, it could be yours. During my school days, the full scandal of the pocket calculator erupted. If you used a pocket calculator in the classroom, you'd never master maths. And it would be the end of mathematics as we knew it. Well, we became used to that technology, and comfortable then, with computers at school. And the latest technology we've got to get used to is artificial intelligence. AI is causing concern. And there are risks. But just like in Matt Esterman’s classroom, AI can also transform how we teach and learn for the better. So, how do we get this right? I'm Mark Scott, the Vice Chancellor of the University of Sydney on The Solutionists, you get to hear from all sorts of incredible minds, the people unpacking the greatest challenges of our time, the people making change happen. Danny Liu works in the educational innovation team at the University of Sydney. He described himself as a programmer by night, a researcher by day, and an educator at heart. So, Danny, what's the biggest misconception about AI and education?

Danny Liu 04:11

A lot of people see AI as the next kind of technology, and it really isn't a technology, but they seem to have a perception that it just like Google with a database, as opposed to like a neural network, which actually is under the surface. And I'm going to do a great disservice to both neuroscientists and computer scientists here because I'm neither of those, but here's my understanding of how our neural network works in terms of how these modern AIs operate. So, these modern AIs are based around this thing called an artificial neural network, and interestingly enough, they were developed many years ago in order to study the human brain and how the human brain works. In our brains. We have neurons, we have biological neural networks with neurons connected to other neurons. And it's a very similar thing in AI too, we have artificial neurons connected to other artificial neurons too. With AI, or these modern AIs, what’s actually happening is your prompt is being transformed into an actual response, much like our brain would transform a question into an answer. And so, a lot of people say that one of the main problems with these AIs is something called hallucination, which means they make up stuff. And it's a really interesting point because these AIs are built to make up stuff. And that's what makes them so powerful now. It does make them a bit tricky to use, because often we rely on computers to give us factual information. Exactly. And it feels weird that a computer system is not able to be authoritative like that. But the very nature of how these artificial brains work is that they do make things up as they go. But because they've had massive amounts of training data feed into how these neurons connect to each other, there's actually a lot of, quote unquote, knowledge baked into that network.

Mark Scott 05:51

It took us all a while to just understand the power of online search, the power of Google, how is the new AI experience different to search, as we've come to understand it.

Danny Liu 06:03

I'll give you an example that I use with my kids. So, they're currently they're going to year two this year. Last year, they had news and one of the news items was ‘Tell us about your favourite animal.’ And so one of my kids said, you know, I like quail for some reason. And we could have just googled ‘quail facts for kids’ and then I would have to maybe wade through the top few results, and then kind of be able to figure out which of those National Geographic pages I should be reading, and which facts to pick for him. But instead, what we did was we open up ChatGPT and I got him, he's practising typing, to type in, you know, ‘quail facts for first grader.’ And then the GPT was able to give us, you know, 10 really interesting facts,

Computer voiceover 06:44

Quails are small birds about the size of a grownup’s hand. They like to stay on the ground. They're good at running but can also fly for short distances.

Danny Liu 06:54

He wasn't quite satisfied with those facts. And so, he's able to ChatGPT ‘What are the best facts about quails.’

Computer voiceover 07:02

Baby quails grow rapidly, and within a few weeks, they are ready to explore and find food on their own.

Danny Liu 07:08

And because these facts are things that are not, you know, third year university level facts, they’re just generalist facts which the neural network is able to know quite well, we could trust this. And it's also able to have those conversations with my son saying, you know, well, ‘what do you mean by best fact, you know, what is best? Here are some facts. But what do you mean by best?’ So, that conversational interface, the ability to actually understand what the human intends out of the question is really powerful.

Mark Scott 07:34

So, it's tailored and iterative, in the conversation that takes place even with your child?

Danny Liu 07:39

So, with the conversation you have with AI, the AI is in that same conversation, it's learning from your previous conversational turns. And because it's able to say, ‘Oh, your first question was this thing, your second question was this, I see where you're going.’ It's able to lead you in that particular direction.

Matthew Esterman 07:58

And then it gives you a couple of options down the bottom as well. Like ‘What inspired Bede to become historian? How did people view history and Bede’s time?’ That's actually a really interesting question. I going to click that one just for my own sake, ‘How did people view history and Bede’s time?’

Student Sophie 08:11

That's actually what I like, how it has almost this stream of consciousness, the AI, how, when you ask a question, and you ask another question it follows on.

Matthew Esterman 08:20

So, it remembers, it’s got working memory at least.

Student Sophie 08:24

Yeah, I know because once I entered in a different prompt, but I forgot to refresh it, and it got a little bit confused.

Matthew Esterman 08:30

Yep. And look, it's giving you some references there as well. ‘Yet their echoes faded in Bede’s shadow.’ So, other historians were kind of ignored because Bede was so dominant. Okay, there you go. Oh, look at even more emojis. An hourglass. A star. Isn't that lovely?

Mark Scott 08:49

When we think about education, we often think about needing to be right, the authority that comes with teaching. You talk a little bit about how AI might change how teachers work? And are you using AI in your classrooms?

Danny Liu 09:05

The question that we often ask teachers, when we speak with them is, ‘If you had a clone of yourself, what would you ask that clone to do?’ We often go through and say, you know, ‘Oh if only I had a clone, then I would get the clone to do X.’ And so, for teachers that clone might be, for example, ‘I would love an entity to be working next to my students 24/7 to give them feedback on an assignment.’ ‘I would love that entity to be able to roleplay with the students. So, if the student is practicing for their next clinical placement or their next teaching placement, maybe a roleplay there.’ ‘I would love for that entity to help me give better feedback to my students.’ ‘I would love that entity to help my students ask better questions.’ And so, I think with this new generation of AI because it can operate in such a multi-dimensional, human-like way, there's an increased ability for teachers now to actually command the AI to do these things for them.

Mark Scott 09:59

What do your students think about using AI in their learning?

Danny Liu 10:04

Some students are very into the AI. And so maybe 10% of students are very into the AI and they can teach us a lot of things about how to use it. The vast majority are waiting for us to maybe set some guardrails and guide them. And there's a minority who have never really experienced it before. And what they're saying to us is, they know it's part of their future. And they want the university, their teachers to help them grasp what this means for them, and how to use it productively and responsibly. And then how to apply it to, not just their study, but more importantly for their future once they leave us. So, students are really aware that this is going to be part of their - everyone's future, and they really need us to respond.

Mark Scott 10:47

Back in the classroom, high school student Sophie is using AI to get a jumpstart on research.

Student Sophie 10:52

I find it's really good to help put the thoughts that I have in my head, I type it into an AI tool, and it will give me a layout or a summary of information, which then makes it really easy for me to then go out and do my own research based on the summary, or the topic sentences, or the ideas it's provided for me.

Mark Scott 11:16

While Ariana uses it to structure her thoughts.

Student Ariana 11:20

Sometimes I will plug in the information I've gathered, the general outline and ask it to construct a paragraph. I will generally have to retype the, pretty much the entire thing anyway, the idea is to give me a skeleton structure so that I don't get caught up in breaking down the information, which has already been gathered. I do have a central thesis but at that point where I could get overwhelmed, AI can help me figure out a direction to go in.

Mark Scott 11:51

As tools like aI become more prevalent, I think there's an argument that says what our graduates will need to be doing is exhibiting the human traits. And the human attributes become more important as the machines can do more and more. What are some human traits that you think AI can potentially enhance?

Danny Liu 12:13

I was speaking with a journalism student last year, she was interviewing me, ironically about AI. And at the end, I got to ask her a few questions. And I asked her ‘As a journalism student who's about to graduate are you afraid of what AI means for your career?’ And without missing a beat, she said, ‘No, I'm not because journalism is not about writing. It's about relationships. And it's about people and connections.’ And that really struck me because this student had the answer to that question, in that what we are trying to develop in our students is not so much, you know, head knowledge, but it's the ability to work in and lead society. AI could take some of the writing away from her, some of the drafting perhaps, but then it would then free up more of her time in order to go talk to sources, to build relationships with families, and to actually inject the human into her work.

Student Ariana 13:07

Really, I think it's what humans have, rather than what AI doesn't have that makes a human perspective so important. It's the distinct humour, the concise, put together narrative humans can create that makes it outshine AI, because while AI is competent, it will inherently not have what a human quality brings to writing.

Mark Scott 13:40

When Open AI, you know, did their big drop, there was an immediate fear about the integrity of assessment. Would this tool allow many students, particularly when they're doing essays and assignments, have the machine do the work? Rather than do the work themselves? And what does that mean about assessing the integrity of learning? How are we thinking through those questions of cheating, and the use of these tools and assessment?

Danny Liu 14:06

The fear is real. And I think we need to acknowledge that. Because this AI works in such a human-like way, it's able to replicate much of what human students can and could do. I think we need to think about assessments in terms of how they serve the students when they graduate. A very key part of assessment is for them to learn the key skills and one of the key skills will be using AI well, and so a lot of our assessments will actually need to embrace the use of AI. So, the approach that we're taking here, which is also the approach that the regulator of higher education is taking too, TEQSA, is basically you might have two kinds of assessments. One kind of assessment is the ‘how we use AI’ it’s not whether but how, we know they're going to be using AI, we need to help students learn how to use it well, and the other one is the one where we confirmed their knowledge. So, the design of those kind of first type of really authentic, exciting AI-infused assessments I think is where we need to put our attention.

Mark Scott 14:56

And that's exactly how our teacher Matt Esterman is thinking about utilising AI, in ways to enhance a student's learning without doing the work for them.

Matthew Esterman 15:08

We've explored ways to basically undertake the research process with AI as a genuine copilot at certain stages. So, we have a lot of conversations about academic integrity and about, sure cheating and plagiarism, but actually talk about how do you show your own work in a world where work can be done so easily for you. And the reality is that a lot of syllabuses, full of content and stuff you have to remember and then regurgitate in certain forms. And that's what these tools are really, really good at, it's actually going to be a massive challenge for students to prove that work is their own. In a world where you can press a button and have an assignment done for you. I think there's usually a bit of a process or a method that you've got to show. It's like in maths where you got to show you're working, you get some marks for the final answer, but you also get marks for the working and I think so long as we're doing that, we'll be okay. But then there has to be a deep discussion about what cheating actually means. Because are we saying that any use of technology is cheating, because I can tell you the first thing students do, staff do, is go to Google and ask some questions. So, are we saying that we're all cheating when we do that? I'd say most people say ‘No, don't be ridiculous.’ And I'm like, ‘Okay, well, if that's the case, then we've actually got a bit of a spectrum happening here, as to where cheating actually kicks in for each individual task.’ If we're okay with parents helping out with assessment tasks, or tutors giving some feedback as they're developing their task. If we're okay with that, shouldn't we also be okay with students using AI to give them constructive feedback along the way? And if not, then we really got to include AI in a bunch of different influences, not just look at it by itself.

Mark Scott 16:39

Part of the fear of AI is whether in fact, it can embed and entrench bias and discrimination, how are we thinking through those kinds of issues?

Danny Liu 16:48

So, when it comes to bias with AI, one of the most powerful ways to show it, I think, is actually through the image generation AIs. And so, if you pull up any of the image generation tools, what you can type in is something like ‘CEOs standing in front of a hospital.’ And for the most part, what you get, and the tools are getting better at this now, but for the most part, what you get is you get a bunch of white men in business suits crossing their arms standing in front of a hospital. And that's a really powerful representation of bias, because it helps to, people to understand very quickly that the bias is baked into these systems in that, in the training data, the AI saw a lot of white male CEOs. And that's why they think that way. In a way, it's not the fault of the AI, because if we think about it, if you had a human being, and they lived in isolation somewhere, and they read books and saw movies, and in those books and movies, all they saw were doctors who were male. And then you asked that person, you know, tell me a story about doctors. And they would say, ‘He did this, and he did that.’ And so the bias is inherent in the data. So, I think it is upon us as educators to help our students understand this and understand the limitations of the AI, not just in terms of the bias, but also the environmental impacts, the privacy implications, the legal implications, corporate implications, all those things.

Mark Scott 18:05

It's often struck me that it's relatively easy to assess a student to find out what they know. And what they don't know. What's very, very hard is to understand why they don't know something. And you know, if you're giving children in primary school, a maths test, half of them might get the question wrong, but the reason why they got that question wrong, that could be as different as every child in the classroom. How does AI help give us insight into that question, not just whether you know or you don't know it, but why might you know? Because that strikes me often as the holy grail of assessment.

Danny Liu 18:43

So, if we take a step back from the assessment, and think about the learning that happens in the classroom situation, for example, or at home before the student hits the assessment, we can picture a near future where perhaps AI is working with the student in order to quiz the students to support them through Socratic dialogue, to support them asking questions or asking the right questions. In those situations, the AI acts kind of like a personal tutor. But importantly, it's not acting as the only tutor, the teacher is still in control of that personal tutor might do things like, you know, ‘Tell me the top 10 things that this student is getting wrong.’ So, that means the next time I'm with them in class, or I can come up to their desk and say, you know, ‘Hey, these are the two misconceptions that you seem to be working on and let me help you with them.’ And so, the idea is before they even hit the assessments to try and understand their misunderstandings, while we can still help.

Mark Scott 19:33

So, that means that before an assessment is done, it's almost like a pre-screening diagnostic that can take place to help teachers provide individual learning or remediation for students, whilst questions are being done. This could be quite transformational for, for learning at a fundamental level in schooling, shouldn't it?

Danny Liu 19:57

I can picture a classroom in the future where perhaps students are working in groups, you have 30, 40 students that tutorial, perhaps and you have one teacher who's got 50 minutes to work with all these students. And you have maybe pairs of students, each interacting with an AI. And this AI is supporting that teacher to teach. Maybe the AI is asking students some pressing questions, trying to unpick their misconceptions, trying to engage them in some dialogue to help them go deeper into their knowledge. And so, the teacher then might be able to go around while the students are interacting with the AI and ask the AI ‘What are the things that the students getting confused about? How can I help as a teacher?’ and then the AI being able to look at the conversation history is able to then say, ‘Okay, these are the top two things, help your students with this.’

Mark Scott 20:44

You've talked about your children, and how you've used Open AI with them to help advance their learning, and you're clearly very comfortable around the technology. Are you worried that an inevitable outcome is that students will spend more and more time in education, engaging not in a classroom with each other, but with a faceless machine?

Danny Liu 21:08

It's a very valid concern. I was speaking to a couple of students towards the end of last year, and I asked them, ‘What does AI mean to you? You're about to graduate, what does it, what are you afraid of? What does it mean for you?’ And they said to me, quite interestingly, ‘We're the last generation of students who can genuinely, hand-to-heart say we don't need AI to succeed.’ And it's an interesting thought, because as we think about how AI infuses or potentially infuses our classrooms, we envisage situations where, perhaps, students might be interacting with AI tutors. But I think what we need to make sure that we do is we need to make sure our face-to-face, live classrooms are full of human interaction. Some of the things that I think about, I have a bunch of, being a teacher alliteration works for me, so, a bunch of 'C’ starting words, the ideas around, kind of, community, connection, care, I think those are the main things that we need to focus on while we're in front of students and with students. And if the AI can support us in a small way to do that, then then that's good. But we really should, like you say, spend most of the time that we have with students, with students.

Mark Scott 22:20

Are the teachers that you are meeting, are they excited, by the opportunities of AI or are they daunted?

Danny Liu 22:26

I think that teachers are rightly daunted because it's another new technology they need to learn. And it's this thing that is moving so fast, that it's almost impossible to keep up. I read Twitter and LinkedIn every day, and I struggle to keep up with everything. So, I think there are teachers who will be leading the way. But we need to help the majority of our teachers be able to explore this in a safe way that they feel comfortable with, and help them to realise how it can change, you know, small parts of what they do. But I think primarily emphasise that it's not about replacing them as teachers, it's about, like you say, empowering them and making them more effective.

Matthew Esterman 23:05

There's very sensible, smart, strategic thinking going on around AI at the moment. There are people really pushing the boundaries and breaking genuinely new ground about what curriculum development and the development of teachers might look like. We've got to experiment, we've got to push the boundaries, and it will be jagged and lumpy and inconsistent across different individuals and groups and systems. But we've still got to try, because our students are in this world. They're not just graduating into the world. They're in the world that is now being more and more infused with AI. And we have to do our bit to increase their literacy around it, increase their awareness about what it can and can't do, and help them gain the assistance they need.

Mark Scott 23:47

This is The Solutionists and I'm Mark Scott, the Vice Chancellor of the University of Sydney. Danny Liu works at the forefront of AI and education here at the University. You also heard from Matt Esterman, who teaches history at Our Lady of Mercy College in Parramatta, and two of his students.

Student Sophie 24:04

I'm Sophie, I'm currently in year 12. At the moment, I'm most interested in allied health or medicine. However, my dad's a lawyer, and so he kind of, I have one eye on law just to just to see how that would pan out.

Student Ariana 24:26

I'm Ariana and I'm also working with Extension History. So, when I leave school, I actually plan on moving into conservation of artefacts at a museum. So, my work will continue to be linked with history as time goes on.

Mark Scott 24:41

Those girls going places, I hope to see them around here at the University of Sydney. Back in my school days, I could have never predicted that a maths teacher could be an international sensation with millions of fans. Eddie Woo was another pioneer of modern education and he shared some of his teaching secret with me.

Eddie Woo 25:01

I realised that a great teacher isn't so much someone who knows all the answers, but is someone who knows how to ask the right questions.

Mark Scott 25:11

Scroll back in your podcast feed to find that episode of The Solutionists and don't forget to hit follow, too we have some great conversations coming up, including a chat with Alastair Campbell, former press secretary, the UK Prime Minister, Tony Blair. He has a pretty good podcast himself. He's going to take us inside Number 10 Downing Street at some critical times in history.

Alastair Campbell 25:32

So, my pager went off, and it said, something like ‘Car crashing Paris. Dodi dead. Diana seriously injured. This is not a joke.’

Mark Scott 25:43

Make sure you're following The Solutionists so you don't miss that episode. The Solutionists is a podcast from the University of Sydney produced by Deadset studios. This episode was recorded at the Faculty of Arts and Social Sciences media room. And our thanks to the technical staff here.

Episode notes

Audio transcript

Production credits and acknowledgement

The Solutionists, with Mark Scott

Episode 2: Clones, cheats, and ChatGPT – How the AI revolution is reshaping education

Episode notes

Audio transcript

Production credits and acknowledgement

Listen to The Solutionists, with Mark Scott