AI And Creativity With Marcus Du Sautoy

Podcast: Download (Duration: 49:20 — 40.3MB)

Subscribe: Spotify | TuneIn | RSS | More

Artificial Intelligence will usher in a new era of what it means to work and create over the next generation, but does this mean that writers and creatives will be made obsolete? In this episode, Professor Marcus du Sautoy discusses the developments in AI creativity and why our stories could be the very thing that helps train AIs to be more human.

In the introduction, I talk about Chirp, the new audiobook promotion tool from BookBub; plus, my writing update and my latest self-narrated audiobook, The Dark Queen.

 

Today's show is sponsored by IngramSpark, who I use to print and distribute my print-on-demand books to 39,000 retailers including independent bookstores, schools and universities, libraries and more. It's your content – do more with it through IngramSpark.com.

Marcus du Sautoy is the Simonyi Professor for the Public Understanding of Science at Oxford University. He's a prize-winning Professor of Mathematics, a fellow of New College, and a Fellow of the Royal Society. He's also the author of several prize-winning books and his latest is, The Creativity Code: How AI is Learning to Write, Paint, and Think.

You can listen above or on iTunes or your favorite podcast app or watch the video here, read the notes and links below. Here are the highlights and full transcript below.

Show Notes

• Why creativity is at the heart of a mathematician’s work

• How AI is moving from learning about games to creating art and music

• Development of AI: Alpha Go beat Go champion Lee Sedol [BBC], Alpha Zero [Smithsonian]

• How humans and AI switch back and forth from being creative to being critical or making corrections

• On the AI version of NaNoWriMo, NaNoGenMo. Also, check out BotPoet, and read about OpenAI's text generator [Wired and The Guardian]

• What happens when AI becomes conscious?

• Copyright implications when AI gets involved in creating art

• On AI translations between languages

You can find Marcus du Sautoy at simonyi.ox.ac.uk and on Twitter @MarcusduSautoy. The Creativity Code is available now.

Transcript of Interview with Marcus du Sautoy

Joanna: Hi, everyone. I'm Joanna Penn from TheCreativePenn.com, and today I'm here with Marcus du Sautoy. Hi, Marcus.

Marcus: Hello.

Joanna: Thanks for coming on the show. Just a little introduction.

Marcus is the Simonyi Professor for the Public Understanding of Science at Oxford University, quite a mouthful. And he's also a prize-winning Professor of Mathematics, a fellow of New College, and a Fellow of the Royal Society. He's the author of several prize-winning books and his latest is, ‘The Creativity Code: How AI is learning to write, paint, and think,' which is a super exciting topic.

Marcus, why is a maths professor writing a book about creativity? I know some people might find that difficult.

Marcus: I think usually the words maths and creativity don't go together if you're not a mathematician. But if you're a mathematician, actually, it's a very important part of our work.

I think it goes to the heart of the fact that we're making a lot of choices, actually, when we're creating our mathematics. We don't want to just create mathematics that's true because quite often that's boring. We're trying to choose mathematics which takes the people who attend our seminars, the people who read our papers, and our journals, on a kind of emotional journey.

We want to transform them, to change them, to make them go, ‘Oh. I didn't realize those two things were connected.' And so in charting out that kind of journey, it requires a lot of choice, aesthetics, and creativity because you're having to go to places which are new.

So, what is creativity?

I defined it in this new book as something which is new, but that's not just good enough because that could be very boring. So it's got to be surprising and it's got to have some sort of value. It's got to kind of be worthwhile in some way.

And I think that's what a mathematician is trying to do, create new kind of truths about numbers, geometries. But it's got to be surprising, it's got to kind of move you in some way. And if it's got value as well, then that really wins the biscuits.

Creativity is something which is very important to me as a mathematician, perhaps more than a scientist. Scientists have to be quite creative, but they're bound very often by the physical universe that we live in.

As a mathematician, I've got much more freedom to be imaginative in my world, create new worlds that perhaps aren't physical, and I think that's much closer to being an artist than a scientist.

Joanna: I love this emotional journey of mathematics. And your book does that really well. I'm not someone who reads mathematics books much, but this crosses the boundaries, which is fantastic.

So let's come to AI, because I really discovered AI in a big way when AlphaGo beat the Go world champion. And I read in your book that that moment was pivotal for you as well. So let's just revisit that.

Why was that such a big deal in 2016?

Marcus: It was a big deal, especially for me as a mathematician, because I've always used the game of Go as a good analogy for doing mathematics. I think a lot of people thought chess was quite a good analogy and a computer beats the world champion at chess in the mid-'90s.

But I think why Go is more closer to being a mathematician is that you're not quite sure why you make certain moves. It requires a little bit of intuition, pattern recognition, a bit of creativity.

And so for me, I'd always use that game as a kind of protective shield against the idea that AI could do mathematics. So I watched this game with a lot of angst and existential angst. But for me, the most significant thing was not just that this computer managed to beat a world champion at this very complex game.

There was a moment in game two where the computer made a move, it's move 37. And I talk about it in the book because I think this is a kind of a pivotal moment, when all the commentators went, “Whoa, it's made a mistake,” because it was doing something that you're taught as a Go player never to do, which is to play on a kind of particular line on this 19 by 19 grid.

But as the game evolved, we realized that it wasn't a mistake.

It was a deeply insightful move. It was incredibly creative because it was new, surprised us, and it had value because it, ultimately, won game two for the AI.

So I think for me, that was one of the most exciting things. It enabled us to see how to play the game in a completely new way. So it was being creative, but not only that, the way this thing had been programmed was significantly different. And I think this is why there's a real kind of phase change in AI.

It's like water going to steam boiling. Because the program hadn't been written by a human and the human knew what it was doing. The human had written the program so the program could learn, adapt, and change.

And so, ultimately, by the end of all the training it did, we actually didn't know how it was making its decisions, why it was making its decisions. And this new AI, which we call machine learning because it learns how to program itself. It's a bit like a child who's born and in the past, the child had nowhere to kind of learn on, but suddenly, we've got this new AI, a child, which can learn by interacting with its environment, change and become something more than its parents as it were.

Joanna: And then what happened after that AlphaGo beat the human with the next iteration, Alpha Zero?

Marcus: In some ways, AlphaGo had been given the rules of the game, and had been given human games to play on. So it learned from what we'd done as humans. And so you feel, “Okay, well, it's extending our intelligence and creativity.”

But then, DeepMind, who developed AlphaGo, developed something called AlphaZero, where they just gave the computer the 19 by 19 grid, the pixels, and a score, and it had to learn how to play the game, the rules of the game. And so this was a kind of tabula rasa learning.

It didn't know anything. By the end of its evolution, it was actually better than the AlphaGo that had beat Lee Sedol. So this is genuinely exciting because it didn't have to learn from things we'd already learned. It started from zero.

That's almost true creativity; something from nothing.

It's very interesting it was able to do that. I'm quite surprised that without any sort of guidance that it reached such a phenomenal level.

And actually, it even learned to play chess in an afternoon and beat all the computers that are programmed to chess, and also kind of a Chinese version of chess. So this is exciting and, perhaps, a little bit scary for some people.

Joanna: Some people will now be listening going, “Oh, yeah, but it's still like a game, it's still Go, it's still chess.”

Give us some examples of how AI is also creating in music and writing.

Marcus: Yes, I agree with you. It looks a nice closed environment, the game of Go, and it is. And I think that's why it was a good place to start.

But now, this AI, if it can learn, well, why not expose it to other things, not just games of Go, but the art that we love, the novels we like to write, the poetry, the music?

Music is an interesting one because it's also quite a self-contained environment. If you think about it, it's got notes on page, certain frequencies, that's why there's a lot of connection between maths and music. So AI learning on what we've composed in the past and extending it has been very successful.

Somehow, AI always starts with Bach as the composer. They try and make more Bach, and partly because Bach is very algorithmic in the way that he writes his music. And I think that's one thing I wanted to illustrate in the book, that artistic creativity isn't as mysterious as we think it is. That actually there's a lot of kind of structure, pattern, almost algorithm in the way that we do our creation or pieces of art.

The book is partly showing why, actually, we're responding to things in the artistic realm because they've got that hidden structure that we're trying to unpick. So if we can understand that, then maybe the AI can go and extend that into other realms.

We've now got examples, for example, a jazz improviser, trained on another jazz musician's riffs, the AI learned those riffs but then extended the sound world of this jazz musician. And what's interesting there is the jazz musician said, ‘Look, I recognize what this AI is producing. It's my world, but it's doing things I never thought were possible.'

I think this is an example of the exciting role that AI can play in a creative's life because it's as if that jazz musician was stuck in a corner of the room with just a small light on, didn't realize that they were sitting in a huge great big hall, and the AI has turned the lights on and showed, well, look at all these other places that you can go to with your sound world.

Music has been an exciting progress. The art world, where there are some curious things like a new Rembrandt was painted because the AI learned what Rembrandt had done in the past, his use of light, the sort of faces that he likes to paint, and by it taking that information was able to produce something which I think is pretty convincing as a Rembrandt-esque painting.

Joanna: It was sold at Sotheby's, right, as well?

Marcus: Yes. At Sotheby's…or think it was Christie's actually, it was the first AI piece of art. And I think this is, again, interesting because Rembrandt, we've already got fantastic Rembrandts, we don't need anything new there. So I think, what we want is AI to take us somewhere new and exciting, not to reproduce the old.

This piece of AI that was sold at Christie's I think it was, it was created, actually, by making art into a bit of a game because it was using something called a Generative Adversarial Network or a GAN, and this is taking two algorithms which kind of compete against each other.

One algorithm is creating art which it tries to make new and not derivative, but not too new that you just don't recognize it as a chaotic mess. The other algorithm then says, ‘No, I spot that, that's very Picasso-esque,' or, ‘No, you've now gone into a realm that's not art.' And the two competed against each other and created something which was kind of a new sort of art, and that's what went on sale at Christie's.

Joanna: And that's the bit I think, is just like writers. I don't know about maths, but I think there's this generative, as you say, which is the creative mind, which is, ‘I'm back in first draft, that's a first draft thing.'

And then the adversarial, which we would call critical voice or the editor is the bit that goes through and kind of says, ‘Oh, no, that's not so great or that needs fixing,' or whatever. So that, to me, almost sounds human-like.

Marcus: I agree with you. I have quite a few quotes from people in the book, the painters especially, and a poet, Paul Valery, who talks about the fact that you need two people in your mind, one being super creative and throwing out ideas, and the other one being critical and making choices about, ‘No, that's not good. That is good.'

And certainly I do that in mathematics, and very often I will do that in partnership with somebody else. So we have a lot of collaborations and I have partners across the world that I create my mathematics with.

Sometimes I'll be the good guy, suggesting loads of mad ideas and then my colleague in Germany, he's the one who shoots it down, or I have a colleague in Israel, he's the generative one and I'm the adversarial one in that context.

So I think that we do use this paradigm quite a lot. I think it's interesting that AI has latched onto it as a powerful way to make new things.

Joanna: What about writing then? Because I hear my audience saying, ‘Yes, but an AI hasn't written anything.'

Tell us about the poetry, automated insights, and what's going on with writing.

Marcus: Your writers will be encouraged to learn that I think of all the arts that I looked at in this book, that I think writing is still the furthest away from AI being able to achieve anything like humans can.

But there have been some examples. People might remember a story about a new ‘Harry Potter.' So again, this is machine learning because what the AI took was the seven volumes that J.K. Rowling has written, saw the sort of sentences she writes, the sort of connections she likes making, and then produced kind of the beginning of an eighth volume.

But actually, here's a warning about AI because I looked under the bonnet of this piece of AI, and actually ran it on some of my own books to see whether maybe I could put myself out of a job or get the AI to write my book for me. And what I understood was, there's still a lot of human creativity going on in an exercise like that.

So the algorithm offered me, at each point, 18 different choices of words which could follow the word you've just had. And I had to choose which of those that I would write.

And this is a warning because I think a lot of the news stories love to say, ‘AI has painted a new picture. AI has written a Harry Potter.' And it doesn't make a good news story if you say, ‘Human writes with the aid of a computer,' and so the human gets kind of put to the side and it's sort of celebrated as a piece of AI.

Actually, I heard Demis Hassabis, he said a nice thing, he was the creator of DeepMind and AlphaGo. He said it's like in the turn of the century when everybody was just putting .com at the end of their companies to hype their value.

At the moment we've just got everyone putting ‘Made with AI machine learning.' So be warned that not everything is always AI.

But that's not to say that there are roles that AI can play in creative writing. Poetry has been a very interesting place because it's quite a constrained environment. Sometimes you're almost putting rules on yourself to push you into new ways of thinking.

I think that's often why I quite enjoy writing poetry because I have to think of something that matches what I've just done if I'm trying to keep a particular rhythm going. So, there have been some interesting examples.

And I suppose they're more successful because poetry has always had a gnomic quality. You're never quite sure what on earth this means. And so I think AI can get away with a lot in this environment because it can write something which you can think, ‘Well, that sounds kind of weird,' and it could easily be sort of human going off on some weird kind of path.

So I referenced a little exercise that you can do, which is trying to spot whether something's written by a bot or not, and it's somebody who puts forward some different poems composed by humans which actually sound quite machine-like, and vice versa.
Some of your writers might have been involved in the November Writing Month.

Joanna: Yes, NaNoWriMo.

Marcus: NaNoWriMo, exactly. My mum has done a couple of NaNoWriMos. But somebody came up with a cunning idea. So this is to write a novel in a month, really disciplined, pump out the words, but this was a kind of variant on that idea, which is, ‘No, just write a piece of AI, write a bit of code that will make the novel for you.'

So you spend the month not writing but coming up with code that will do the writing for you. So there've been some very interesting examples of that. And most of them, again, are quite derivative. They're taking things like ‘Moby Dick' and running it through a Twitter filter.

But I thought one of the most interesting was won by somebody called thricedotted, that's her pseudonym. And she wrote something called, ‘The Seeker.' The AI takes wikiHow, which if anyone's gone on wikiHow, it's how to ask a girlfriend out on a date or how to bake bread.

The code has thought, ‘Well, I want to learn what it's like to be human. So I'm going to go through these pages of how to on wikiHow and learn what it is to be human.' And the algorithm generates responses to the wikiHow pages.

Now, for me, this is most interesting because I think this is where creativity and AI is going to be richest, which is when an AI becomes an entity in its own right, and it wants to try and communicate with us, and we want to try and understand its world.

Why do we write novels? We write novels because we want to get inside the mind of the other or to share our minds with others. I think it's trying to solve the whole problem of consciousness, that we can't know what it feels like to be you or what it feels like to be me.

Our novels are almost like an fMRI scanner, reading into the brain of the other. So I think this will become most interesting when AI becomes conscious. And then we will need to hear their stories in order to understand what it's like to be that machine.

Joanna: People are going, ‘When AI becomes conscious,' which you know, I wasn't going to get here so fast. But there are lots of people who worry about AI becoming conscious, obviously, Elon Musk, Stephen Hawking, famous names, dystopian sci-fi writers.

You mentioned that story might be the answer to the evil AI. So maybe just talk about that a minute.

Marcus: Yes, because I think it has been a little bit too dystopian, and I'm hoping this book is actually a more positive take on AI and how AI can be a useful tool but perhaps go further.

If it does become conscious, then we're going to need it on our side. We're going to work together. And there was an exercise which…it took the idea of how to tell a story.

And actually, many of your writers might have watched ‘Bandersnatch' just recently, the ‘Black Mirror' on Netflix, where you get to make choices along the way about what the characters do. Of course, this is a very old idea, books I used to love as a kid, where, you know, ‘Turn to page 37 if you go through the left door or…'

What this research team did was to train AI on the way that humans tell stories. We tell stories which aren't too dystopian most of the time, or at least we tell stories about what it means to be human.

And then the AI, having trained on this, was let loose on a tree of possibilities of a story to tell. And what was encouraging was because it had learnt how humans tell stories, it chose a pathway that was more human-like, wasn't horrific choices which weren't emotionally involved. It took a pathway that humans responded to.

I think that if we can train the AI that's emerging, in a sense, to be empathetic by reading our stories, by understanding our art, and therefore, being sympathetic to producing something similar, then we might have an empathetic AI will not be one that will, hopefully, wipe us out.

I actually put the quote by Ian McEwan, whose response to 9/11 was, if those hijackers had been able to put themselves in the minds of the passengers on those planes, would they have been able to carry out the act that they did? And I think this is partly why we have art is to be able to share our different ways of looking at the world, and to try and to mix minds, and not separate minds.

Joanna: I love that. I love that story might be the answer to the whole AI thing. I just love it.

I want to ask you some technical questions that I think authors are really concerned with. So the first one is the copyright question. We have a new quote, ‘The macaque selfie, the monkey selfie,' which I'm sure everyone can remember. It was an item created by a nonhuman cannot be copyrighted.

What does that mean if we're using AI as a tool? If I feed the AI my 17 novels and it spits out something I can use, what happens with copyright?

Marcus: Yes, I think this is still a very gray area, and it's partly why I spent a couple of years on a committee at the Royal Society in London, looking at the impact that machine learning is going to have on AI on the future.

I think these legal issues are ones we just are not quite sure about yet. I think fundamental things like driverless cars, if it causes an accident, who is to blame? Is it the person who programmed the car? Is it the driver who owns the car? So I think similar issues come up with copyright.

If somebody writes a piece of code, but they take a material that the code is learning on which belongs to somebody else, and so the result is then a product of, say, your novels, but a bit of code written by somebody else, and so who owns the copyright there?

I think this is really interesting because I actually start the book with something called the Ada Lovelace Challenge. Ada Lovelace was one of the first programmers that was interested in the idea that this analytical machine that Babbage had made might be able to do more than just mathematical calculations, and she suggested music could be one of the things. But she cautioned and said, ‘Look, this will never be able to do more than the programmer who wrote it.'

And so that's the challenge of the book. Is that really true anymore? Because these things seem to be really creative. So if it's going beyond the person who's coded the thing, it seems to be creating things which are not what the coder expected. Is that still the coder's property?

Or should it start to be something else? If you think about the way movies are made, just generally the ownership of a movie, because there are so many people involved in that, it generally has to be owned by a company. So it's not actually a person, it belongs to a legal identity, which deals with the fact that there are many creative processes going in, and you just couldn't pull this thing apart, if everyone said, ‘But that line was my line'.

I wonder whether we're going to get to a similar sort of situation where we will have to recognize maybe some legal status for AI which will incorporate the creativity of the coder, the creativity of the things that are being learnt on. But I think we're going into unexplored territory here.

Joanna: I also wanted to ask about translation because this is something I'm really super excited about. Because just last year, a translation AI translated a nonfiction, and that's important, a nonfiction book, 100,000 words in 30 seconds into Mandarin, from English to Mandarin, and then an editor took a week to clean that up.

It would have taken six months, apparently, to have translated with a human. So I wonder about that because in that case, surely the AI did the first draft, which is normally what an author does, and therefore, that just really confused me.

I'm very interested in what's going to happen with translation with AIs. What are your thoughts on that?

Marcus: Language translation has been very successful, and it is a great thing for machine learning to work on because, in the past I suppose a translation would be top-down coding where you would say, ‘Okay, well, you've got a dictionary, you translate things'.

But very often subtlety of sentences means that just a simple translation of each word using a dictionary doesn't capture it. And the language tools now are being very effective at really capturing the meaning of a sentence because they learn on the way that we use language, but they still aren't perfect.

That's where you said a human had to come in and tidy it up. And I think one of the things I kept on hearing when I did my research for this book was the words, ‘Good enough,' that the AI can produce music which is good enough for, say, a game or a corporate video, but isn't going to be performed in a concert hall.

Again, when it came to translation, the translations were good enough to communicate the message that somebody was trying to write in one language, but if you really wanted the full subtleties then you needed a human to come in.

There's an interesting guy that I've always been very interested in called Douglas Hofstadter, who wrote the book, ‘Godel, Escher, Bach.' He's actually been looking at AI for 50 years or so, but he's very down on AI as far as translation goes.

He produced some very interesting examples which just throw a computer because they just don't understand context. Things between languages, for example, you know in English, we don't have…words don't have a masculine or feminine form, but in French they do.

That can cause real problems when you start translating because if you say something like, ‘His car and her car, his house and her house, his book and her book,' that translates very difficultly into French.

And you've picked up on one thing which I think is quite exciting that, although computers are very good, humans are also very good. And actually, it's going to be the combination of the two which is best.

If you go back to the game of Go, which we already talked about, if you combine a human with the AI, then together they can beat, certainly a human, but they can also beat the AI on its own.

We've seen this also in medical research as well. AI, one of the big things it's being used for is in health care. It's able to scan pictures and pick up tumors, for example, which are being missed by human radiographers.

But again, the combination of a human and the AI seems to be better than both of them. So I'm hoping that's the future, that we're going to use this as a very powerful tool to speed up translation, but it won't ever be as good as a human in picking up the subtleties of use of language that the AI is just missing.

Joanna: Fantastic. A bit earlier you said it's a bit like when everyone stuck .com on the end of everything and of course, then you're probably talking 1998, '97, '98 to 2000, you know, that kind of .com boom. But of course, we are now, gosh, nearly 20 years later, and you and I are talking over the internet, I run a business on the internet, you're collaborating over the internet, and we all are in a .com world.

Marcus: Yes, yes.

Joanna: Are we talking really fast change? Are we talking 20 years? What did you conclude? Are you out of a job? Am I out of a job and how fast?

Marcus: I think speed is very important here because people are comparing this revolution to something like the Industrial Revolution, which had a massive impact on work and people's lives and caused a lot of poverty.

But the Industrial Revolution happened over a generation. It was your son or daughter that didn't get the job that you had. I think the speed of this revolution is way faster. And I think that what we're doing now, 10 years time, we will have to be doing something completely different.

We have to be ready for change. We have to know how to learn new things, which I think is exciting. I enjoy the challenge of not getting stuck in my ways and having to do something new. But I think that's where AI is gonna help us. I think that too often we get stuck in our ways.

And actually, we end up behaving more like machines than the machines because we just keep on churning out the same sort of things. We get stuck in certain formulas for the way we write or the way we think. And AI is being able to analyze what we're doing and suggest to us new pathways. Oh, maybe you could try this, maybe you could try that.

We might not like all the suggestions but some of them may resonate and take us off into a new direction. So I think that's the really exciting, positive side of this AI, that it's going to open up huge possibilities in our creative process that are kind of sitting there ready to be ignited, but we didn't know were there.

Joanna: I'm excited too and that's why I wanted to talk to you, I was like, ‘Yay, someone else who's excited about our future.' So thank you so much for your time.

Where can people find you and your book and everything you do online?

Marcus: Well, all my books in the UK are published by 4th Estates, who are a wonderful publisher. I've loved them and stuck with them all the way. And I have a website… So I'm the Simonyi Professor for the Public Understanding of Science, as you said, quite a mouthful. So I have a website where I put a lot of the activities that I do, radio work that I archive, television work, and also my books. So that's www.simonyi, which is spelled, S-I-M-O-N-Y-I, .ox.ac.uk. I'm also on to Twitter, where I kind of use as a microblog, so that's @marcusdusautoy.

Joanna: Thanks so much for your time, Marcus. That was great.

Marcus: Yeah, real pleasure.