How artificial intelligence can aid and replace higher order human creativity

Computers ‘set up to be creative’

Ed Newton-Rex, who composes for choirs, says JS Bach demonstrated how the greatest creative artists draw on a wide range of qualities.

“It wasn’t just his knowledge of music, although that was a big part of it,” he says. “It was also his fervent religious belief and very high sense of academic rigour.”

Until machines can encompass these, they are not likely to compose anything rivalling the Goldberg Variations.

Nevertheless, as the founder of AI music composer Jukedeck, Newton-Rex thinks computers are capable of creativity. He says creativity can be defined as immersion, assimilation and recombination.

“You could say computers are quite well set up to do that. They can immerse themselves in data, they can assimilate features of that data, they can recombine that data,” he adds.

Jukedeck, a London-based company set up in 2010, trains its neural network AI system using Midi data of out-of-copyright classical music and material written by its staff, all of who are musicians.

Newton-Rex says they use a lot of music theory: “The biggest thing we do with our time is [learn] how to set this system up to learn music most effectively. One of our big theses is that there are very common traits among all musical styles. Certain musical concepts – such as the idea of key, scale, repeat, pattern or inversion – are common across many different kinds of music, even though they are used differently.”

Some of the company’s AI-generated music has been mistaken for human-composed material, effectively passing a musical Turing test, although Newton-Rex says there is still a lot to do.

“We’re a huge way off being at that stage all the time. There are so many elements of music our system can’t deal with,” he says. “We have no delusions of grandeur there. It’s also not the aim. We’re a group of musicians, we’re not out to replace musicians.”

Instead, the aim is to do things human composers cannot do, such as creating large quantities of music or personalising it. At present, Jukedeck’s music is used largely as backing for video material, although it is also collaborating with a group of South Korean K-pop music producers after finding users who sing over music they create.

“We think AI composition can be incredibly useful to both amateur and professional musicians,” says Newton-Rex. “Essentially, you have the ability to give everyone in the world their own personal composer, or personal composing assistant.”

Fill in the gaps with AI

Drew Silverstein, co-founder of a New York-based equivalent outfit Amper and also a composer, sings from the same hymn sheet. “Our goal is to empower anyone to express themselves creatively through music, regardless of their background, level of experience or access to resources,” he says.

For a video editor or a game developer, Amper can do most of the work, he adds: “If you’re a musician, composer or artist, you have a tool to further your existing creative process, either to fill in the gaps or to make your creative expression more efficient and more productive.”

The company currently provides free access to a beta version of its AI composer, aimed at generating music to go with video footage. This generates music to an exact length based on styles such as “cinematic” and moods such as “dark dramatic” or “uplifting atmospheric”.

Unlike stock music, it will not have been used elsewhere, which should also avoid problems over copyright – human composers sometimes use elements of in-copyright work unwittingly, and then have to pay retrospectively to license it.

But training an AI system in creativity has its problems. “Unlike almost all other endeavours to which AI is applied, which tend to solve problems with objective answers – is this a dog, is this a cat, do I have cancer, do I not? – music is subjective. There is no right answer,” says Silverstein, calling this “the greatest challenge in creative AI”.

To this end, Amper collects feedback from users with the aim of finding what works for them.

Silverstein draws a line between functional music, which Amper is designed to generate, and artistic. “Artistic music is valued for the collaboration and creation that goes into making it. Functional music might be played in an elevator, versus the score to a new Star Wars film,” he says.

Technology in general will not replace the value of creating art and collaborating with people. It’s not going to happen

He hopes that those creating artistic music will adopt Amper as part of their toolkit, but adds: “Technology in general will not replace the value of creating art and collaborating with people. It’s not going to happen. It’s what makes us, us.”

While Amper’s output can pass a musical Turing test, Silverstein thinks people value human creation and collaboration. He compares it buying a cheap cup of coffee: “It’s a functional need, we need caffeine to get on with our day. Yet, in New York especially, we still find people who go to boutique coffee shops and pay $20 or whatever for a cup of coffee. It’s about the process by which that coffee and caffeine was created.”

Novel-writing robots

Away from music, others stress the need for software and humans to collaborate. Last year, a group of academics at University of Antwerp’s computational linguistics and psycholinguistics research centre were invited by a society that promotes writing to get a robot to write a book.

The group replied it would be more interesting for a machine and a human to collaborate: “The story could be more complex, and would reveal to the public more about both writing and AI mutually,” says Ben Burtenshaw, one of the researchers.

Burtenshaw says there are problems when evaluating creative text, unlike functional natural language generation where there may be a single correct answer.

“[In fiction] sentences with completely different contents could be equally right or wrong,” he says. “It’s important to get down to deeper and complex forms of evaluation, akin to the processes a writer goes through as they edit their text.”

The group worked with Dutch author Ronald Giphart to write a short story, providing him with a text-editor app trained on eight models, each one based on novels by specific authors that provided suggestions based on what he wrote.

“Giphart could then incorporate their writing into his own, and edit it to fit his plan for the story. We would later analyse how he edited this text to understand some of the more nuanced characteristics of our models,” says Burtenshaw.

The researchers assumed that the model had done something wrong if the author extensively edited a sentence. “We later learnt that this wasn’t the case, and Giphart could simply be changing the tenses of the verbs because he wants a sentence about the future, not the past,” says Burtenshaw. “We’re working on experiments to analyse these changes in a way that does reflect quality.”

They also opened the system to the general public, holding a writing competition inspired by the works of Isaac Asimov, which saw people change spellings to ones using dialect rather than standard Dutch – mistakes, but arguably creative ones.

Burtenshaw dismisses the idea that software is currently capable of consistent or deliberate creativity. “Computational systems have been able to produce text and images that are interesting for a while now, but it rarely resonates against a meaningful context, and when it does, it’s often by chance,” he says.

Research suggests that people struggle to relate to computationally creative work – although they are also suspicious of software collaborating with humans. “In short, I don’t think software can be creative until we let it,” says Burtenshaw.

The group’s system has been trained on sci-fi novels, so uses this voice regardless of the task. “It would write your legal contracts like a sci-fi novel, which could be interesting,” says Burtenshaw.

But the same method could be used to help produce legal contracts, by learning their language, structure and norms then generating a draft based on a human user’s notes, he adds. “This would be quite effective in a domain like law, where terminology is very specific yet unnatural to most writers,” he says.

Robot Rumpoles

Some law firms are making use of AI, including to automate analysis of text. Kemp Little, a London-based firm focused on the technology industry, has trained contract analysis software from Kira to find clauses that may be affected by introduction of the EU’s General Data Protection Regulation (GDPR) in May, and is developing it for similar work on Brexit.

Emma Wright, a commercial technology partner at the firm, says such clauses cannot be found simply by searching for certain words or phrases.

“Lawyers have had to train it and it reduces the review time afterwards. Normally you’d have junior lawyers reading these contracts from back to front, maybe using ‘find’,” she says, adding that the AI software is faster and more consistent.

She says the next step is to tackle new issues presented by clients “based on our understanding of the underlying technology, our abilities to train the tool, our datasets and our understanding of what we’ve already done, to configure the AI to the client’s problems”.

The firm has already used the software for a client that had suffered a data breach and needed to check a lot of documents quickly and thoroughly.

“AI won’t replace the lawyer, but it will make us far more efficient and consistent,” says Wright. The same may turn out to be true of composers, writers and others who work creatively.