Taking a JEDI approach to AI

With roots in the film and TV industries, Narrative Muse founders Brough Johnson and Teresa Bass knew all too well that the industry wasn’t exactly friendly to women. 

But it was only when they dug into the proportions globally of films, TV shows, and books by and about women, as well as gender-diverse people, that they realised how underrepresented these voices were. Determined to change this, Johnson and Bass set to work. 

“While this is common information in 2024, this ‘ah ha’ moment took some time to unravel – #Metoo and the broader conversation around the lack of gender diversity in media hadn’t happened yet,” says Nigel Lopez-McBean, Narrative Muse’s co-CEO. 

With a goal of uplifting underrepresented voices, Narrative Muse launched in 2016 as a review site built with the help of passionate volunteers. It has since become a platform for tailored book, movie and TV recommendations that match an individual’s taste and identity preferences.

The startup was global from day one, with users and volunteers primarily based in the UK, US, Canada, Australia and New Zealand. 

It didn’t seek to be a global company; it just happened, says Lopez-McBean. “My theory is that the solution was needed everywhere, and people from around the globe were seeking it.” 

Once launched, Narrative Muse found support and investment know-how in Angel HQ founder Suse Reynolds, who helped the startup raise its first round. This allowed the company to offer paid work to volunteers and hire its first employees. 

Since then, Narrative Muse has raised additional funding from angel, venture capital and family office investors, as well as through crowdfunding and grants – including $500,000 from the Ministry for Culture and Heritage Manatū Taonga in 2022. 

The interest in Narrative Muse led it to launch its Content Analytics Suite, which provides insights to screen and publishing industry professionals on what audiences are actively seeking to read and watch.

While these insights are helpful for market opportunity analysis and audience planning, Lopez-McBean says Narrative Muse’s vision still holds true. 

“The analytics help content producers prove the value of stories by and about underrepresented people, [expressing] the untapped market of readers and watchers seeking fresh stories with fresh characters representing them and others,” he says. 

‍Lopez-McBean discussed with Caffeine how Narrative Muse uses AI through a human lens. 

‍

How does Narrative Muse use AI?

We want to help people find content that reflects their tastes and identity. To do this, we need to know the qualities of the content audiences relate to, which requires a certain amount of data. 

Take film as an example. You need to look at the character traits, the locations where the story takes place, how it feels to watch, any triggering topics and so on. 

Traditionally, we’ve done that by working with people, watchers and readers of the stories, who then curate the information. As you can imagine, that’s fairly labour intensive, so we’re now developing AI techniques to support the data collection process. 

In saying that, humans will always be the gold standard. AI will never replace people-based curation; for us, it is additive to our content curation process. 

We tend to use the phrase ‘AI-assisted’ to emphasise that people are always the leaders and decision makers.

‍

How are you fostering AI literacy within your organisation?

As a start, our team works collaboratively on all our projects, which is helpful because it means we can share resources on the state of AI. Our data science team members are always interested in how AI is progressing in other spaces in case there is anything we can learn from. 

The entire company also partakes in a justice, equity, diversity and inclusion (JEDI) programme, which means that all decisions within the company, including [about] AI, are considered through that lens.  

We consistently ask ourselves if there are any ways and means that AI could negatively impact justice, equity, diversity and inclusion for our stakeholders and users. The JEDI lens has been transformative, helping us view the technologies critically, understand their context and question their design and implementation. 

‍

What has been your biggest challenge?

Combining natural language processing (NLP) with large language models (LLM). 

LLMs are large neural networks trained to recognise and predict patterns in written texts, but that doesn’t mean they’re any good or particularly accurate. They’re terrible at reasoning and synthesis, which is what humans do really well.

One of the big challenges has been designing programs that can meet the level of accuracy that human readers or watchers can. 

We’ve spent quite a lot of time trying those approaches using prompt engineering processes and ChatGPT. Our results have been okay, but we know that prompt engineering has limitations preventing it from providing accurate results.

However, when we’ve used fine-tuning models, which train an LLM to analyse the text of a book and accurately provide answers, our results have been really good. We’ve got to 99 percent accuracy between AI being able to categorise content data in the same way a human can, albeit on a small set of purchased titles. 

It shows there is potential to create this and do it artfully.

How do you manage privacy concerns with AI?

Historically, using text to train classifiers was considered fair use because the classifier did not produce new texts. 

However, generative AI is a whole different thing – creators are rightfully angry that their content can be used to train generative models, which can then be asked to write text that mimics their style without their consent. 

We are hyper-aware of those issues within the creative community. Our use case for AI is classification: we do not and will not generate any stories; we only classify them using a fine-tuning method.

‍

How does government AI legislation help or hinder your efforts?

It’s not really a challenge for us, but it’s helpful to know that different geographies take different approaches to AI legislation. 

Ideally, the public and private sectors could work together on how best to use the technology. 

‍

Are there any emerging AI trends you are watching? 

Any LLMs that get good at reasoning or synthesis are worth watching because then you’re in human territory. 

My personal education involves monitoring a US company called Palantir. It’s a US tech company that offers great insights into varying versions of the AI future. Understanding what they do is a bit of an education in real-world AI applications — both positive and negative. 

‍