​​Redefining Personalisation & Interactivity in the Emerging Post Pandemic World

Personalisation and interactivity have been an integral part of Child-computer Interaction, or CCI, since the beginning of this field of study. Now, it’s time to see what they have brought to the field and what they will mean in the emerging post-pandemic future.

During my Amplified Publishing research as an industry partner, I had the great pleasure of interviewing two fascinating women in publishing – the founder of Shelby x Studios, Grace Kress, and the author, journalist and comment editor at The Bookseller, Molly Flatt. While I include a few quotes from the interviews in this article, you can find the interview transcripts as a PDF at the bottom of the page.

Re-defining personalisation

So, what is personalisation? During my research, I’ve noticed that the answer isn’t as simple as it might at first seem. This is due to personalisation and customisation often being treated interchangeably (and, needless to say, they can vary between different fields of publishing), which can be a little confusing. Therefore, for the purpose of this article, let’s take a moment to redefine them.

Personalisation is tailoring the content to a person based on the data available about the user/audience (for example, personal data, such as name, location, time of day or weather in an area). Customisation is allowing the user/audience to tailor the content without the need to share any personal information. For instance, when browsing the internet we often get personalised ads, but when we’re playing a video game we can customise our character.

Re-defining interactivity

In a similar way to personalisation, interactivity is often treated interchangeably with gamification, but, while gamification is based on it, there’s much more to interactivity than that. Interactivity is a process of two people or things (for example, Human-Human, Human-Computer or Computer-Computer Interaction) working together and influencing each other. Gamification is a method of utilising gaming elements (such as progress bars, goals and quests) in non-gaming environments in order to increase engagement, enjoyment, immersion etc. For instance, children’s digital bedtime stories are often interactive (whether physical, voice-based or through any other means) to provide a higher level of immersion while maintaining the quality wind-down time needed, but fitness and language apps use gamification to keep users motivated and engaged throughout the programme they offer. With that in mind, let’s explore what digital storytelling methods have to offer.

Technology as a toolset

In the field of Amplified Publishing, we can see that technology provides a set of tools for creators and publishers, allowing them to shape, form and empower content to give their audience the most suitable output. Due to its long and iterative history, a good example of utilising technology to shape the form originated in the 1970s genre Text-based Adventure Games or Interactive Fiction (IF), which later defined the home-computer entertainment (and was a genre often discussed during my time in the Pathfinder). As Grace noted:

“There’s something about interactivity in publishing that feels really exciting. I like the idea that there is a relationship between the content creator and the audience and that these change through that process, the audience become the content creators and vice versa. I think interactivity can ignite a sense of play, which helps creativity to thrive. Digital publishing opens up new avenues for interactivity and I’m interested in how these features can help make things more accessible too.”

If allowed digital literature to take advantage of personalisation and interactivity in a new way by basing literary adventure on user input and creating a unique experience for each reader. Users were able to choose the directions, behaviours, inventory objects and much more, shaping their stories and overcoming challenges in different ways with various outcomes.

Evolving over the years, this genre was later empowered by many technological advancements, such as artificial intelligence (AI), resulting in a unique and personalised guided literary experience. An example of such empowerment can be seen in AI Dungeon, a Text-based Adventure powered by OpenAI. AI Dungeon allows users to create and share custom adventure settings, according to which AI generates content. The user can choose the narrative genre, character dialect, actions, dialogue sentences and story events, alongside many other customisable features. These allow both the user and the artist to create a tailored digital storytelling experience through the use of empowering personalisation and interactivity.

There are many great examples of digital storytelling that use the same principle of tailored and unique experiences through interactivity and personalisation. During our interview, Grace highlighted an especially interesting one:

“I’m also hoping to commission Danielle Brathwaite-Shirley, an Animator/Artist that creates work centering Black Trans people, to create a zine that people would interact with differently based on their identity. A kind of choose your own adventure.

That kind of personalisation is maybe about challenging people’s perceptions, affecting the way they navigate spaces and asking them to interrogate how their identity impacts the way they experience things.”

However, despite the benefits that personalisation and interactivity bring to digital storytelling, there are concerns around using them in a wider publishing space.

Two sides of personalisation and customisation

Just like all the technological tools, personalisation and customisation have two sides – they can both bring benefits to publishing and also generate concerns or controversy. On one side, personalisation is often seen as a way of allowing the content creator to adjust the world of the story based on the user, explore the narrative in a unique and individual way, or provide a more immersive experience by customising the protagonist. Alternatively, it can also be seen as a disruption within the story structure, an innovation at the cost of quality, or, more worryingly, an encouragement for the user to base the story world on their own life and lose the possibility of personal development through exposure to new experiences and challenges.

Discussing this subject can raise questions such as: if you can change the path or outcome of the story, wouldn’t the quality of certain paths/outcomes decrease? Does adding personalisation to linear storytelling increase the quality or is it an artificial innovation without a clear need? The purpose of using personalisation and customisation might be one of the main concerns of publishers and artists and Grace explained why that could be the case:

"Is it used to increase engagement and interaction or is it just a shiny add-on? Ultimately, for me, it comes back to intention. If we’re using personalisation and customisation to further meet individuals needs then sure, it’s a great idea and these tools can really help people engage in meaningful ways. Too often though it’s about increasing profit margins and people just become pawns used to make sales."

Those two sides, however, tend to be connected to the form chosen for the content – the digital or the physical. The challenges both forms face with the use of personalisation and the ways they overcome them defines the significance of personalisation. Equally, the difference in the use of this method can help us to understand the increasing dissonance between both forms (physical and digital content). While the digital thrives under the challenges personalisation brings, the physical struggles to adapt in as meaningful a way as the first medium.

During our interview, Molly Flatt summarised it perfectly:

“I think personalisation can be amazing in publishing. I love companies such as Wonderbly or Picabook, that are using personalisation to help a range of kids and families see themselves more closely reflected in books, and I enjoy how projects such as Visual Editions’ We Kiss The Screens and Breathe can help reflect your choices and perspectives back at yourself. But I also think people turn to books - particularly novels - to immerse themselves in others’ lives, preferences and viewpoints; to escape, challenge and disrupt themselves. They use books to flex their empathy, to go deep into a stranger’s vision or version of reality rather than their own. I think that’s hugely important, now more than ever. And I’m not sure there’s any other art form that does it so well. It’s a sort of superpower.”

Empowering the form

Personalisation (much like interactivity) can be a powerful and empowering tool when meaningfully used. It is not all about putting the audience in the centre of action, creating a protagonist identical to the recipient of the content, or allowing the audience to choose which path the story takes. As storytelling tools, personalisation and customisation are determined by the context they exist in. For instance, this might mean a user creating their own character in a video game or changing which vegetable children dislike in a movie based on the country the movie is localised for (as in Disney’s Inside Out). Despite the variety of forms in which it is used, the overarching purpose of personalisation is to provide a context familiar to the recipient in order to make it more relevant.

It is also worth highlighting that personalisation and customisation tools are not only used during the creation process but also in the processes of discovery and distribution. Sellers use these tools to better understand the audience and to personalise the experience for customers. And as Molly notes:

“Bookshops, such as Daunts, offer personalised book subscriptions where their booksellers tailor monthly book mail to a customer profile. And I suppose you could argue that any use of personal data to target something is customisation - such as Amazon or Audible's recommendation algorithm, for example. That’s huge. There’s a lot of talk around publishers and booksellers needing to use reader data more effectively. But that’s very much behind the scenes stuff.”

One of the first questions I asked myself in my previous article was: what can these two empowering storytelling tools (personalisation and interactivity) bring to content creation and publishing for children? Personalisation and interactivity have been important parts of CCI since the field of study emerged. The use of these two tools was initially focused on educational value, helping children develop their cognitive and creative skills. Later, with technological advancements and digital devices reaching more homes, their use shifted to utilise a more entertainment-based value. Personalisation and interactivity became an important part of improving enjoyment, engagement, etc. But, putting aside everything these methods already bring, what more could they achieve? The simple answer is that there are numerous possibilities related to emerging technologies.

With emerging technologies, such as the power of 5G, metaverse and voice recognition coming to children’s fingertips through their devices, there are new storytelling experiences on the horizon. How can we utilise these technologies to enhance new models of immersion that are to be seen in this space? This is probably one of the most interesting questions I have asked myself over the course of my research. And, well, the answer is that the possibilities are endless. Emerging technologies bring new ways to tell stories across all levels of publishing, from content creation, through production and to distribution. Let’s take a closer look at them.

The power of 5G

When it comes to networking innovation, 5G is undoubtedly one of the most prevalent emerging technologies that the world is focusing on right now, but how does it impact publishing? 5G provides three major advantages over 4G; more devices can be connected to the same mast, there is lower latency and faster speed connections:

· More devices are able to have mobile network connections without causing problems – for instance, allowing the adding of more Internet of Things or IoT devices (such as cars, vending machines, pacemakers, security cameras, bus stops and anything else you can imagine) to the network. This also means that at large events (such as football games and music concerts), where previously local towers might have been overloaded, more people can be in a smaller area and still get a great connection.

· Enabling more latency sensitive activities – for instance, faster feedback from a remote device (such as drones, bionic hands or cars ) or fast reaction based games.

· Large data activities – for instance, streaming 8k video on the go.

You can just imagine the possibilities and improvements these bring to the amplified publishing world. Low latency, for instance, can improve shared online creation enabling people to edit videos together or jam out creating music with friends while on a bus as it allows people to keep in time with each other. It can enhance the AR space shared by multiple users – low latency means what someone else is doing is instantly synced instead of jarring delays. It can make the remote learning of dexterous activities a smooth reality, such as the teaching of a highly hand dexterous activity (cross stitch, magic tricks, origami, clay/putty, etc) remotely. Low latency makes remote controlling robotics possible for precise hand movements. Large data enables us to stream high quality videos and games anywhere we go. This could mean that if a person was wearing smart glasses and recording everything they were doing and sharing their perspective you could watch through their eyes. More devices (combined with the low latency) allow us to use larger IoT data with higher frequency updates for object tracking. This could mean true AR x-ray vision, enabling you to see where the next bus is through buildings (or anything). More devices mean vehicles are all connected to the network and combined with low latency, which ensures that the position can update with a higher frequency allowing for smooth AR object tracking.

The new horizon of metaverse

The other big subject in the current digital world is the evolving concept of a Metaverse. With multiple big companies already investing in its development, such as Epic Games and Facebook, the new digital reality seems to be emerging fast. But what actually is Metaverse? It is quite a common question nowadays and the answer seems to vary depending on which vision we focus on. For Facebook it means:

“A new phase of interconnected virtual experiences using technologies like virtual and augmented reality. At its heart is the idea that by creating a greater sense of “virtual presence”, interacting online can become much closer to the experience of interacting in person. The metaverse has the potential to help unlock access to new creative, social and economic opportunities. And Europeans will be shaping it right from the start.
No one company will own and operate the metaverse. Like the internet, its key feature will be its openness and interoperability. Bringing this to life will take collaboration and cooperation across companies, developers, creators and policymakers. For Facebook, it will also require continued investment in product and tech talent, as well as growth across the business.”

For Epic Games, it seems to be an interconnected space with its own economy, creating “connected social experiences in Fortnite, Rocket League and Fall Guys, while empowering game developers and creators with Unreal Engine, Epic Online Services and the Epic Games Store”.

Metaverse is often seen as a successor to the internet and, just like 5G, it brings what seems like endless possibilities to the digital world. Regardless of which vision we choose to believe, metaverse will undoubtedly have a great impact on publishing as we know it over the next few years. The question is how will the children’s publishing space adapt to it? Unfortunately, as it has not been developed yet no one knows for sure. However, having an interconnected digital space with shared experiences seems to give a promise of being able to create your own digital identity transferable between content, opening new ways of creativity, personalisation and interactivity.

Voice recognition

Voice User Interface (VUI) can be found all around us, assisting us with everyday tasks. The tech giants, such as Amazon, Google and Apple, brought the most popular Voice Assistants into our lives, using Machine Learning to advance them and help them better understand not only the Natural Language itself but also our dialects and accents. Smaller companies are still coming up with brilliant new ideas for utilising voice recognition in new ways, such as Readmio, a start up company who created a voice-triggered fairytale application. While such companies have invested in the field of voice recognition, it can feel as a well established rather than emerging technology but there is definitely still much room for further development.

During the pandemic, Amazon announced their new Echo Show 10 with motion-capable Alexa Skills, simultaneously investing in new hackathon inventions of next generation Alexa Experiences ‘that go beyond voice’:

“Today we announced the all-new Echo Show 10, completely reimagined and designed to move with you as you interact with Alexa. We’re also excited to announce new motion and sensing APIs, generally available, that enable you to add motion to your skills on Echo Show 10, unlocking customer experiences that weren’t possible before. Learn more about Echo Show 10 and the motion and sensing APIs. You can also register for the new Alexa Skills Challenge: Beyond Voice for a chance to win almost $100K in prizes and work toward building compelling multimodal experiences.”

The new motion and sensing APIs allows developers and content creators to “enable customers to easily see your recipes, follow workout instructions, and more naturally engage with other content”, such as games or other experiences. This allows the user’s movement to interact with the experience created. And while this combination of voice recognition and motion response is still emerging in the digital field, there is already much it could be beneficial for within the children’s space – from motion-enabled skills allowing for easier engagement with experiences, such as noted by Amazon “Trolls and How to Train Your Dragon by Universal Games and Digital Platforms” to more motion based games and interactions supporting more engaged motor skill development.

Can we use such collaborative spaces to create a positive impact on the post pandemic world by bringing friends and families together through entertaining, educational and personalised content? I believe we can. As we can see a lot of the emerging technologies are focused strongly on bringing people together and creating a more digitilised reality, from 5G to the metaverse and many others. The digital part of the publishing space is shifting, creating new ways of content creation, discovery and distribution. It will, undoubtedly, greatly impact all fields, including the children’s publishing space. However, the upcoming technological advancements responsible for it can offer numerous benefits improving the publishing as we know it and giving us new and more immersive ways to create content.