Borderless and mixed reality projects are pushing the boundaries of entertainment.

Thanks to advances in digital technology, entertainment is now delivering richer, deeper and more immersive experiences for audiences. Content and experiences now deftly blend reality with the virtual, offering multiple entry points and deeper experiential layers. Today’s empowered audience expects freedom: to explore, experiment and most importantly, participate.

This trend towards much deeper audience engagement, like so many others, is seeing some acceleration. The COVID-19 pandemic has presented enormous challenges for production, and live performances are still on hiatus, but the constraints of lockdown are proving to be an engine, spurring innovation.

For instance, by rights, we should be deep into festival season right now, but the tents remain unpitched. Undeterred, the team behind the legendary Shangri-La space at Glastonbury has created a virtual alternative. Called Lost Horizon, it is the world’s largest arts and music festival in virtual and mixed reality.

WEB_gas_tower_screenshot_014a
WEB_gas_tower_screenshot_013a
Lost Horizon
WEB_gas_tower_screenshot_011a

Offering an interactive, multi-stage experience with a stellar line-up of DJs, underground acts and visual artists, Lost Horizon was streamed live via Twitch and Beatport and across social media on 3 & 4 July. Created in partnership with Sansar, Wookey Technologies’ photorealistic platform for events, the festival took place in a 3D rendered game-like landscape complete with “computer-generated avatars and green screen ‘hologram’ performances.”  Much like any other festival, attendees in virtual reality (VR) were able to wander from stage to stage and chat and interact with others, with one key difference: at Lost Horizon, they were avatars.

There is growing appetite for virtual events like this that are redefining the very concept of ‘live’ entertainment. Virtual shows have pulled in substantial audiences, most notably the Travis Scott Astronomical performance in Fortnite, which clocked up 28 million unique viewers across five shows in April.

WEB_SansarClient-2020-06-04-15-42-43
Lost Horizon

More than mere pandemic stopgap solutions, virtual gigs and festivals have long-term potential. Apart from the huge, borderless audiences, they also provide valuable opportunities for both inclusivity and transcendence. As Lost Horizon’s creative director Kaye Dunnings explained at the (also virtual) CogX festival in London recently, “You can transcend anything through an avatar. You can create it how you want and customize it yourself. People that maybe are limited physically can walk anywhere, jump and bounce and teleport around the space.”

WEB_SansarClient-2020-06-04-15-45-50
Lost Horizon

This idea of transcendence, leaving reality for exciting new worlds, is something that is increasingly key to storytelling too. Advances in technology are allowing viewers to step inside narratives, try on new personas and engage directly with their favorite characters.

Earlier in 2020, Sky TV celebrated the UK launch of a second series of their hit crime procedural Bulletproof with a complementary immersive digital experience, Bulletproof Interrogation. Players join two leading characters from the show (cops Bishop and Pike, played by Ashley Walters and Noel Clarke) in a tense sequence where they must help interrogate a serial killer bent on committing three murders. Created in partnership with Charisma.AI, the game deploys natural language processing and AI to allow fans to conduct a back and forth discussion with the suspect, where everything they say might influence the outcome.

WEB_IMG_1090
WEB_StudioPeakyBlinders02
WEB_Small+Heath+layout+v14b+NO+TEXT
Maze Theory's digital recreation of Peaky Blinders

Bulletproof the Interrogation borrows heavily from gaming, and content creators are increasingly looking to fuse the thrill of the gaming experience into traditional entertainment mediums. UK startup Maze Theory is currently developing an interactive gamescape based on the hit BBC TV drama Peaky Blinders. Players will be able to enter a meticulous recreation of the world of the Shelby crime syndicate, infiltrating their circle, earning their trust and helping them defeat a rival gang. Peaky Blinders: The Kings Ransom promises ‘humanlike’ interactions, made possible through advanced AI technology which will allow characters to respond in real-time to players’ words, gestures and body language.

Game mechanics are also crossing over into the world of theater. Punchdrunk, the immersive and interactive theater company behind hits like Sleep No More and The Drowned Man recently announced that “the future of interactive audience experience will be at the cross section of gaming and theatre.” Earlier this month, the company revealed a new partnership with Niantic, creators of the augmented reality Pokemon Go game. The venture promises to break new ground, taking the theatrical experience outside the walls of the theater and into real world.

WEB_Grayson3
Grayson's Art Club

These concepts reflect a desire on the part of audiences to see themselves at the heart of the action, to actively engage rather than passively consume. Shows that can authentically offer viewer participation will reap the rewards, according to Neil Crombie of TV production company Swan Films. Crombie is executive producer of Grayson’s Art Club, an arts show conceived for lockdown for the British broadcaster Channel 4. The show ditched the traditional format of expert art critics dispensing wisdom and instead created a collective, meaningful experience for the nation.

In a run of six shows, viewers submitted artworks on a theme proposed by contemporary artist and presenter Grayson Perry, who conducted Zoom chats with the artists to hear the stories behind their work. Crombie, speaking at a recent CogX festival panel, said that the show was “radically empowering for the audience” who embraced the participation element, and that the concept “has energized and revitalized what the whole proposition of an arts programme is.”

WEB_Beatie-Wolfe---2017-Raw-Space---Gimme-Some-Love---Live-AR-Capture-(Design-IO)
WEB_Beatie-Wolfe---2017-Raw-Space---Design-IO-Tech
Beatie Wolfe's Raw Space project
WEB_Beatie-Wolfe---2017-Raw-Space---As-You---Design-IO-Capture

The continued blur of physical and digital, which the pandemic has only served to accelerate, is provoking debate over which might take precedence. The work of artist, musician and innovator Beatie Wolfe suggests that it’s the dance between the two that is most potent.

As a recording artist, Wolfe has observed the increasing intangibility of recorded music. Her work seeks to restore meaning, pioneering formats that restore stories, ceremony and tangibility to the listening experience and reinject physicality, often by leveraging technology itself.

WEB_Beatie-Wolfe---2017-Raw-Space---VR-test-with-Design-IO
Beatie Wolfe
WEB_Beatie-Wolfe's-V&A-exhibition-for-LDF---8ight's-PPT---by-Dexter-Robinson

Wolfe explained to Wunderman Thompson Intelligence, “From the time I started writing songs (age 8) and discovered my parents’ record collection, I saw records as musical books, with the artwork providing the perfect backdrop for the story, and I loved opening them up and entering into the world of the album. There was also a ritual to the occasion. I started imagining what my album could look like, what it could feel like, what worlds I could create. When it was time for my first album to be released, it was a very different era with the digital replacing the physical. So I thought about how to connect the two and that’s what my work became centered around: reimagining the vinyl experience but for today.”

Wolfe’s innovations range from a 3D ‘theatre’ for the palm of your hand, a wearable album in the form of a jacket and the world’s first live 360 augmented reality (AR) stream for her album Raw Space—which was the first album to be beamed into space.

WEB_beatieWolf_test_wide_v009-1152
WEB_beatieWolf_test_v009-1682
Beatie Wolfe's From Green to Red
WEB_beatieWolf_test_v009-2512

Her recent work ‘From Green to Red’ is an environmental protest piece that will premiere at the London Biennale in June 2021. Harnessing 800,000 years of data from NASA, the visual piece tracks human impact on the planet through time by visualizing CO2 concentrations. The full installation seamlessly fuses the digital and the physical. As visitors approach, the timeline and music will change in response, the intent to give each individual an understanding of their own impact and a sense of agency.

The fusion of digital and physical is helping to amplifying and elevate entertainment experiences, eliminating boundaries and expanding possibilities. Looking to the future, technologies from mixed reality to gestural, haptic and neural interfaces promise ever more intuitive, responsive and personalized experiences, with the viewer at the heart of the action.

Main image courtesy of Lost Horizons