Exploring Immersive Media: Florida

A diversity of uses and locations

The potential of XR seems almost infinite, deployable in any sector. XR is already widespread in the arts and culture: there are numerous museums and exhibitions featuring immersive works, be they video games, films, rides, etc. Other, more unexpected uses are emerging, at school, at work, in society. Despite barriers to entry, industry players in Florida are taking the risk of innovation.

Florida’s arts and culture scene

Mainstream access to immersive media is often done via art and cultural venues. Indeed, the technologies fit right in and their potential can be fully exploited in relation to works and space.

In Villa Albertine’s interactive map and study “Immersive Spaces”, there are nearly twenty immersive spaces in Miami alone. This shows how dynamic and unique Florida is: “Immersive reality is indeed becoming important, but this is a general statement. Florida is also embracing this trend. I believe there are site-specific opportunities. For example, we have a unique coastal environment with diverse marine life and wetlands. This lends itself to submersed explorative technologies and the creation of underwater experiences”, says Neil Ramsay, Research Associate at Florida International University, a.k.a “The Creative Economist”. Proof of this, for instance, is the success of Hidden Worlds, an immersive experience combining art, technology and ocean conservation while being first and foremost a pop-up restaurant. The first run took place in 2022 at the world-renowned Rudolf Budja Gallery in Miami Beach, the second in 2023 in partnership with The Islands of The Bahamas at Ampersand Studios in Wynwood.

Florida has a booming XR scene in the fields of arts and culture, despite a “slow start [and] tough time when we started 10 years ago”, explains Diliana Alexander, Executive Director of FilmGate Miami. “Now University of Miami has an excellent Department of Interactive Media, University of Florida has strong female immersive artists and academia, Miami Dade has an animation school focused on gaming, and Florida International University is entering the space. Museums are focusing their attention on immersive initiatives – the Frost Science Museum, Superblue, Pamm, the History Museum are now working on a five year digital plan. The local foundations like The Knight Foundation and the Downtown Development Authority (DDA), are supporting new tech initiatives. Art house theaters like O Cinema are also programming immersive projects. We know that Mixed Reality engagement awakens citizens to participate and we value inclusivity and equity in a digital age.  Wonder and expression can be magnified”, she analyzes.

FilmGate Interactive (FGI) is an interactive media festival at the crossroads of visual storytelling, entertainment, and immersive technology. It invites curious and tech-savvy audiences to explore how emerging technology empowers new forms of storytelling. The festival is twofold: on the one hand, there is a monthly local festival of short works; on the other hand, there is the international film festival which leads Art Basel Miami, and which celebrates in 2023 its 10^th edition. “Our interactive fest received close to 500 submissions this year, and we selected 15 out of the submissions and invited 8 more from meeting creators at different festivals and industry events.  We have our biggest regional presence of immersive projects ever – the South, Florida and Miami. It’s very exciting”, is satisfied Alexander.

FGI aims to bring together art, science and technology: “We do not only program VR but interactive installations, dome experiences, immersive sound and this year, we have a robust academic and business focus due to our partnership with University of Miami, Snap, Microsoft and the DDA. The festival will feature talks, workshops, experiential installations, parties, music, and more, closing with our Sunset Cruise on the bay” This year’s festival theme is “REGENERATE X”: “In an era where the echoes of past upheavals reverberate loudly, “REGENERATE X” emerges as a call for collective reinvention — whether through ecological rejuvenation, audacious reimagining of aesthetic paradigms in arts and culture, or resilience and resurgence post COVID. This year, Filmgate Interactive reinvents itself for its 10th anniversary — FILMGATE X. This festival seeks not only to mirror the world’s recuperative capabilities but to posit art and technology as vital agents of this renewal. Drawing inspiration from the regenerative capacities of our environment and society, we delve into narratives that explore the possibility of a brighter, more inclusive future. Further, with the evolution of generative AI, we’re reminded that regeneration isn’t always a return to a former state: regeneration can radically redefine.”

In addition to FilmGate Interactive, ScreenDance and Superblue are two important cultural institutions in the XR scene. ScreenDance Miami is an international dance-on-film festival featuring choreographers and directors who are working with emerging concepts involving movement, dance on film, and dance on camera. This is a festival which focuses on immersive contemporary dance. Finally, Superblue Miami is a 50,000-square-foot venue opened in May 2021, entirely dedicated to the production and exposition of large-scale experiential mediums. Similar to a gallery, Superblue compensates the artists for their long-term installations with a portion of the ticket sales while their pieces are on display.

With museums focusing their attention on immersive initiatives, FGI, ScreenDance and Superblue are the major Floridian cultural institutions entirely dedicated to XR. However, there is still work to be done on the programming of other festivals: “We are the only Festival that focuses exclusively on immersive and unfortunately, outside of ScreenDance, almost no one programs immersive. We aim to partner and change that”, Alexander regrets.

Innovative applications

Outside of arts & culture, XR presents many innovative uses.

In healthcare, for instance, XR is a tool for both training and care. The University of Central Florida College of Nursing implemented AR and VR tools for nurses. Here, XR gives health professionals a virtual look inside the human body and helps visualize scenes of care. “There are scenarios where the student can walk inside the body and see what’s happening behind the scenes with that pathophysiology, anatomy and physiology, so they can really get that understanding of what is happening with the patient. This can help with interventions.”, said Mindi Anderson, Interim Associate Dean for Simulation and Immersive Learning, per HealthTech Magazine. This way, “Students can see what happens during a stroke or heart attack.” UCF uses both Microsoft HoloLens and Meta Quest headsets, as well as a variety of prepackaged simulations. “The school chooses cases in which real-life simulations would be challenging, including maternity complications such as postpartum hemorrhage. In maternal care, an image can be overlayed on a manikin’s abdomen”, she completes.

More concrete uses of XR exist beyond the training stages.

St Joseph’s Children’s Hospital in Tampa is using the Surgical Theater Precision VR technology in its pediatric neuro and cardiac surgery programs to plan and strategize the best route for surgery. The program uses a F-16 flight simulator technology with traditional 2D medical images (CT, MRI, etc.) to create a 360-degree virtual model of a patient’s own anatomy. St Joseph’s president Sarah Naumowich explained: “The immersive technology provides our pediatric heart and brain surgeons with a never-before view of the anatomy and pathology of their patients’ complex conditions. […] Being able to see what the doctor sees provides our patients and their families with a better understanding of the diagnosis, which helps alleviate some of the stress and anxiety that comes with preparing for brain or heart surgery”, per PR Newswire.

VR may also provide distraction from pain and anxiety, as well as allow patients to explore different environments and scenarios they may not otherwise be able to experience in real life. While it may not treat the underlying cause of a patient’s symptoms, it shows promising potential as a way to improve quality of life for patients in palliative care, per Forbes Health. “With the ability to personalize a patient’s VR experience, we can do more than bring the world to our patients — we are able to give our patients those ‘what matters most’ moments without additional physical stress”, said Kristie Henry-Roling, an employee engagement manager involved in the VR program at St. Francis Reflections Lifestage Care in Melbourne, Florida. The same idea is being used at Nicklaus Children’s Hospital in Miami with the BEAR program.

In sports, XR can be used to enhance athlete’s performance. For instance, the University of South Florida men’s and women’s tennis teams have partnered with sports virtual reality company Sense Arena to use VR tools to supplement IRL training. The system uses the Meta Quest 2 headset and a properly weighted and balanced racket-type device to give players visualization tools aimed at improving the mental aspects of the game — like timing, anticipation and quick decision-making — without requiring a partner, ball or court. “It’s the perfect tool to use on recovery days when we want to exercise our mind and train our brain, really focusing on the mental aspects of tennis”, Cristina Moros, head of the USF women’s team, said.

In security and defense too, XR conveys a definite interest. Tequesta and Fort Myers police officers use VR to practice their skills, with programs simulating a wide range of situations, from everyday incidents to high-stake situations. “This Apex Officer system allows the officers to really interact with somebody in crisis or different scenarios and different environments so they really get to develop de-escalation skills, de-escalation techniques, having a conversation with someone, all that we couldn’t do prior. […] This is about as real as you can get without being there”, said Fort Myers Police Department Lt. Jason Paid.

The Florida based contractor Engineering & Computer Simulations (ECS), which builds digital training and other tech solutions for the U.S. military, has recently received a grant from the federal Small Business Innovation Research program (SBIR Phase II) to pilot a VR training program for U.S. Army medics.

With all this in mind, it is worth noting that Florida hosts the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), the world’s largest modeling, simulation and training event, organized annually by the National Training and Simulation Association.

A key industrial sector

It would be wrong to reduce Florida to its service economy and tourism. The state offers an emerging tech industry with one company, MagicLeap, at the cutting edge of technology, and a fertile ground for innovation, just waiting to be cultivated: “Florida is great at marketing, but traditionally we have not had the quantity of successful tech startups, with the exceptions of MagicLeap. I honestly think that with increased investment in immersive by funding and government agencies, that is changing”, comments Alexander.

It is worth taking a closer look at MagicLeap, which, precisely, makes Florida a noteworthy industrial sector. Since its creation by Rony Abovitz in 2010, the Florida-based company has raised no less than $4 billion, with minority investors including Google, Alibaba, Qualcomm, AT&T, and Axel Springer. The vision that forms the basis of MagicLeap is the idea that the human brain takes in continued sensory inputs in order to dynamically update a model of the world which we partially inherit and partially create – this is Sensoryfield Computing, which ultimately results in Neurologically True Reality.

Of this ideal that continues to seduce investors, MagicLeap spent years of R&D, including the takeover of the Franco-Belgian startup Mimesys, until unveiling its AR goggles at the end of 2017 – the Magic Leap One. At this stage, Abovitz always seemed to have a consumer focus for his company. But the product was not as accomplished as had been hoped, and the critics were numerous. Despite a lackluster response from the public, investors renewed their confidence in MagicLeap’s innovative approach and ability to design hardware. The results nevertheless led to Abovitz’s departure. The company then gradually changed its strategy, switching from a B2C to a B2B approach. The company has since been able to release the second-generation of its technology in 2022, the MagicLeap 2, this time for developers and enterprises, still failing to win unanimous support, but with much better results. BNP Paribas Real Estate, for instance, uses MagicLeap’s solutions since 2019, a partnership renewed in 2022. That being said, high production costs and an underwhelming customer base make MagicLeap a fragile company with a fragile business model. For the time being, the company can count on Saudi Arabia’s Public Investment Fund acquiring a majority stake over the course of 2022 to sustain its loss.

MagicLeap’s journey since 2010 is all the more interesting to understand the stakes of XR development in general, which require companies and investors to take the risk of innovation. Neil Ramsay is as optimistic as Diliana Alexander: “The technology load and expense are currently prohibitive for many potential contributors. However, technology is a rapidly advancing field, and costs tend to decrease as it becomes more commonly used in daily applications.” And to the critics of XR being a niche industry: “I don’t think that a media distribution outlet addition, which immersive reality technologies are, can be considered niche. It is soon to be as commonplace as television is today, I suppose”, he adds.

The role of universities

Scientific research has played a decisive role in the development of XR technologies.

As evoked by Diliana Alexander, Florida’s universities now jumped the bandwagon of XR. “Florida’s universities are really leaders in the Florida immersive space. We work with many students from University of Miami (UM), Florida State University (FSU), and Miami Dade College’s Animation & Gaming International Complex (MAGIC). […] UM specifically is focused on immersive storytelling, gaming, architecture, music and healthcare. MAGIC focuses on VR gaming and animation. FSU has very strong programs in digital heritage and 3D modelling. One of our favorite artists is a resident at Florida International University (FIU) and is creating immersive and emotional experiences with robots”, she analyzes.

Speaking of FIU, it is worth mentioning the Ratcliffe Art + Design Incubator (RA+DI) in which Neil Ramsay operates. RA+DI is a creative entrepreneurial laboratory based at FIU’s Biscayne Bay Campus. RA+DI’s mission is to bridge the gap between creative practitioners and entrepreneurial success while promoting innovation within the FIU community. “We are an interdisciplinary unit, possibly even trans-disciplinary when describing the mentors. The fellows often bring two of the three (art, tech, science) into their own projects. Moreover, within our open-space designed facility, we witness another unique blending with each new cohort”, he explains. For Ramsay, “There’s a significant role for academic research. Immersive realities present unique challenges and opportunities. These include studying human psychological and physiological responses to immersive reality technologies. Additionally, it plays a crucial role in modeling experiences and forecasting for the development of real-world applications.”

Last, UCF College of Nursing has been awarded the Healthcare Simulation Standards Endorsement from the International Nursing Association for Clinical Simulation and Learning (INACSL) for its Nursing’s Simulation Labs. It also received a $5.5M funding from the Helene Fuld Health Trust to support the creation of a state-of-the-art simulation center in their new building. And it conducts a $5 million project, the VERA – Virtual Experience Research Accelerator –, to improve XR technologies for the general population and make them more inclusive to groups such as older adults or people with disabilities.