Days after reviewing Meta’s newest VR headset, the Quest 3, last fall, I was hospitalized with high blood pressure. Although the two events were unrelated, the latter changed my life quite a bit. Besides needing to pay even more attention to my eating habits, and going on a lot of new meds, I was told to focus on getting regular cardio exercise. These were familiar reminders: I’ve faced this path before.
When I told a friend I needed to exercise more, she recommended a VR fitness app called Supernatural on the Quest. I laughed because I’d already tried it, but this time I was hearing a testimonial from someone I didn’t even know used VR. She got the Quest 2 as a gift earlier in the year and has been using it ever since. So I joined again too.
I’ve been doing daily VR workouts for months now. The Quest 3 headset is my purpose-based fitness device. Meanwhile, downstairs, my oldest son plays Beat Saber and The Walking Dead on Quest 2. For him, it’s his game console.
The future is now the present. Welcome to VR and AR, circa 2024. While Apple and its forthcoming Vision Pro, arriving in February, represent an exciting potential future for the technology, Meta’s Quest and headsets like it are the already real and sometimes functional present.
But can these devices do even more? Can they transcend being game consoles, fitness devices and experimental toys? Will VR and AR, in a sense, grow up? It’s something I’ve seen slowly emerging throughout the pandemic years and the obsession with the metaverse, but in 2024 we may finally see products and apps that bring virtual augmented reality into the same functional universe as our phones and computers, maybe finally leading to mass adoption like smartphones over a decade ago.
Apple Vision Pro: The device that could change the landscape
I’ve seen the immediate future of AR and VR. Apple’s Vision Pro headset, arriving Feb. 2, melds camera feeds of the real world with incredibly high-fidelity virtual visuals. It’s a technique previously used by Meta and in higher-end headsets like the Varjo XR-3 and XR-4, but Apple has refined the interface with hand and eye tracking that, at least in early demos I’ve had, feels remarkably easy and fluid.
Apple is also changing the sales pitch by replacing the terms “VR” and “AR” with its own phrase: “spatial computing.” Instead of focusing on games or fitness or even social connection to start, the company’s marketing discusses how the Vision Pro can run all sorts of iOS apps and connect with Macs. Apple also emphasized a wearable display that’s good enough for viewing movies and photos in fantastic quality. In my demos, that’s actually what impressed me most. Apple’s going big on supporting 3D movies and immersive videos in Vision Pro right out of the gate.
The Vision Pro headset is Apple’s first major new product since the Apple Watch and is the most anticipated new piece of hardware in the VR/AR space since the original Oculus Rift. Despite its high price ($3,499) and likely limited availability, the Vision Pro could completely redefine virtual and augmented reality devices for the next decade. Or it could be another product blip in a landscape of forgotten headsets. Based on Apple’s history of successful products, the former seems more likely to me.
However, Apple’s starting out from the beginning and with likely missing features that VR veterans might have expected. It’s Apple’s style to launch products with limited features at first and gradually add more over time. It means that the Vision Pro is emphasizing visual quality and working with iOS apps at launch instead of gaming and fitness, which means Apple and Meta are, for the moment, pursuing different paths.
Will app compatibility with mixed reality become a larger trend after Vision Pro? It needs to be. Will Apple’s controller-free design push other manufacturers to think of their own hand-and-eye interfaces, too? It seems likely. Meta already wants to eventually sell its VR headsets without controllers.
Mixed reality at large
Everyday AR glasses, where virtual images and information appear in the lenses alongside the reality you see, aren’t a thing yet. There are many roadblocks: prescription concerns, safety, social acceptance, app compatibility, gesture interfaces, and also just keeping them running for more than an hour on a charge.
In the meantime, VR with mixed reality has become the stand-in. Meta’s Quest headsets, Apple’s Vision Pro, Varjo’s industrial headsets and likely every VR headset going forward will have capabilities to blend virtual things with the real world using depth sensors and improved headset cameras. Sony’s recently announced business-focused mixed reality headset, announced at CES and arriving in 2024, is yet another entry with its own flip-up visor.
The concept is still new, and so far, limited. Meta’s Quest 3 relegates mixed reality to being a gimmick in a few games right now. The Vision Pro, as we’ve seen so far, mostly uses mixed reality to float 2D displays over the real world.
In 2024, the door seems wide open for developers to figure out ways to apply toolkits used today in phone- and tablet-based AR apps like Pokemon Go. While we think of VR at the moment as being “Hey look at me, I’m in another world,” the experiential shift could move quickly to pockets of immersion embedded in real-world environments. The Quest 3 is a device I expect to lead that change.
Expect a new wave of apps
Apple won’t be the only new VR/AR platform in the picture. Samsung and Google and Qualcomm are about to debut their own mixed reality headset in 2024, likely offering another high-end alternative to the Vision Pro with a high-end display, but in a way that opens up development via Google’s software. It sounds like a return to Google’s long-dead Daydream dreams, but it’ll likely mean a whole new set of worlds opens up: Apple’s App Store, and very likely Google Play as well.
For Apple, it starts with familiar 2D iOS apps translated to a mixed reality interface, but the possibilities beyond that could get vast. Apple could add 3D movie support to Photos, for instance, or the Persona avatars to FaceTime. App developers already working with the VisionOS software are going to have to solve for different ideas than on the Quest’s hardware, which could also change the type of apps offered on each device. Games, for instance, won’t be able to use standard VR controllers on the Vision Pro: instead, we’ll have a lot of hand- and eye-tracking-driven apps.
Meta’s CTO, Andrew Bosworth, has already welcomed 2D apps onto the Quest platform, and told me that Google moving Play onto Quest would be “trivial.” It hasn’t happened yet. Google could open up Android into mixed reality as Apple opens iOS, both of these moves possibly opening the floodgates to developers who may not have been tempted by more limited Quest, PlayStation, Vive or SteamVR avenues. VR’s pipeline of new games and apps has slowed down over the past few years. Will Apple, Google and Samsung spark a new movement?
AI enters the mix
After a year entirely defined by AI, it’s no surprise that AI would somehow find its way into VR and AR. Especially for Meta, that’ll be a big part of 2024’s plans. Meta’s Bosworth and CEO Mark Zuckerberg already laid out AI-infused plans for their metaverse dreams at the last Meta Connect developer conference, and both the Quest 3 and Meta’s Ray-Ban glasses have new chipsets with more robust AI processing capabilities.
Wearables like Humane’s Ai Pin and Meta’s new Ray-Ban glasses will both use cameras to fuel AI on-device. Camera-driven AI, which should work in a similar spirit to how Google Lens “sees” the world to interpret the environment, objects and text, would make a ton of sense in camera-studded mixed reality headsets. There could be assistive uses, safety features or just better ways that software could recognize the world and fold augmented reality into it.
The greatest immediate movement in this space could come in smart glasses instead of VR headsets.
What about glasses? Or other wearables?
Qualcomm, and a number of other companies, are actively exploring ways to make glasses even smarter and more phone-connected. In the short term, glasses like Meta’s latest pair of Ray-Bans focus on being camera-enabled, microphone- and speaker-connected phone peripherals. In 2024, expect more layers of AI to help these devices listen and use camera data more intelligently, and maybe act more like wearable assistants.
Glasses with displays already exist from a number of companies including XReal, Lenovo and TCL. Expect more plug-in, monitor-on-your-face solutions in glasses form, especially since more devices are friendlier to USB-C connections and the display tech is improving constantly. I’ve already tried using an AR glasses-based “laptop,” so the idea is possible. Now they need to work better with regular glasses and feel less clunky.
We may even see some AR glasses that aim to do (in smaller forms) what mixed-reality VR headsets are doing. Still, the signs are clear that the progress in that zone has slowed down. Until mixed reality software improves, and phone manufacturers — namely, Apple and Google — find ways to dovetail glasses seamlessly as everyday phone peripherals, AR glasses won’t become a truly everyday thing for anyone.
Would I make my next pair of glasses a set of smart Ray-Bans? I’m already test-driving a prescription pair, and the experience is already sometimes profound. Smarter glasses will figure out the territory as VR headsets improve, and someday they’ll meet in the middle. That someday isn’t this year, and it won’t be the year after. In 2024, all VR and AR need to solve is finding a few more ways to be really useful. My family and I have already found a couple in fitness and gaming. I bet 2024 finds a few more.
There’s also the possibility that wrist accessories and other peripherals finally start entering the mix more for both VR and AR. Meta’s working to develop its own neural input-based wristband for AR glasses, and could be working on its own watch — possibly a device that does fitness tracking to work with its already-popular VR fitness apps like Supernatural. The Apple Watch already works with Supernatural for heart rate tracking, but the company hasn’t announced any watch-to-Vision Pro compatibility yet… although the Apple Watch is already adding hand gestures. Sony’s recently announced mixed reality headset includes a ring with gesture tracking, along with a handheld stylus, and Sony already has a content creator-focused full-body tracking system called Mocopi. Hand tracking may become the norm for headsets, but more wrist and body wearables are bound to emerge soon.
