Deep in the second quarter of the New York Giants’ win over the Chicago Bears on November 20, Giants quarterback Eli Manning slunga short pass to running back Rashard Jennings, who was forced out of bounds at the 28 after a five-yard gain. Nothing about the play, which lasted all of six seconds, stood out as particularly noteworthy. Except for one thing: Safety Deon Bush and linebacker Danny Trevathan shoved Jennings out of bounds right in front of NextVRs bug-eyed sideline camera. Watching the play on a VR headset made you instinctively want to get out of the way. The “you-are-there” sensation was strong.
NextVR is all but ready to flip the switch on live football.
OK, time out. You’re rolling your eyes, with good reason. Video on aVR headset is usually pretty disappointing. The novelty ofwatching something from a cool virtual vantage pointerodes quickly. There’s a screen door effect when seeing pixels on a magnified screen, shatteringthe illusion of being there. The content typically lacks narrative, offering nothing to direct your attention or provide urgency or drama.
But NextVR’s remarkable work with the NFL and the NBA provides a virtual experience that is far more engaging and dramatic. The slick highlights packages they produce for the NFL are released after the games, but NextVR is uniquely equipped to do live VR video. The company uses the same crew, gear, and processes to broadcast one live NBA game each week. They useeight camera rigs, hopbetween camera angles, and include onscreen graphics and play-by-play announcers.
We aren’t talking 360 video, either. There’s no confusion about where to look, as NextVRs rigs use a 180-degree field of view to record only the action in front of them. The hardware is more 3-D camera than VR camera, with processing tricks that add depth and boost the sense ofrealism. Each double-barrel RED rig captures 6K video for each eye. That’s tens of millions more pixels than any headset can display now, but NextVR wants to future-proof its content.
The result represents the dawn of a new kind of VR video, an experience that truly combines the best parts of watching a game on TV and being in the stadium.
The on-field cameras each use a pair of super wide-angle 8mm lenses to send two fisheye views of the action to the production truck, where the producers watch it on the monitors. You’ll find several Gear VR headsets in the truck, but no one is wearing them during a game. They’re there just so the crew and on-air talent can give the NFL VR experience a spin before blasting it out to the masses later on.With the NBA, it just goes out live,” Earl says.
The barrier to live NFL coverage goes beyond inking deals. The execution is already as polished as anything you’ll see on TV, but NextVR is still feeling its way through some details of shooting sports. Something as simple as panning is a no-no, because it may make peoplesick. Zooming is also off-limits, as it chops off the field of view. Both limitations make it tough to capture fast breaks in the NBA and long bombs in the NFL. When switching between cameras, NextVR producers favor a slow fade between angles, often when the action slows down.
The company has optimized its NFL field coverage by mountingcameras under each crossbar, placing manned cameras on the sideline of each end zone, and having four camera operatorsroam the sidelines. The setup favors action in the red zone overplays between the 30s, where each team has its bench. “The hard part about doing football is that with the 8mm cameras, you need the action to come to you,” Earl says. It’s a bit easier with basketball, where most of the action happens in front of the NextVR camera under each hoop. Anything that happens, we have a great angle of it,” he says.
That said, NextVR is all but ready to flip the switch on live football. The crew essentially produces complete games already, they just don’t broadcast them.The reason we do the NFL highlights the way we do it is because we want it to be exactly the same, says VP of content Danny Keens. We dont want there to be any loss of quality, any loss of resolution, any of that stuff. Itd be easier to just piece that show together and record it bit by bit. But we go start to finish and do live graphics and replays in real-time.
3D + VR = OMFG
VR is only now moving into the mainstream, but NextVR has been at it for two years. What started as a simple one-camera setup at midcourt during an exhibition game evolved into the current multi-camera setup with all the fixins. Earlier thisyear, NextVR announced it would carrylive NBA gameseach Tuesday as part of NBA League Pass. It marked a huge step forward for virtual reality.
For the first time ever, we announced a production schedule, says company CEO Dave Cole. We had more than 500 hours of live VR production under our belt before the NBA deal, but they were all one-off productions. Thats not the type of thing you can get viewers to schedule. I liken it to going to Best Buy and buying a television and the salesperson saying, ‘Well, theres probably going to be a broadcaster for this device someday.
NextVR’s secret sauce is its3-D effects, which date toits origin in 2009 developing 3-D television transmission tech. The company’s video-compression technology shrinks files by removing redundancies between each rig’sleft camera and right camera. Because that process involves detecting the edges of objects in a scene, the same technology can create wireframe replicas of everything the camerasshoot. When the video is rendered in a VR headset, it overlays stereoscopic video on top of those wireframes, creating the illusion of volume. It’s a mix of video and video-game tech.
It sends a hugely impactful message to your brain that youre actually in this environment, Cole says. Right now, the mesh, the number of vertices in the wireframe is quite low. In the next generation camera, which is rolling out in the middle of the season for the NBA, we are quadrupling the resolution of that mesh. That sense of presence is what were amping up.
Hacking Your Memory
NextVR outpaces the VR competition with the quality of its tech and scale of its deals, but it’s already eager to refine the experience. We will have done our job right when people cant remember whether they actually went to the game or watched it on NextVR, Cole says. I think thats an achievable goal.
Crazy as it sounds, that actually could be an achievable goal. At least in part. My answer to whether or not that could be possible is … sort of, says Julia Shaw, a memory expert, criminal psychologist, and author ofThe Memory Illusion. Shaw hasn’t studiedVR’s effect on memory, but she has successfully implanted false memories into the brains of test subjects.Because our memories are unreliable, convincing VR experiences could fool our brains. But only to an extent.
“Reality is multi-sensory,” she says. “When youre looking at something, no matter how high-def it is, if you dont have things like proprioception, your sense of space, you dont have smells, you dont have taste, you dont have temperature. These are things that we generally rely upon as markers to let us know weve experienced something instead of just imagined it.
So if Cole is serious about taking NextVR to that level, he’ll have to figure out how to infuse the experience with a lot more sensory input. Things like smart thermostats synced to the action and stadium-smell simulators.
VRs Biggest Challenges
NextVR has some more important technical hurdles to clear first, but they likely won’t be barriers for long. You can only watch its programming on the Samsung Gear VR, but Cole hints you’ll see NextVR on Google Daydream and PlayStation VR soon. And NextVR is talking with other sports leagues and entertainment companies about getting more content in the pipeline. Cole says 4k-capable phones will make the viewing experience that much better within the next year, and 5G connectivity will make accessing content easier on mobile devices.
All of this begs the obvious question: Willviewers embraceVR as a first screen option for sports? Already, watching a game on NextVR is a better eyeball experience than watching it on TV. But here’s the thing: VR is a solitary pursuit, one that requires clamping a headset on. Watching sports is a social endeavor, one that revolves as much around the camaraderie of the experience as it does the game itself.
So there are compelling reasons to strap on a VR headset to watch a game, but it’ll likely only happen if you’re home alone. Duncan Stewart, director of technology research for Deloitte Canada, says the solitary viewing experience is just one mainstream adoption barrier. According to his research, a few trends are shaking out in these pioneer days of VR: The medium still appeals primarily to males, hard-core gamers, and those with deep pockets. Deloitte’s global surveys show that more than 95 percent of people dont own a VR device and arent interested in buying one.
“There are indeed some people who are interested enough in the VR perspective to watch sports and wear a device on their head that blocks out their wife or husband, kids, parents, friends, pets and smartphone for hours at a time,” Stewart says. “But not many.”
There are other impediments, not the least of which is VR’sillusion of immersion is shattered every time you need to use the bathroom or grab a snack. And then there’s the simple fact that watching a game isn’t as simple as flipping on the TV, popping open a beer, and flopping down on the couch. You’ve gotta hook up the gear, launch the app, and find the content. NextVR’s incredible videos may represent the future of VR, but sports fans may determine the future of NextVR.