Working It – IU benefits from 360-Degrees of Immersive Technology

BLOOMINGTON, Ind. – Alyssa Rosati and Nathanael Tavares are working it.

Wondrous things are happening on the five monitors in front of them while, a few yards away, on Simon Skjodt Assembly Hall's Branch McCracken Court, Indiana and Iowa stage a Big Ten basketball thriller that will have a heartbreaking Hoosier conclusion.

Fingers fly on keyboards. A cursor darts across a monitor. Deft computer mouse manipulation combines with head-spinning data crunching to make 360 degrees worth of video magic.

Seeing is believing.

The result -- Tavares makes an Iowa player's missing leg reappear to complete Rosati's every-angle game highlight video.

Soon a clip will flash through the Internet -- and on Assembly Hall's bigger-than-life video board -- showcasing the full power of Intel Sports' Intel True View immersive experience technology.

The goal -- you don't just watch sports, you become part of it.

Indiana -- thanks to ultra-successful alum Mark Cuban and the Mark Cuban Center for Sports Media and Technology -- is on the cutting edge of that, and so much more.

"That was the goal," Andrew Rosner says. He's IU's assistant athletic director for Creative Services and the man in charge of ensuring it all works. "It's to put the greatest technology here at Indiana and allow students to gain real-world experience to match up with what they're learning in the classroom."

Only IU offers this kind of learning. No other university in the country has this technology.

"It's not only a tremendous opportunity for our students, who are getting great jobs right out of school," Senior Associate Athlete Director Jeremy Gray says, "but it adds to the stadium game experience.

"An Indiana men's or women's game has the same production value of an NBA game or the finals. That's a great benefit. It sets us apart."

There are other benefits, and Rosati and Tavares, unseen by the thousands of Assembly Hall fans trying to will the Hoosiers to victory, are taking full in-game advantage. They are among the few people on the planet to have mastered the technology.

There's plenty to master. Images from 28 Assembly Hall high-resolution cameras are blended together by data algorithms to form the kind of computer-generated video that, just a few years ago, was science fiction.

"It's too complicated for my B-minus-in-science brain to properly articulate," Gray says, "but they stitch together each of the camera angles into a full 360-degree view. It's a computer regeneration of the camera images. It's not a true picture."

He pauses.

"I'm sure I mangled a key detail and a computer science major would laugh at what I just said, but you get the gist of it."

****

The Indiana gist starts in a windowless room in the bowels of Assembly Hall. Monitors are everywhere in a set up that looks complex enough to coordinate a mission to Mars.

Instead, the immediate mission is to give that Iowa basketball player a leg.

Tavares works his mouse to make it so.

It's not a real leg, of course, but the digitized image of one to fully and accurately depict, in 360-degree glory, a basketball highlight.

As the system engineer, Tavares monitors the system, handles post-production flows and "cleans things up," which, in this case, means recreating that leg. Before games, he makes sure the system is functioning correctly.

As the associate producer, Rosati builds game video clips. Building comes from an athletic background. She's a former IU soccer and softball player who once made Sports Illustrated's Faces in the Crowd by earning a martial arts black belt at age 6.

They handle everything during a game, with Intel providing two to three off-site support people in case something goes wrong.

It rarely does.

"Most of the time," Tavares says, "we don't need them. The system is pretty stable."

Completed clips -- which can show every angle of a play except from below -- are displayed on the Assembly Hall video board. They also are posted on social media and sent to TV broadcast partners BTN, CBS and ESPN.

"Last year we averaged about 12 clips a game," Rosner says. "Every year we get better. From the beginning of the season to now we've seen a marked improvement. When we first started it was taking 20 minutes to get a clip out. Now it's five minutes from when it happens to when we (post) it."

*****

It all starts with eyes up high. State-of-the-art cameras, installed in small circular holes at the bottom of the Assembly Hall balconies, plus on the sides above and below the championship banners, capture everything.

"It's extraordinarily elaborate and delicate in how they install those," Gray says.

One of the biggest challenges, Tavares adds, is "Making sure from the beginning that all 28 of our cameras are set up and focused correctly.
"Then you have to make sure all of the computers are up and running. After that, we know we're getting quality images that are being captured."

Capturing good images is technical. Turning them into eye-catching videos is art.

"The next hurdle is the post-production," Tavares says. "It depends on how the system interprets the images. The clip it produces depends on the environmental situations."

In other words, how good are the contrasts between court, uniform, skin, hair, ball and background. It takes a ton of computer power to figure that out.

Tavares says making 360-degree videos from basketball games are difficult because, "When you get big groups of people close together, the system has difficulty interpreting things. You might have a yellow shoe against a yellow court, and (the shoe disappears). Sometimes you have to bring back an arm."

Or, in the case of the IU-Iowa game, a leg.

Assembly Hall's steep-sided design adds to the difficulty.

"We are constrained to the architecture of the building," Rosner says. "The sides are so steep that the camera angle is steep. Newer arenas are flatter and that enables the system to perform better. Nathanael has to account for the steepness."

At IU, the system is only set up at Assembly Hall, but it's also being used at NBA arenas, NFL stadiums, pro soccer stadiums and more.

Some of the world's most popular sports and teams use this technology -- Major League Baseball, the NCAA basketball tourney, the NFL, the NBA and professional soccer's LaLiga Soccer, Arsenal FC and Liverpool FC.

Still, in the world of college sports, Indiana stands alone, and if that won't last forever, nobody in this mission control room cares.

Halftime has arrived and no one rests. The Hoosiers, seeking to build off the upset victory at Michigan State a few days earlier, trail No. 20 Iowa 46-36. Rosati and Tavares work to capture the drama and stream it for the world to see.

What they don't know is that the real Cream 'n Crimson drama is about to begin.

****

The idea to bring this system to IU came from a June of 2015 meeting with Cuban, who was donating $5 million to start the Cuban Center. He mentioned several emerging sports technologies, including virtual reality as well as the 360-degree technology that became True View.

"He talked about these as being the next, best, greatest things in sports technology," Rosner says. "I had not heard about it before."

By February of 2017, the system was installed at Assembly Hall.

"It's been a fixture at IU basketball games ever since," Gray says.

Plenty of training was necessary, and expertise initially came from a start-up company called FreeD. Twelve IU students participated. They spent four to six hours per day training.

"At the time only about three people in the world knew how to use it," Gray says. "(The company) flew people in to Bloomington, and did a six-to-eight week crash course with our students. They were immersed in the technology, with a rigorous test at the end."

Adds Tavares: "The amount of training and hands-on experience was very intricate and in-depth. It went from building the clips to publishing them."

The technology was first used at the 2012 London Olympics. Back then it took five minutes from the time an event was run until a video was posted. Now it's less than a minute, Gray says.

"There is the belief that, in the coming years, it will be instantaneous."

When that happens, the system could be used in a variety of ways, including enhanced instant replays to aid officiating.

"You will know if a runner crossed the goal line in football, or if it was a 2-point or 3-point field goal in basketball, or whether or not the runner was out at first base in baseball," Gray says.

Rosner says baseball is an ideal candidate.

"It's not to replace umpires," he says, "but help them on the bang-bang calls at first base."

Can coaches use the technology for instruction in the same way they currently use regular videos?

"Not now," Gray says. "You'd have to have a technician at every practice to cut up a ton of clips. You couldn't do it for games because of the rendering time. You could only do about 10 clips a game. The volume is not what a coach -- who wants to see every play -- would want."

As the technology and technique advance, all things are possible.

"What about streaming an entire game into homes so you could watch the competition as if you were playing the game?" Gray asks. "Take it several generations forward and you could be totally immersed in the game."

****

In the end for Indiana University, it's about providing a learning experience like no other; it's training students to land well-paying, interesting jobs. Intel already has hired multiple students -- including Rosati -- and sending them all over the world.

"If you want to get into sports media, you can't be good at just one thing," Gray says. "You have to be versatile. Students are much more employable when they can do multiple things. This provides them with another option."

Rosati, for instance, traveled multiple times to Barcelona for Intel last year to work a major international soccer match between FC Barcelona and Real Madrid. Tavares got to work, remotely, last summer's NBA finals between the Cleveland Cavaliers and the Golden State Warriors.

Students also have worked high-profile events such as the Super Bowl and the U.S. (tennis) Open.

"It's been a remarkable thing for our students, and the technology itself," Gray says.

Tavares, a computer science major, has traveled to cities such as Washington D.C., Boston, Denver and Los Angeles (Intel has a big studio there). He's also joined forces with an IU professor to start a virtual reality company.

Writing software and technology development top his to-do career list.

It's what he calls, "a cool idea."

The next challenge for IU, Rosner says, is finding new students to train.

"That's been the greatest thing about this partnership with Intel, that they've allowed us to utilize students and train them on this great technology. That sets them apart job-skills wise when they enter the market."

That skill comes to a dramatic conclusion against Iowa. The Hoosiers launch a fierce second-half comeback, only to see Hawkeye guard Jordan Bohannon hit a couple of back-breaking, are-you-kidding-me three-pointers.

Cream 'n Crimson victory hopes are dashed, 77-72, but with Rosati and Tavares working it, technology wins.

In the big picture, so does Indiana.

Working It – IU benefits from 360-Degrees of Immersive Technology

Related Videos