Skip to content
  • Stories
    • Create
    • Connect
    • Capitalize
    • Intelligent Content
  • Events
  • Sign Up
  • Sign In
August 4, 2021

How An Olympic Event Was Teleported Live

author
Adrian Pennington
NTT’s Kirari system in action

A holographic substitute for live sports events has been demonstrated at the Tokyo Olympics. While it’s a rudimentary and arguably hollow experience for now, it has the potential to get spectators located anywhere in the world as close as possible to a life-size representation of the action — live.

The technology has been in development for many years at Japanese national telco NTT but the pandemic has thrown the need for it into the spotlight.

Called Kirari and described as an “Ultra-realistic Communication Technology,” the system has been trialed during rounds of competition badminton at the Olympics.

The project, claimed as a world first, projected matches held at the Olympic’s Musashino Forest Sport Plaza at a venue (Miraikan – National Museum of Emerging Science and Innovation) 22 km away.

It requires the build or utilization of a similar sized spectator arena to that where the actual event is held but could transmit a realistic goggles-free 3D reproduction of a sports match or music performance anywhere in the world.

NTT’s Kirari system in action
NTT’s Kirari system in action

Here’s how it works.

At the venue in the Tokyo, a number of 4K cameras capture video of the match. The feeds are composited together to enable 180-degree panoramic displays as well as ultra-wide, surround image displays.

Unlike conventional chroma keying which relies on a green or blue screen to separate the background from the foreground, NTT has developed a means of real-time image segmentation that enables the extraction of just the moving players and the shuttlecock without using chroma key. This can happen regardless of background.

The extracted video plus additional audio, spatial and environmental information including depth and camera position and orientation, are transmitted as separate streams to the remote venue.

“We have succeeded with robust and accurate subject extraction,” NTT explains, “[using] a deep learning model optimized for the space of a badminton court by adding a pseudo-depth image to the input image.”

“Unlike conventional chroma keying which relies on a green or blue screen to separate the background from the foreground, NTT has developed a means of real-time image segmentation that enables the extraction of just the moving players and the shuttlecock without using chroma key.”

This makes it possible to extract players individually even in a fast-moving sport like badminton where players are separated in the front and back sides of the court.

For the shuttle, which moves both subtly and at very high speed in video, NTT says it can accurately extract the contours of the shuttle (travelling up to 300 km/h) as it deforms into various shapes by identifying its position using a deep learning model developed specifically for badminton. Opposing players and the shuttle are extracted independently, allowing for multi-layered projection at the remote venue.

The streams are synchronized together at the remote site using a proprietary technology called “advanced media streaming and synchronization” (NTT term this Advanced MMT or MPEG Media Transport).

  • NTT’s Kirari system in action
  • NTT’s Kirari system in action
  • NTT’s Kirari system in action

At the remote site an arena has been set-up to mimic the dimensions of the actual venue. This includes arranging spectator seats at a height close to that of the actual competition venue. On the “court,” two half-mirrors reflect projected images.

NTT says, “This allows for multi-layered projection of the two layers of aerial images at different depths, with the players in front on the court in front, and the players at the back across the net on the opposite side of the court, each holographically reproduced.

READ MORE: Bringing an ultra realistic viewing experience to the people of the world – Kirari! (NTT Group)

NTT’s Kirari system in action

In addition, “continuous, connected and natural shuttle movement” is reproduced by dynamically changing the display position of the shuttle image between the two layers of aerial images.

It’s also possible to display live match images in the form of giant signage floating above the court, and to enhance the live viewing experience with projection mapping on the court, according to the telco.

Allied to the images, NTT is using fiber-optic cables to transmit real-time cheering to remote venues. This experiment has had to be scaled down for Tokyo given the lack of fans in stadia but the telco hopes to use “5G or 6G cellular networks to deliver a similar experience without the need for a fixed fiber-optic connection.”

Kirari was originally developed to provide people who cannot get to the venue perhaps because of disability “with a sense of presence and unity”. If it works commercially then there’s also potential for sports rights holders to sell more than the host stadium ticket capacity.

The tech was tested as far back as 2017 when it transmitted Japanese music artists onto a stage at SXSW in Austin, TX over 6000 miles away. The bandwidth required for this was 1 Gbps, the latency was 75 milliseconds, and there was apparently no packet loss.

  • Content Creation
  • Live Event Production
  • Broadcast
  • Intelligent Content
  • Connect
  • Acquisition and Production
  • Distribution and Delivery
  • Media Content
  • Connected TV / OTT I Hybrid / IPTV
  • Telepresence / HD Videoconferencing
  • Sports / Sportscasting

Related Content

olympics immersive

All the Ways the Olympics Are (and Aren’t) Immersive

Olympic stadium in Tokyo: The absence of fans “effectively makes this year’s Olympic Games a made-for-television event while also rendering it more difficult to make for television.” Cr: Greg Martin/IOC
LIVE PRODUCTION

Can Creative Olympics Coverage Distract From Those Empty Seats?

VR Olympics: The Telemundo team can rotate the set to be more efficient with the green screen. Cr: Telemundo
LIVE PRODUCTION

Tour the Telemundo Deportes VR Set for Tokyo 2020 Coverage

Subscribe

for more content like this sent directly to your inbox:

Sign Up
  • Main Pages
  • Homepage
  • Stories
  • Events
  • Companies
  • Products
  • Policy
  • Privacy Policy
  • Terms of Use
  • Code of Conduct
  • Cookie Policy
  • Quick Links
  • Advertising & Thought Leadership
  • NAB Amplify Press
  • FAQs
  • Technical Difficulties
  • Contact
  • Cookie Preferences
  • RSS Feed
The Angle Newsletter

Weekly editorial newsletter covering the latest content, events and more taking place on NAB Amplify.

Subscribe

© 2023 National Association of Broadcasters. All Rights Reserved.