Fukuoka SoftBank HAWKS create a new way of watching baseball with Azure AI Services

An AI avatar of a retired baseball star generates massive interest among corporate lounge users

Fukuoka SoftBank HAWKS is a professional baseball team based at the MIZUHO PayPay Dome in FUKUOKA, Japan. In 2017, the HAWKS set a Pacific League record for the fastest championship win and looked set to break the record in 2024. While it didn’t manage to achieve the feat, it still won the league comfortably.

It also achieved a remarkable accomplishment away from the field in 2024. In July, it concluded a naming rights agreement with Microsoft Japan for its stadium’s executive lounges, opening Microsoft Premium Suites. A stellar feature of these suites is an AI avatar of Ryota Igarashi, a commentator, YouTuber, and former HAWKS star. MIZUHO PayPay Dome has about 119 VIP rooms spread over the 4th, 5th, and 6th floors. The 4th floor features 16 Microsoft Premium Suites requiring an annual contract, one of which features the AI avatar.

July 2024 marked the official launch of the AI avatar. Igarashi’s avatar appears on a large display and communicates in real time, answering questions from users, giving player information, commenting on the current game, and cheering for the SoftBank HAWKS. The display also shows information about the starting lineup and multiple-choice quizzes about the HAWKS. Microsoft Premium Suite users can interact with the AI avatar while enjoying the action on the field.

A firm conviction of consumer interest in AI avatars

Takuya Nishikawa, Deputy Head of Sales at Fukuoka SoftBank HAWKS, details the development of the AI avatar below.

“The Fukuoka SoftBank HAWKS are the team of the SoftBank Group, so we’ve always felt a need to make the stadium’s interior a symbolic space. SoftBank’s head office has a facility called the Executive Briefing Center to make proposals to customers: we needed to create a satellite booth of it inside the stadium. We made the AI avatar as part the project.”

The question of digitally transforming a stadium using AI arose in 2023. An existing strategic partnership with Microsoft Japan inspired SoftBank to consider Microsoft’s cutting-edge AI technology.

“We’ve had plenty of stadium DX initiatives, but most of them have ended in the testing phase,” says Nishikawa. “No matter how impressive the technology is, it’s useless if consumers visiting the stadium don’t enjoy it. But I was convinced that an AI avatar would be intriguing.”

First release completed in just six weeks using five one-week development sprints

The AI avatar development project began in May 2024. Development work started in June, with Microsoft recommending Avanade as a development partner. An overview of the completed system is shown in the diagram below. Senior Director of Avanade Hiroshi Senga, who worked on the AI avatar, explains the Azure AI services at the core of the project below.

“Azure’s AI services are accessible via APIs, compatible with other PaaS, and provide brilliant functionality and development productivity. Fine tuning was a cinch and we could adjust intonation, speed, and lip syncing swiftly. In addition, processing on Azure on the cloud side minimizes the load on devices.”

The development team created the AI avatar through five week-long sprints, with the first release coming just a month and a half after starting the specification review. Hiroyuki Ishida of the Fukuoka SoftBank HAWKS Sales Department says this speed was astonishing.

However, development did not proceed without incident. The biggest development challenge was a time delay when using external data.

“We couldn’t obtain data on ongoing matches from Data Stadium’s real-time stats,” explains Senga. “We tried to get data from the AI avatar’s side using pull delivery, but there was a delay of about one minute.” However, adopting push delivery, in which Data Stadium sends the data, and tuning the generative AI to minimize unnecessary processing shortened the delay to about 10 seconds, he says.

This speed is comparable to that of sports streams. This amount of lag occurs with any data transmission, so shortening it further would require communication infrastructure changes and the cooperation of baseball data providers.

Learning from video and audio data completed in only three to four days

Fukuoka SoftBank HAWKS identified potential AI avatars as development progressed.

“We originally considered active players, but we couldn’t organize time with them during the season,” says Ishida. An examination of famous retired HAWKS players revealed Igarashi as the perfect choice.

After the team made a proposal in June 2024 and agreed on a schedule with Igarashi, Microsoft arranged a studio in Tokyo to record video and audio for learning. Igarashi was only required for about two hours, recording about 30 minutes of video and 15 minutes of audio.

Takashi Okawa, a Partner Solution Architect from Microsoft Japan, explains the AI learning and data use below.

“3D models are widely used for realistic avatars, but the creation and learning process takes a lot of time. Microsoft’s AI avatars let us convert video into training data, which made the learning time significantly faster than conventional methods. Learning for audio only took about an hour, and three to four days including video. Data has to be pre-approved by a trainer, which prevents unauthorized learning.”

A broader prospective audience for the entire SoftBank Group

July 2024 marked the official debut of the AI avatar. Ishida details its level of sophistication as follows.

“The Azure-based AI avatar has realistic speech intonation, speed, lip sync, and blinking. It really feels like Igarashi is there. It features around 10 preset gestures, which automatically occur based on on-field events, including a special pose for the Fukuoka SoftBank HAWKS’ 2024 “Viva” slogan.”

While the AI avatar is only available in one of the 16 Microsoft Premium Suites, it’s impressing many visitors. The site is positioned as a satellite of the Executive Briefing Center, so most users are corporate visitors. Trained hosts provide hands-on demonstrations to these guests.

“The first users of the Microsoft Premium Suite were amazed when they saw Igarashi as an AI avatar. Word of mouth spread and reservations for the remaining 40 games of the season and the entire next season sold out. The AI avatar is creating a broader prospective audience for the entire SoftBank Group.” says Nishikawa.

Nishikawa hopes to expand the AI avatar outside of Microsoft Premium Suites so more visitors can share the experience. He also wants to link the service to the Fukuoka SoftBank HAWKS official app to tailor the experience to customer preferences and enable food and beverage ordering.

* The information in this customer story is current as of November 2024, when the interview was conducted.