E

With teams of increasingly “pro” gamersmillions of viewers, advertisers and even sponsors, eSport is a social phenomenon. According to a study by the media agency Initiative“electronic sport” is one of the new online entertainment practices that will punctuate our lives over the next 5 years, along with social gaming, video streaming, podcasts and the metaverse. Improve the viewing experience “matches” is therefore a major challenge for the organizers of competitions and events.

During an eSport broadcast, for example a multiplayer game of Counter Strike, StarCraft II or League of Legends, what takes precedence is the position of the “camera”, as well as the analysis made by the commentators. So far, human beings take care of commentating the game, while others, the “observers”, control the camera. However, when the players are numerous and they are scattered all over a map, decisive actions sometimes escape their vigilance. Thus, even if the human observers know the game well, have a good “instinct” and are very reactive, they can be confronted with technical problems (unstable connection) which slow them down, or with complicated game situations to follow, when they occur simultaneously.

AI to the rescue of the eSport business

Since eSports games are starting to weigh in money, you understand why this is a real issue. In South Korea, a team of researchers from the Gwangju Institute of Science and Technology (GIST) may well have found the solution: use AI to “maximize observer action”. Their system, recently unveiled in ScienceDirectis an “automatic observer, which uses an object detection algorithm and learning data from human spectators”, explains Dr. Dr. Kyung-Jong Kim, who leads the project.

As the South Korean researchers point out, several “automated observation” methods have already been developed by others, with the aim of create “artificial observers”. Kyung-Jong Kim classifies these methods into two types: those based on “rules”, and those based on learning. Used in games such as League of Legends and StarCraft, cameras that use “rules” track predefined “significant events” across the map. Learning-based methods, for example to track Dota (Defense of the Ancients) 2 games, use several approaches, such as “teamfight prediction”, classification of event types predefined, and the “future importance ranking” of the players. “So far, rule-based and learning-based methods are both event-based. They have to determine them and predefine their importance, which requires deep knowledge of the domain during development. In addition “, they cannot observe events that have not been specifically defined. Also, the importance of events can change depending on the current state of the game, which makes the observer unreliable,” they say. .

Another catch: As events “become more varied”, such as in real-time strategy games such as StarCraft, which feature “a lot more units and strategies than Dota 2”, these event-based methods become “even less viable, because the events are more difficult to define”.

An “intelligent observer”

According to them, the AI ​​method followed by the GIST researchers stands out because it learns “observation styles” from human “game observation data”. “Unlike event-based approaches, the observation module of our approach can observe the most important scenes of the large search space without having to predefine the events in detail, thus covering situations that are ambiguous or difficult to clearly define. Dr. Dr. Kyung-Jong Kim.

The researchers used data from 25 human spectators watching the same game (StarCraft 2), along with an object detection algorithm, Mask R-CNN. “It’s about focusing on the two-dimensional spatial regions seen by the viewers. Whereas event-based learning focuses only on the actions of game objects (e.g., attacking units, creating units in buildings and improving attack/defense in buildings), our method focuses on patterns in a region that humans observe,” they write.

The idea is therefore to define the object to be observed by the system as the two-dimensional space seen by the spectators, and not as a single object to be observed. For this, areas observed by human viewers are “marked” with a “1” unit, and the rest of the screen is filled with “0”. The data thus obtained fed the neural network of the “automatic observer” of the GIST, in order to allow it to define “regions of common interest” (ROCI). These ROCIs are, according to the researchers, “the most interesting areas for viewers to see.”

Dr. Kyung-Jong Kim believes that this “intelligent observer” project could greatly improve the experience of watching eSports competitions, in particular those (more and more numerous) which are transmitted in “multi-screen” mode. According to him, this AI system could also offer a “less expensive solution for small competition organizers, who do not have the budget to afford a dedicated observer”. In other words, human observers could soon find themselves in the hot seat…