Study finds rating board’s loot box warnings being applied inconsistently

New research suggests that alerts for games with loot box mechanics are inconsistent and unreliable.

Led by Leon Y Xiao In collaboration with universities in Denmark, London and the USA, the study examines a random selection of smartphone games that contain loot boxes and can be downloaded from the Google Play Store. The study aimed to find out how consistent lootbox presence alerts are across PEGI and ERSB, as well as the consistency of age ratings across the Google Play Store as monitored by IARC.

In August last year, Xiao examined the implementation of Belgium’s loot box ban and concluded that the country’s legislation had proved ineffective.

Xiao found that 60 percent of the games included in this latest study were not rated similarly by PEGI and ESRB. Of the 40 games that had different ratings, 88 percent were due to the ERSB not applying the loot box warning. Out of a random selection of 100 games in the Google Play Store, 71 percent of the games with loot boxes were not marked as such.

Minecraft and Roblox were considered separately because the titles are “unique games where third-party user-generated content (including loot boxes) can be created and published, and then bought and sold for real money.” Xiao noted that as of January 12, 2023, none of the games were flagged with the lootbox warning on the Google Play Store and urged Roblox Corporation and Mojang Studios to attach the warning.

There are of course limitations to this study. The sample size of 100 games is a small number compared to the sheer volume of apps available on the Google Play Store. Xiao also pointed out difficulties in accommodating the different regulations, where countries have their own individual boards and labels on different platforms.

“Consumers, parents, regulators and everyone else involved should focus on the [loot box] Label cautiously,” Xiao concluded, expressing concern about the inconsistencies found. Xiao suggested that organizations work together to ensure standards match and try to apply the label retrospectively to already-released games.

Xiao’s full report can be read on-line.