Penalty kicks. Five-set tennis matches. Game-winning baskets. Tiebreakers. Sudden death overtime. Seeing someone win or lose by the thinnest margin is one of the most thrilling aspects of watching athletes compete.
But that’s not the goal on Caldera, the virtual island setting of “Call of Duty: Warzone.” Popular “Warzone” players on Twitch and YouTube do their best to seek out game lobbies filled with less skilled players — often called “bot lobbies” — and completely dominate them. It’s the gaming equivalent of LeBron James looking to join pickup games at the local YMCA. Prominent gamers have specifically called for developers to let them match with players at lower skill levels.
“I know you’re probably tired of hearing about skill-based matchmaking,” wrote Matthew “Nadeshot” Haag, CEO of the esports and lifestyle organization 100 Thieves, in a 2020 tweet, “but I truly believe it is imperative that [Call of Duty developer] Treyarch dials back the difficulty of lobbies.” When Haag, a former Call of Duty world champion, gets into a Call of Duty match, he wants to play against gamers who’ve jumped on after school or work, not hardcore gamers like himself.
Skill-based matchmaking is a system multiplayer games typically use to place players of similar skill levels in matches against each other to fairly balance teams and maximize the enjoyment players get from the game. It keeps track of a player’s performance and uses win-loss ratios, kill streaks, death counts and other measures to calculate their skill level — though the exact formula is unique to each game and one that developers keep under wraps to stay competitive in the crowded landscape of competitive multiplayer games.
Haag isn’t alone in his dislike of the ubiquitous system. Michael “Shroud” Grzesiek, a retired Counter-Strike pro who has made a name for himself as one of the best FPS players in the world with 10 million Twitch followers, has also established his distaste for skill-based matchmaking, arguing that it “doesn’t work.” Jack “CouRage” Dunlop, a co-owner of 100 Thieves, has also complained about it online. With skill-based matchmaking, he wrote, “you have to sweat 100 percent of the time.” They contend their audiences want to see them pull off amazing victories, not struggle endlessly against other top players. While most players may want a fun, fair game, streamers want to put on a show.
For Jordan “HusKerrs” Thomas, a popular streamer and competitive “Call of Duty: Warzone” player, skill-based matchmaking is a labor issue. It “negatively affects the top 1 percent of players/streamers the most because it forces us to ‘sweat’ or try hard for good content and to entertain our viewers,” Thomas wrote in a Twitter DM. High-level play against skilled opponents in shooting games can be opaque or boring for casual audiences. By racking up high kill streaks or stringing together multiple crushing victories in less balanced matches, streamers can more clearly show off their skill to viewers.
Eric “Snip3down” Wrona, one of the best Halo players ever, made his professional debut in “Halo 3” back in 2008. Since then, he’s played on the rosters of more than a dozen esports teams over seven different games. Speaking with The Washington Post over the phone, Wrona, who is signed to FaZe Clan’s “Halo Infinite” team, described the quirks, difficulties and blind spots of various matchmaking systems over the years.
Some matchmaking systems were definitely better than others, though over time, he said, matchmaking seems to have become both more complex and more opaque. While playing ranked matches in “Halo 5: Guardians,” Wrona consistently struggled — and he had no idea why. “I even tweeted out to the head developer of the skill-based matchmaking system because I was winning 23 percent of my games.”
Wrona sometimes felt that the better he performed, the worse his teammates became. It felt like the system, in its quest to find him fair fights, had gone haywire. “It was like, ‘I’m one of the best players in this game and I’m losing 70 percent of my games, how is this possible?’ There was a hidden MMR … and it was such an intricate system.”
The hidden MMR Wrona referred to stands for “matchmaking ranking,” a value that, like the Elo rating popularized by chess, attempts to establish a player’s ranking compared to their peers. Elo is known for being a standardized statistical measure of relative skill that’s fairly easy to calculate, so a player’s Elo rating can be figured out by anyone with some patience and a scientific calculator. MMR, on the other hand, is a secret sauce. While it has the same ostensible goal of representing a relative measure of skill, it is a generic term for an array of measures developers use that can vary dramatically between different games.
Complex systems that ensure fair matches sound like a good thing. Grouping people by their skill level is a time-honored structure employed to ensure a balanced playing field for all competitors involved. Everything from beer leagues to semipro sports are organized so that every team has chance at victory. Gaming industry giants like EA, Epic and Activision Blizzard use this same structure for online multiplayer, incorporating sophisticated techniques like machine learning to tune their matchmaking algorithms so that gamers are pitted against similarly skilled opponents.
Activision Blizzard, Bungie and EA did not reply to repeated requests for comments on their matchmaking algorithms.
Technical advancements make skill-based matchmaking techniques better every year, enticing average audiences to play more. But those same changes have also left a sour taste in some players’ mouths who publishers have a vested interest in keeping happy — their live streams help market games. Game companies have the seemingly impossible task of satisfying both sides; on one end, the massive player base of everyday gamers that define their bottom line and, on the other, the pros and content creators they use as PR for those same audiences.
But if these systems are indeed built to maximize players’ enjoyment, it can sometimes seem like they’re not working very well. Hate for skill-based matchmaking is hardly a phenomenon confined to top streamers or salty Call of Duty players. As awareness about these algorithms grows, communities in “Valorant,” “Overwatch,” “Apex Legends” and even more casual games like “FIFA” and “Dead by Daylight” have all, at one point or another, sharply criticized matchmaking for reducing their enjoyment of the game. In part, it’s an easy scapegoat for frustrated players. As Vice’s Steve Rousseau puts it: “The issue today is not that skill-based matchmaking exists, but that players are now aware of just how prevalent it is.”
Today, speculation about how matchmaking “truly” works has spawned several analyses as well as its own cottage industry on YouTube, where videos on the subject range from neutral explainers to rants delivered as if from the pulpit: “The algorithm is so wicked,” announced YouTube creator J. “Murdashow” Guidry in one video. “It combs through the labyrinth of players looking for your nemesis.” The topic is a perpetual driver of viewership, in part because there are few satisfying answers available to players.
In a phone interview, popular “Call of Duty: Warzone” streamer and XSET content creator JaredFPS said he thought companies like Activision, the studio behind the Call of Duty series, base their matchmaking algorithms on more than a player’s skill in any single game.
“They know everything about you,” said Jared, who requested The Post not publish his full name due to safety concerns. “They have information from every single Call of Duty ever made. They know how much money you’ve spent, they know if you spend money, they know if you use the buy station [in ‘Warzone’] a lot … the way your movement is, how many loadouts you buy … they know all that information.”
Zhengxing Chen, a research scientist at Facebook, is the lead author on a paper about engagement-optimized matchmaking that gets an unhealthy amount of attention from aggrieved gamers who believe it proves a conspiracy against players. In reality, the paper only confirms, in formal terms, the widespread annoyance that streamers and other players feel when they’re constantly pitted against opponents who are an even match.
“Are fairly matched games always beneficial for player experience?” the paper’s introduction asks, proposing that a purely skilled-based matching algorithm could be improved with reference to data about risk of what the authors call player churn — that is, how likely players are to put down the game for a period of time after playing it.
Armed with that extra info about player behavior, Chen and his co-authors simulated 10,000 rounds of 1-vs-1 matchmaking based on real data from a popular undisclosed EA game. The results showed that their engagement-optimized matchmaking strategy showed a small but statistically significant improvement in keeping players playing over a pure skill-based matchmaking strategy.
In a phone interview, Chen confirmed the growing complexity of matchmaking techniques: “Previously, they only looked at your win-loss history … and tried to develop one scalar score [like Elo or MMR] for you to summarize your skill. But as time goes on, I can see that there’s work using neural networks to summarize your skills in multiple aspects, not just one single score, and trying to use more history, more information to estimate your skills in different areas.”
As matchmaking strategies have advanced they have broadened too, using insights from fields like machine learning and data science to further refine player experiences.
A shooter’s matchmaking system might consider factors like previous wins and losses, kills and deaths, how often players quit, what mode they’re playing, how many hours they’ve played, whether they’re playing with friends, or even what time of day it is. These parameters are constantly updated as more information about player performance becomes available. Advanced statistics are then used to draw inferences about the plausible outcome of every game before it happens.
“Even the people who are putting together the algorithms — maybe there’s one or two people at a company who really understand everything that’s going on in the matchmaking, which are often one of the most complicated pieces of server code,” said Naomi Clark, a game developer and the chair of New York University’s Game Center.
According to Clark, games in the mobile space, like Zynga’s “Farmville,” were among the first to hop on the trend of engagement-optimization, which uses data to keep players playing. The single-player “Farmville” gobbled up player data to determine more efficient ways of keeping them around, increasing their play time and getting them to spend money. In a multiplayer setting, these systems anticipate the complaints of gamers who quickly tire of playing against opponents just as good as they are and models their frustrations, curbing them before they throw the controller across the room.
Advances in matchmaking are just one tool in a larger strategy developers use to keep existing players and attract new ones. But the notion of a “good” match can drastically vary between individuals. Some players enjoy struggling against peers as skilled as themselves. Others might prefer more casual games between players with a wide range of skill levels. Still others might prefer matches for reasons unassociated with relative skill level, such as whether their teammates have microphones for in-game communication.
Even developers themselves don’t always agree on a concrete answer. A recent “Halo Infinite” blog post explaining the game’s matchmaking was followed by a public dissent from Max Hoberman, the designer of the ranking systems in “Halo 2” and “Halo 3.” In a series of tweets, Hoberman disagreed “that perfectly balanced games were always the most fun; in fact, I felt they were often the most stressful.”
While matchmaking algorithms have hoovered up progressively more in-game variables over the years, they don’t yet appear to account for all the ways gaming has ballooned into a cultural mainstay — chiefly on streaming platforms.
In some games, enjoyment of close matches is what keeps players coming back, and matchmaking in those games has a close relation to pure measures of skill. In others, developers may have determined that giving players an easy match every so often is a similarly valid way of designing the best player experience, encouraging them to spend more time in the game. But what defines the dynamic is the fact that skill-based matchmaking is a business strategy designed to keep players coming back. How players define fairness is subjective; their engagement metrics are not.
News•
News•
News•