In a nutshell: Activision has released a white paper taking a deep dive into how its SBMM system works. It explains the ins and outs of Call of Duty matchmaking and reveals the secret testing that “proves” players generally like its system.
In April, Activision posted a 1,700-word essay on the intricacies of the game’s skill-based matchmaking (SBMM) titled “Call of Duty: An Inside Look at Matchmaking.” It carried the subheading of “Starting the conversation about Matchmaking in Call of Duty.”
Activision said the blog post fulfilled a promise to start a discussion with the CoD community on how the game forms teams. This one-sided conversation (nobody can comment directly in the blog) pointed out that Activision designed its SBMM to match opponents and teammates of similar skill levels, which is intuitive at its heart.
While no amount of exposition will ever quell the SBMM debate, the company’s best writers sat down and produced a ridiculously in-depth second white paper called “The Role of Skill in Matchmaking.” The 25-page follow-up points out that player skill is only one of Call of Duty’s SBMM components and is not even the most important. Other factors the system considers include:
- Ping rates
- The amount of time to start
- Input types (M/KB vs controller)
- Platform
- Voice chat
- Recent maps and modes
Some of these items seem trivial, like maps, modes, and voice chat, but are arguably more impactful than matching players by skill. If you have played multiplayer shooters, you know how frustrating it is to organize with a teammate whose primary communication consists of one jump for yes and two for no. Of course, nothing beats ping as a matchmaking factor, not even skill.
“As the community will attest, Ping is King,” Activision states. “Connection is the most critical and heavily weighted factor in the matchmaking process.”
It’s all fascinating and worth the read if you are into heavy gaming-related treatises. However, a section about testing matchmaking was arguably more intriguing.
Team formation is not just a procedure defined by complicated algorithms with dozens of factors sorted and considered. Optimizing any system requires extensive testing, and interestingly, many CoD players have experienced SBMM testing for the last five years without even knowing it.
For those times when it seemed like you could not get a fair fight, you might have been part of an A/B test that split the servers 50/50, with half getting their SBMM turned way down, also known as “loosening” the skill disparity.
Analysts then looked at several metrics, including how long players continued to play the game, match-level quit rates, and match outcomes, and compared them between the test and control groups. They found that players in the control group played longer sessions and quit fewer matches than those who had SBMM all but turned off. Match outcomes also show much closer scores in the control group, whereas the test group tended to have more “blowouts.” Activision noted that entirely disabling SBMM would have significantly amplified the disparities.
While its results are not likely to settle the community debate, it shows that Activision is continuously testing and tweaking the matchmaking system to try to make it fair and balanced for everyone.