Jeff Huang, who will graduate with his doctorate degree in June, had done research on games before so he thought, why not do something fun in his fourth and final internship at Microsoft Research and apply methods he used in his dissertation research to look at how video game skills develop and what sets the top players apart from the rest.

The result was a Best Paper Honorable Mention for “Mastering the Art of War: How Patterns of Gameplay Influence Skill in Halo” that will be awarded at CHI 2013 in Paris in April.

This is Huang’s second Best Paper Honorable Mention. His first was at CHI 2011 for “No Clicks, No Problem: Using Cursor Movements to Understand and Improve Search.”

“My usual work is looking at people’s interactions with technology and seeing how we can use this to understand human behavior; use it to improve the design of systems. I normally look in search, but for this study, I applied the same techniques to games, such as using time series analysis," says Huang.

Microsoft owns Halo, a first-person shooter game played online by millions of people. They had never done a large-scale analysis of the data available from the game, so the Microsoft Research Software Engineering team decided to enlist Huang for this project.

What made the project feasible was the existence of a very large and complete data set, including how many games users play over what time period, what effect this had on their TrueSkill ranking (a rating generated from repeated multiplayer matches given to players after every game), what happened to skill levels when people took a break from playing or what was going on immediately before they stopped completely.

After looking at data from 7 months of games involving over 3 million players, the results were both expected and surprising.

For example, there was a correlation between the number of games played and skill level – players who spent more time on the game had a higher level of skill than those who played occasionally during a set period of time. The data also showed that players’ skills diminished when they took a break, but recovered quickly after they began playing again, similar to what happens to people’s muscle mass when they lift weights in a gym.

However, it turns out that people who had lower skills continued to play the game, while people who became good and then saw a peak in their TrueSkill ranking would stop. Were the lower-skilled players highly competitive? Did the good players just get bored?

To try and answer some of the qualitative questions, the researchers conducted a survey and asked the gamers to say what they believe contributed to their skill at playing Halo. Only 70 people participated in the survey, however, the overall results agreed with the data analysis. The researchers report that the participants “were keenly aware of their skill level, and it was an important determinant of how much they played, their enjoyment, and their in-game behavior.”

Will the research provide a potential Halo player with a winning formula? Unfortunately, the best players or "Master Blasters" as the researchers called them, had "varied skill patterns that often ran counter to the typical player." In other words, the researchers couldn't predict who would be the best.

Huang hopes the findings will assist game developers create better experiences for players by increasing enjoyment and eliminating frustrating scenarios. Although the findings only pertain to gaming, he believes they might contribute to understanding learning behavior in other areas such as sports training and education.