Does a players KD stat affect the outcome of a 12v12 game of Conquest?

Author: Pink_Eyed_Recon
Date: June 10, 2014

1. Introduction

It is a hotly debated topic among players in the Battlefield community on the direct effect of a players Kill to Death ratio on the outcome of a game of Conquest. These debats are always filled with personal bias and selective examples to try to provide support of one side of the argument or the other. The problem with these forms of arguments is they do not provide any substantial evidence to give a conclusive answer to the question and the evidence is so sparse, it is difficult to make use of it to make any significant claims in support of any side of the debate. The motivation of this report is to try to put this topic to rest.

In this analysis I provide an answer to the question:

Does a players KD stat affect the outcome of a game of 12v12 Conquest in Battlefield 4?

The analysis evaluate the given question as well as the null-hypthesis that player KD stat does not have an effect on the outcome of a 12v12 Conquest game. I show with a high degree of confidence that player KD is able to predict the outcome of a game of Conquest more accurately than random guessing (the null-hypothesis).

Note in this analysis I am stricly speaking about a player's overall KD as recorded on BF4stats, I am ignoring the players performance in the battle reports being analysed.

2. The Data

I collected 5282 conquest reports directly from battlelog using the website's JSON API. Specifically, 2862 PS3 reports and 2420 XBOX 360 reports. The battle report data gives us key pieces of data that are required in answering the question in this analysis.

1. The report data provides a list of players on each team, including players who did not complete the round
2. The report data provides the final outcome of the round so that we know which team won the match

The table below provides a summary of the number of wins, losses, and draws for each team in the battle reports collected:

Along with these reports, I collected stats on every player that played in these games from BF4stats. There are a total of 100815 collected with 54576 players from PS3 and 46239 players from 360.

3. The Experiment Design

To answer the question asked in this analysis we need to test two questions:

1. Does a teams average KD have an effect on the chance of a team winning?
2. Can the result of answering quesiton #1 be explained by random chance

Why am I focusing on these questions instead of looking at the direct effect of an individual players KD on the outcome of a game? The simple answer is: a players KD is automatically included in the calculation for a team's average KD. So if I can answer question #1 then I am also able to answer the original question being asked in this analysis.

Now let me outline the experiment:

1. Calculate the expected team's KD for each team by taking the average KD of all players on the team
2. Take 1000 random subsets of the battle report data and try to predict the outcome of the game using: random guessing (the control group) as well as by selecting the winner based on the team with the larger expected KD (test group)
3. Compute the accuracy of all the experiments and perform a T-test on the control and test groups

Step 1: Calculate expected Team KDs

In the team data I am provided with a list of player's along with an indicator that tells me if they completed the round or did not finish (DNF). In order to figure out the expected team KD I need to take into account both players that did and did not finish the game. The formula I use to compute the team's expected KD is:

\[ E[\text{KD}] = \frac{\sum_i^n w_i\text{KD}_i}{12 + 0.5d} \] \[ n = \text{the number of players on the team} \] \[ \text{KD}_i = \text{the KD of the $i^{th}$ player} \] \[ d = \text{the number of players who dnf} \] \[ w_i = \left\{ \begin{array}{l l} 0.5 & \quad \text{if player $i$ dnf}\\ 1 & \quad \text{otherwise} \end{array} \right. \]

This equation is a weighted average and is convenient because it has the properties:

1. It accounts for teams that may not have 12 players
2. It accounts for players who were in the game but left while also not putting as much weight on those players

Step 2: Run the experiment

First, I remove all battle reports that end in a draw, these reports are omitted because a game of Conquest that ends in a draw is often a result of the server crashing. Because of this, these reports are uninformative in helping answer the question asked in this analysis.

Second, I partition the battle reports into 1000 random subsets containing 100 reports each. For each subset I generate two sets of predictions, a control and a test. The control is generated by randomly assigning team 1 or team 2 as the winner of the round with a probability of 0.5135 and 0.4865 respectively. This is because I want to match the bias for the probability of team 1 and team 2 winning that is found in the data. The test group predicts that the winner of the round is team team with the higher expected KD.

Step 3: Compute Results

To evaluate the results of the experiment I will compute the accuracy of each of the 1000 experiments and then perform a T-test to evaluate the confidence intervals of the test results compared to the control group. The T-test will help us determine if we should accpet the null-hypothesis - that player KD doesn't affect the outcome of conquest games - or the test-hypothesis - that player KDs affect the outcome of conquest games.

3. Results

The T-test results in staggaring evidence that we must reject the null hypothesis. On average, using KD to predict the outcome of a game of conquest gives us 76.556% accuracy, while the null-hypothesis gives us an accuracy of 50.712%. Futhermore if we assumed the null hypothesis to be true, the probability that we should see such a high accuracy in the same experiment when using KDs to predict game outcomes by random chance is 0. Do you know why it says 0? Because the p-value is so small that it goes beyond numerical precision on a computer. This value is smaller than 2 × 10^-16. This is extremely low. As a comparsion, the size of a hydrogen atom is 1e-10m. You're 500000 times more likely to find a single hydrogen atom on a meter line in a vaccume than you are in believing that a player's KD has no effect on the outcome of a game of 12v12 conquest.

Here is the output of the T-test for the experiment to see:

## 
##  Welch Two Sample t-test
## 
## data:  testResults and controlResults
## t = 124.9, df = 1930, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2544 0.2625
## sample estimates:
## mean of x mean of y 
##    0.7656    0.5071

Below is a plot showing the distributions of the experimental results so you can see for yourself.

4. Conclusion

I have shown with high degrees of confidence that the claim that a player's KD does not affect the outcome of a game of 12v12 conquest is not correct. And I also know that this isn't going to put the debate to rest. So finally I would like to provide all the data and code I used in this analysis so that anyone with a computer and the motivation can verify my results.

The code and data will be available at my github repo within the next 2 weeks. I am currently witholding the data until I finish my class project. Everything will be available after I submit my project.