Project1_Chess_Tournament

Project 1 Chess Tournament
Loading the File
Calculate the average score

Project 1 Chess Tournament

The goal of this project is to load a chess tournament file, parse its content, interpret its contents programmatically and export a csv file in a specific format suitable for SQL import.

First, we assume the file is located in a current working folder shown below.

All input and output files will be located in the current working directory.

setwd("~/Documents/DATASCIENCE/DATA607/PROJECT1")

library(tidyverse)
library(dplyr)

Loading the File

The strategy is to separate two distinct tasks: * load the text file from disk * Parse and extract meaningful content

For this task, the readLines command is most suitable since it returns a vector of strings. One string per line from the file.

vectorLines = readLines("tournamentinfo.txt")

## Warning in readLines("tournamentinfo.txt"): incomplete final line found on
## 'tournamentinfo.txt'

length(vectorLines)

## [1] 196

head(vectorLines)

## [1] "-----------------------------------------------------------------------------------------" 
## [2] " Pair | Player Name                     |Total|Round|Round|Round|Round|Round|Round|Round| "
## [3] " Num  | USCF ID / Rtg (Pre->Post)       | Pts |  1  |  2  |  3  |  4  |  5  |  6  |  7  | "
## [4] "-----------------------------------------------------------------------------------------" 
## [5] "    1 | GARY HUA                        |6.0  |W  39|W  21|W  18|W  14|W   7|D  12|D   4|" 
## [6] "   ON | 15445895 / R: 1794   ->1817     |N:2  |W    |B    |W    |B    |W    |B    |W    |"

tail(vectorLines)

## [1] "   63 | THOMAS JOSEPH HOSMER            |1.0  |L   2|L  48|D  49|L  43|L  45|H    |U    |"
## [2] "   MI | 15057092 / R: 1175   ->1125     |     |W    |B    |W    |B    |B    |     |     |"
## [3] "-----------------------------------------------------------------------------------------"
## [4] "   64 | BEN LI                          |1.0  |L  22|D  30|L  31|D  49|L  46|L  42|L  54|"
## [5] "   MI | 15006561 / R: 1163   ->1112     |     |B    |W    |W    |B    |W    |B    |B    |"
## [6] "-----------------------------------------------------------------------------------------"

Starting at line 5: We read the data content, ignore every third row of dashes used to separate entries. The contents prior to line 5 are preamble material that do not need to be parsed. Every third row is a line of dashes separating the players. Each entry is regarded as two lines:

The first line contains the wins/loss record and the ids of the opponents and the total points. The last line contains the state and the pre-tournament score of the player.

vecFirstRow = vectorLines[seq(5, length(vectorLines), 3)]
vecLastRow = vectorLines[seq(6, length(vectorLines), 3)]

(numPlayers = length(vecFirstRow) )

## [1] 64

Now we aggregate the raw text entries into a dataframe of length equal to numPlayers players. We trim the whitespace of the last row of each entry but handle the first row in subsequent processing. Note that stringsAsFactor is set to FALSE to ensure text can be parsed.

ds = data.frame( row1 = vecFirstRow, row2 = str_trim(vecLastRow), stringsAsFactors = FALSE)

Now we tackle the first row. The key insight is that each row is divided by pipe delimiters between relevant fields of interest. Between two pipes, we’ll also need to trim whitespace and extract text or numbers.

ds$row1 %>% str_split("\\|") -> rawFieldsListRow1   # split each row into a list of list of 11 text fields.

# Next we flatten the list of lists of fields and trim whitespace.
# Note that the entire name is easily extracted without need to parse first or last names since there
# is some variation in spelling and character usage.
# Moreover, the raw list of text fields per row is consistent in having 11 fields.
rawFieldsRow1 = str_trim(unlist(rawFieldsListRow1), side="both" )  

# Now we compress the raw trimmed data fields into a data frame of 11 columns and 64 rows.
( row1data = as_tibble( matrix( rawFieldsRow1 , nrow=numPlayers, byrow=T ) ) )

## Warning: `as_tibble.matrix()` requires a matrix with column names or a `.name_repair` argument. Using compatibility `.name_repair`.
## This warning is displayed once per session.

## # A tibble: 64 x 11
##    V1    V2           V3    V4    V5    V6    V7    V8    V9    V10   V11  
##    <chr> <chr>        <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
##  1 1     GARY HUA     6.0   W  39 W  21 W  18 W  14 W   7 D  12 D   4 ""   
##  2 2     DAKSHESH DA… 6.0   W  63 W  58 L   4 W  17 W  16 W  20 W   7 ""   
##  3 3     ADITYA BAJAJ 6.0   L   8 W  61 W  25 W  21 W  11 W  13 W  12 ""   
##  4 4     PATRICK H S… 5.5   W  23 D  28 W   2 W  26 D   5 W  19 D   1 ""   
##  5 5     HANSHI ZUO   5.5   W  45 W  37 D  12 D  13 D   4 W  14 W  17 ""   
##  6 6     HANSEN SONG  5.0   W  34 D  29 L  11 W  35 D  10 W  27 W  21 ""   
##  7 7     GARY DEE SW… 5.0   W  57 W  46 W  13 W  11 L   1 W   9 L   2 ""   
##  8 8     EZEKIEL HOU… 5.0   W   3 W  32 L  14 L   9 W  47 W  28 W  19 ""   
##  9 9     STEFANO LEE  5.0   W  25 L  18 W  59 W   8 W  26 L   7 W  20 ""   
## 10 10    ANVIT RAO    5.0   D  16 L  19 W  55 W  31 D   6 W  25 W  18 ""   
## # … with 54 more rows

# The raw tibble columns names are not informative.
# So we clean up the columns with informative labels before analyzing data.
#
row1data = plyr::rename(row1data, c("V1" = "ID", "V2"="Name", "V3" = "Points", 
                                    "V4"="Round1","V5" ="Round2" ,
                                    "V6"="Round3", "V7"= "Round4" , "V8"="Round5",
                                    "V9"="Round6", "V10" = "Round7"
                                  ) )

It will be necessary to further extract the identifier of each opponent. This is done on the string between consecutive pipe delimiters. Points are always stored as 2 digits and a decimal. Identifiers are always integers. We coerce the values to integers for each of processing.

row1data %>% mutate( ID = as.integer(ID)) %>%
        mutate( Points = as.numeric(str_match(Points, "\\d\\.\\d+" ) ) ) %>%
        mutate( Round1 = as.integer(str_match(Round1, "\\d+"))) %>%
        mutate( Round2 = as.integer(str_match(Round2, "\\d+"))) %>%
        mutate( Round3 = as.integer(str_match(Round3, "\\d+"))) %>%
        mutate( Round4 = as.integer(str_match(Round4, "\\d+"))) %>%
        mutate( Round5 = as.integer(str_match(Round5, "\\d+"))) %>%
        mutate( Round6 = as.integer(str_match(Round6, "\\d+"))) %>%
        mutate( Round7 = as.integer(str_match(Round7, "\\d+"))) %>%

# Useful raw data is selected into a smaller data frame.
select( ID, Name, Points, Round1, Round2, Round3, Round4, Round5, Round6, Round7) -> row1tidy

str(row1tidy)

## Classes 'tbl_df', 'tbl' and 'data.frame':    64 obs. of  10 variables:
##  $ ID    : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ Name  : chr  "GARY HUA" "DAKSHESH DARURI" "ADITYA BAJAJ" "PATRICK H SCHILLING" ...
##  $ Points: num  6 6 6 5.5 5.5 5 5 5 5 5 ...
##  $ Round1: int  39 63 8 23 45 34 57 3 25 16 ...
##  $ Round2: int  21 58 61 28 37 29 46 32 18 19 ...
##  $ Round3: int  18 4 25 2 12 11 13 14 59 55 ...
##  $ Round4: int  14 17 21 26 13 35 11 9 8 31 ...
##  $ Round5: int  7 16 11 5 4 10 1 47 26 6 ...
##  $ Round6: int  12 20 13 19 14 27 9 28 7 25 ...
##  $ Round7: int  4 7 12 1 17 21 2 19 20 18 ...

Let extract the raw data from the last row of the entry. The State is simply a two letter code at the beginning of the line. The player’s pretournament score is a 3-4 digit string preceding by either widespace or a letter ‘P’. We use str_match to extract the score.

state = tibble( State = as.vector( str_match( str_trim(ds$row2), "^.." ) ), 
                preScore = as.integer(str_match(ds$row2, "/ R:\\s+(\\d+)[P ]?" )[,2] ),
                ID = seq(1:numPlayers) )

We need to join the data from the first row and second row of each player’s entry. This requires the use of inner_join. Certain datacolumns need to be renamed for clarity.

fullData = inner_join( row1tidy, state, by = "ID")
fullData = plyr::rename(fullData, c("row1tidy$ID" = "ID"))

## The following `from` values were not present in `x`: row1tidy$ID

The following display shows that data that will be used to (a) calculate the average score of opponents and (b) generate all file output.

library(knitr)
knitr::kable(head(fullData), digits=2, align=c(rep("l", 4) ) )

ID	Name	Points	Round1	Round2	Round3	Round4	Round5	Round6	Round7	State	preScore
1	GARY HUA	6.0	39	21	18	14	7	12	4	ON	1794
2	DAKSHESH DARURI	6.0	63	58	4	17	16	20	7	MI	1553
3	ADITYA BAJAJ	6.0	8	61	25	21	11	13	12	MI	1384
4	PATRICK H SCHILLING	5.5	23	28	2	26	5	19	1	MI	1716
5	HANSHI ZUO	5.5	45	37	12	13	4	14	17	MI	1655
6	HANSEN SONG	5.0	34	29	11	35	10	27	21	OH	1686

Calculate the average score

# Returns a list of the average pretournament score of opponents of player i
# and the number of opponents during the tournament of player i
# This function excludes NA associated with matches with no opponent or score.
preTournamentAverage <- function( i ) {
  
    s1 = fullData$preScore[ fullData$Round1[i]  ]
    s2 = fullData$preScore[ fullData$Round2[i]  ]
    s3 = fullData$preScore[ fullData$Round3[i]  ]
    s4 = fullData$preScore[ fullData$Round4[i]  ]
    s5 = fullData$preScore[ fullData$Round5[i]  ]
    s6 = fullData$preScore[ fullData$Round6[i]  ]
    s7 = fullData$preScore[ fullData$Round7[i]  ]
    
    vals = c( s1, s2, s3, s4, s5, s6, s7 )
    
    v =  mean( vals, na.rm = TRUE)  
    
    return(v)
}

We stage all the contents of the export file into a single data frame. The goal is to arrange the content and order as desired into a single data structure and then dump it to file.

ptl = map_dbl(1:numPlayers, preTournamentAverage ) 

names_vector = fullData$Name
points_vector = fullData$Points
prescore_vector = fullData$preScore

outputDF = tibble( Names = names_vector, State = fullData$State, Points = fullData$Points, 
                       PreScore = fullData$preScore, 
                       OpponentAvg = ptl )

As a sanity check, we inspect the contents of the entire file. This format preserves the order of the players in the original file.

knitr::kable(outputDF, digits=2, align=c(rep("l", 4) ) )

Names	State	Points	PreScore	OpponentAvg
GARY HUA	ON	6.0	1794	1605.29
DAKSHESH DARURI	MI	6.0	1553	1469.29
ADITYA BAJAJ	MI	6.0	1384	1563.57
PATRICK H SCHILLING	MI	5.5	1716	1573.57
HANSHI ZUO	MI	5.5	1655	1500.86
HANSEN SONG	OH	5.0	1686	1518.71
GARY DEE SWATHELL	MI	5.0	1649	1372.14
EZEKIEL HOUGHTON	MI	5.0	1641	1468.43
STEFANO LEE	ON	5.0	1411	1523.14
ANVIT RAO	MI	5.0	1365	1554.14
CAMERON WILLIAM MC LEMAN	MI	4.5	1712	1467.57
KENNETH J TACK	MI	4.5	1663	1506.17
TORRANCE HENRY JR	MI	4.5	1666	1497.86
BRADLEY SHAW	MI	4.5	1610	1515.00
ZACHARY JAMES HOUGHTON	MI	4.5	1220	1483.86
MIKE NIKITIN	MI	4.0	1604	1385.80
RONALD GRZEGORCZYK	MI	4.0	1629	1498.57
DAVID SUNDEEN	MI	4.0	1600	1480.00
DIPANKAR ROY	MI	4.0	1564	1426.29
JASON ZHENG	MI	4.0	1595	1410.86
DINH DANG BUI	ON	4.0	1563	1470.43
EUGENE L MCCLURE	MI	4.0	1555	1300.33
ALAN BUI	ON	4.0	1363	1213.86
MICHAEL R ALDRICH	MI	4.0	1229	1357.00
LOREN SCHWIEBERT	MI	3.5	1745	1363.29
MAX ZHU	ON	3.5	1579	1506.86
GAURAV GIDWANI	MI	3.5	1552	1221.67
SOFIA ADINA STANESCU-BELLU	MI	3.5	1507	1522.14
CHIEDOZIE OKORIE	MI	3.5	1602	1313.50
GEORGE AVERY JONES	ON	3.5	1522	1144.14
RISHI SHETTY	MI	3.5	1494	1259.86
JOSHUA PHILIP MATHEWS	ON	3.5	1441	1378.71
JADE GE	MI	3.5	1449	1276.86
MICHAEL JEFFERY THOMAS	MI	3.5	1399	1375.29
JOSHUA DAVID LEE	MI	3.5	1438	1149.71
SIDDHARTH JHA	MI	3.5	1355	1388.17
AMIYATOSH PWNANANDAM	MI	3.5	980	1384.80
BRIAN LIU	MI	3.0	1423	1539.17
JOEL R HENDON	MI	3.0	1436	1429.57
FOREST ZHANG	MI	3.0	1348	1390.57
KYLE WILLIAM MURPHY	MI	3.0	1403	1248.50
JARED GE	MI	3.0	1332	1149.86
ROBERT GLEN VASEY	MI	3.0	1283	1106.57
JUSTIN D SCHILLING	MI	3.0	1199	1327.00
DEREK YAN	MI	3.0	1242	1152.00
JACOB ALEXANDER LAVALLEY	MI	3.0	377	1357.71
ERIC WRIGHT	MI	2.5	1362	1392.00
DANIEL KHAIN	MI	2.5	1382	1355.80
MICHAEL J MARTIN	MI	2.5	1291	1285.80
SHIVAM JHA	MI	2.5	1056	1296.00
TEJAS AYYAGARI	MI	2.5	1011	1356.14
ETHAN GUO	MI	2.5	935	1494.57
JOSE C YBARRA	MI	2.0	1393	1345.33
LARRY HODGE	MI	2.0	1270	1206.17
ALEX KONG	MI	2.0	1186	1406.00
MARISA RICCI	MI	2.0	1153	1414.40
MICHAEL LU	MI	2.0	1092	1363.00
VIRAJ MOHILE	MI	2.0	917	1391.00
SEAN M MC CORMICK	MI	2.0	853	1319.00
JULIA SHEN	MI	1.5	967	1330.20
JEZZEL FARKAS	ON	1.5	955	1327.29
ASHWIN BALAJI	MI	1.0	1530	1186.00
THOMAS JOSEPH HOSMER	MI	1.0	1175	1350.20
BEN LI	MI	1.0	1163	1263.00

Write the dataframe to an output file in csv format.

readr::write_csv(outputDF, "tournament_output.csv")

Project1_Chess_Tournament

Alexander Ng

2/22/2019