raw_lines <- read_lines(
"https://raw.githubusercontent.com/AnissSahraoui/DATA607/main/Week4/tournmentinfo.txt"
)
length(raw_lines)[1] 196
Turning a formatted text file into a tidy CSV
The file tournmentinfo.txt holds the results of a 7-round chess tournament as a formatted text table, the kind meant to be read by a person rather than by software. The goal of this project is to turn it into a CSV file with one row per player and these five columns:
The last column is the interesting one. It is not in the file: each player’s opponents are listed only by pair number, so their ratings have to be looked up and averaged. For the first player, Gary Hua, the expected result is Gary Hua, ON, 6.0, 1794, 1605.
The file is read from my GitHub repository, so this document runs on any machine.
raw_lines <- read_lines(
"https://raw.githubusercontent.com/AnissSahraoui/DATA607/main/Week4/tournmentinfo.txt"
)
length(raw_lines)[1] 196
cat(raw_lines[1:7], sep = "\n")-----------------------------------------------------------------------------------------
Pair | Player Name |Total|Round|Round|Round|Round|Round|Round|Round|
Num | USCF ID / Rtg (Pre->Post) | Pts | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
-----------------------------------------------------------------------------------------
1 | GARY HUA |6.0 |W 39|W 21|W 18|W 14|W 7|D 12|D 4|
ON | 15445895 / R: 1794 ->1817 |N:2 |W |B |W |B |W |B |W |
-----------------------------------------------------------------------------------------The structure repeats every three lines: a row of dashes, then two lines per player.
W 39 means a win against player 39.Three things make the parsing less than routine:
H, U, X, B) has no pair number, so it must not count as a game.I keep only the data lines, drop the two header lines, and then take the odd-numbered lines as the first line of each player and the even-numbered lines as the second.
data_lines <- raw_lines |>
keep(\(line) str_detect(line, fixed("|"))) |> # drop the rows of dashes
tail(-2) # drop the two header lines
first_lines <- data_lines[c(TRUE, FALSE)]
second_lines <- data_lines[c(FALSE, TRUE)]
c(data_lines = length(data_lines), players = length(first_lines))data_lines players
128 64
Both lines are split on the | character, which makes every field a column I can pick by position.
field <- function(lines, i) {
str_split(lines, fixed("|")) |> map_chr(i) |> str_trim()
}
players <- tibble(
pair = as.integer(field(first_lines, 1)),
name = str_to_title(field(first_lines, 2)),
total_points = as.numeric(field(first_lines, 3)),
state = field(second_lines, 1),
# "15445895 / R: 1794 ->1817" or "15142253 / R: 1641P17->1657P24"
# The pre-rating is the first number after "R:". The "P17" on some ratings marks a
# provisional rating and is not part of the number, so \\d+ stops before it.
pre_rating = field(second_lines, 2) |> str_extract("(?<=R:)\\s*\\d+") |> as.integer(),
# Fields 4 to 10 of the first line are the seven rounds. Taking the digits gives the
# opponent's pair number, and gives NA for a bye or an unplayed round.
opponents = map(first_lines, \(line) {
str_split_1(line, fixed("|"))[4:10] |> str_extract("\\d+") |> as.integer()
})
)
players |> select(pair, name, state, total_points, pre_rating) |> head(5)| pair | name | state | total_points | pre_rating |
|---|---|---|---|---|
| 1 | Gary Hua | ON | 6.0 | 1794 |
| 2 | Dakshesh Daruri | MI | 6.0 | 1553 |
| 3 | Aditya Bajaj | MI | 6.0 | 1384 |
| 4 | Patrick H Schilling | MI | 5.5 | 1716 |
| 5 | Hanshi Zuo | MI | 5.5 | 1655 |
With every player’s rating in hand, each player’s opponents are looked up by pair number. The average uses only the rounds that were actually played, which is what the project asks for: the ratings are divided by the number of games played.
rating_of <- set_names(players$pre_rating, players$pair)
players <- players |>
mutate(
games_played = map_int(opponents, \(o) sum(!is.na(o))),
opponent_ratings = map(opponents, \(o) unname(rating_of[as.character(o[!is.na(o)])])),
avg_opponent_rating = map_dbl(opponent_ratings, mean) |> round()
)
players |>
select(name, total_points, pre_rating, games_played, avg_opponent_rating) |>
head(5)| name | total_points | pre_rating | games_played | avg_opponent_rating |
|---|---|---|---|---|
| Gary Hua | 6.0 | 1794 | 7 | 1605 |
| Dakshesh Daruri | 6.0 | 1553 | 7 | 1469 |
| Aditya Bajaj | 6.0 | 1384 | 7 | 1564 |
| Patrick H Schilling | 5.5 | 1716 | 7 | 1574 |
| Hanshi Zuo | 5.5 | 1655 | 7 | 1501 |
gary <- players |> filter(pair == 1)
tibble(
name = gary$name,
state = gary$state,
total_points = gary$total_points,
pre_rating = gary$pre_rating,
opponent_ratings = paste(gary$opponent_ratings[[1]], collapse = ", "),
games_played = gary$games_played,
average = gary$avg_opponent_rating
)| name | state | total_points | pre_rating | opponent_ratings | games_played | average |
|---|---|---|---|---|---|---|
| Gary Hua | ON | 6 | 1794 | 1436, 1563, 1600, 1610, 1649, 1663, 1716 | 7 | 1605 |
This matches the project exactly: the same seven opponent ratings, and an average of 1605.
check_results <- list(
n_players = nrow(players),
pair_numbers_1_to_64 = identical(players$pair, 1:64),
missing_values = sum(is.na(players$name), is.na(players$state),
is.na(players$total_points), is.na(players$pre_rating),
is.na(players$avg_opponent_rating)),
states = paste(sort(unique(players$state)), collapse = ", "),
rounds_per_player = paste(range(map_int(players$opponents, length)), collapse = "-"),
games_played_range = paste(range(players$games_played), collapse = "-"),
# every game has two players, so the pair numbers should appear in each other's lists
games_are_mutual = {
games <- players |> select(pair, opponents) |> unnest(opponents) |> drop_na()
all(map2_lgl(games$pair, games$opponents,
\(p, o) p %in% games$opponents[games$pair == o]))
}
)
tibble(check = names(check_results),
result = map_chr(check_results, as.character))| check | result |
|---|---|
| n_players | 64 |
| pair_numbers_1_to_64 | TRUE |
| missing_values | 0 |
| states | MI, OH, ON |
| rounds_per_player | 7-7 |
| games_played_range | 1-7 |
| games_are_mutual | TRUE |
All 64 players are present with no missing values. The last check is the strongest one: for every game, the two players list each other, which means no opponent number was misread.
Every player has seven rounds, but not everyone played seven games.
players |>
count(games_played, name = "players") |>
arrange(desc(games_played))| games_played | players |
|---|---|
| 7 | 41 |
| 6 | 13 |
| 5 | 7 |
| 4 | 1 |
| 3 | 1 |
| 1 | 1 |
players |>
filter(games_played < 7) |>
select(name, total_points, games_played, avg_opponent_rating) |>
head(5)| name | total_points | games_played | avg_opponent_rating |
|---|---|---|---|
| Kenneth J Tack | 4.5 | 6 | 1506 |
| Mike Nikitin | 4.0 | 5 | 1386 |
| Eugene L Mcclure | 4.0 | 6 | 1300 |
| Gaurav Gidwani | 3.5 | 6 | 1222 |
| Chiedozie Okorie | 3.5 | 6 | 1314 |
These players had a bye or an unplayed round. Their average uses only the games they played, so a bye neither counts as an opponent with a rating of zero nor drags the average down.
tournament_results <- players |>
mutate(
# written with one decimal place, so a score of 6 appears as "6.0" as in the project example
total_points = sprintf("%.1f", total_points)
) |>
select(
player_name = name,
player_state = state,
total_points = total_points,
player_pre_rating = pre_rating,
avg_opponent_pre_rating = avg_opponent_rating
)
write_csv(tournament_results, "tournament_results.csv")
head(tournament_results, 10)| player_name | player_state | total_points | player_pre_rating | avg_opponent_pre_rating |
|---|---|---|---|---|
| Gary Hua | ON | 6.0 | 1794 | 1605 |
| Dakshesh Daruri | MI | 6.0 | 1553 | 1469 |
| Aditya Bajaj | MI | 6.0 | 1384 | 1564 |
| Patrick H Schilling | MI | 5.5 | 1716 | 1574 |
| Hanshi Zuo | MI | 5.5 | 1655 | 1501 |
| Hansen Song | OH | 5.0 | 1686 | 1519 |
| Gary Dee Swathell | MI | 5.0 | 1649 | 1372 |
| Ezekiel Houghton | MI | 5.0 | 1641 | 1468 |
| Stefano Lee | ON | 5.0 | 1411 | 1523 |
| Anvit Rao | MI | 5.0 | 1365 | 1554 |
The file tournament_results.csv has 64 rows and 5 columns, and is ready to load into a SQL database. The first line of the file is:
read_lines("tournament_results.csv", n_max = 2) |> cat(sep = "\n")player_name,player_state,total_points,player_pre_rating,avg_opponent_pre_rating
Gary Hua,ON,6.0,1794,1605
The CSV is the deliverable, but the data answers a couple of questions easily now.
ggplot(players, aes(x = pre_rating, y = total_points)) +
geom_smooth(method = "lm", se = FALSE, color = "grey70", linewidth = 0.8) +
geom_point(size = 2.6, alpha = 0.8, color = "#2a78d6") +
scale_y_continuous(breaks = 0:7) +
labs(x = "Pre-tournament rating", y = "Points scored (out of 7)") +
theme_minimal(base_size = 12)cor(players$pre_rating, players$total_points) |> round(2)[1] 0.61
Rating and score are strongly related, as expected, but the fit is far from perfect. The players furthest above the line did better than their rating suggested:
model <- lm(total_points ~ pre_rating, data = players)
players |>
mutate(expected_points = round(predict(model), 1),
points_above_expected = round(total_points - expected_points, 1)) |>
slice_max(points_above_expected, n = 5, with_ties = FALSE) |>
select(name, pre_rating, total_points, expected_points, points_above_expected)| name | pre_rating | total_points | expected_points | points_above_expected |
|---|---|---|---|---|
| Aditya Bajaj | 1384 | 6 | 3.5 | 2.5 |
| Jacob Alexander Lavalley | 377 | 3 | 0.6 | 2.4 |
| Dakshesh Daruri | 1553 | 6 | 3.9 | 2.1 |
| Anvit Rao | 1365 | 5 | 3.4 | 1.6 |
| Stefano Lee | 1411 | 5 | 3.5 | 1.5 |
Aditya Bajaj stands out: the third-highest score in the tournament from the 26th-highest rating. That is also visible in the file, where his rating rises from 1384 to 1640.
The file’s structure is regular, which is what makes parsing it possible: every player occupies exactly two lines, and every field sits between the same pair of | characters. The work was in three details:
1641P17 is read as
The results were checked in three ways: the project’s own example for Gary Hua reproduces exactly, no values are missing for any of the 64 players, and every game appears in both players’ round lists.