assignment5

R Markdown

Question1:

Three unclear columns in the dataset that require reading the documentation to understand:

1)bbrID

This column contains unique identifiers like “abdelal01” and “abdulma02.” Without documentation, it’s unclear what these represent. Likely, these are Basketball Reference (BBR) player IDs, used for linking stats across datasets. If we didn’t read the documentation, we might mistake them for random codes instead of unique player identifiers.

2)GmScMovingZ & GmScMovingZ2

These appear to be variations of “GmSc” (Game Score), but their exact calculation isn’t obvious. They likely represent standardized (Z-scored) versions of Game Score over different moving windows. If we didn’t check the documentation, we might misinterpret them as simple transformations of GmSc without realizing their statistical significance.

3)GmScMovingZTop2Delta

The meaning of “Top2Delta” isn’t immediately clear. It likely represents the difference between a player’s top two Game Scores in a given moving window. Without documentation, we might not understand that this metric is designed to highlight variability in peak performances.

Why was the data encoded this way? Using unique IDs (bbrID) helps with data merging across sources. Standardized (Z-score) metrics make it easier to compare performance across different seasons or players. The “Top2Delta” metric may be useful for identifying players with standout performances rather than just consistent averages.

.

Question 2:

One unclear element in the dataset, even after reviewing its structure, is:

“Date2” Column

It appears to be another date column, but it doesn’t match the main “Date” column. The number of unique values in “Date2” (1,427) is slightly higher than “Date” (1,375), which is unexpected. Some “Date2” values seem to belong to different years than the “Year” column suggests. Why is it unclear? The dataset does not explicitly explain what “Date2” represents. Is it an adjusted date, a reference to a different dataset, or a transformation of “Date”? Without documentation, it’s unclear why certain entries have a different “Date2” compared to “Date.” If “Date2” represents an alternate timeline (such as a reprocessed game log), there should be an explanation of how it was generated

Question3:

Visualization 1: Scatter Plot Comparing Date vs. Date2

This will show if there are any systematic shifts or inconsistencies between these two date columns. Visualization 2: Histogram of Differences Between Date and Date2

This will illustrate how frequently the dates differ and by how many days.

# Load necessary libraries
library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(lubridate)

## 
## Attaching package: 'lubridate'

## The following objects are masked from 'package:base':
## 
##     date, intersect, setdiff, union

library(ggplot2)

# Load the dataset
nba_data <- read.csv("C:/Statistics/nba.csv")

# Convert date columns to Date format
nba_data$Date <- as.Date(nba_data$Date, format="%Y-%m-%d")
nba_data$Date2 <- as.Date(nba_data$Date2, format="%Y-%m-%d")

# Calculate the difference in days between Date2 and Date
nba_data$Date_Diff <- as.numeric(difftime(nba_data$Date2, nba_data$Date, units = "days"))

# Scatter plot: Comparing Date vs. Date2
ggplot(nba_data, aes(x = Date, y = Date2)) +
  geom_point(alpha = 0.5) +
  labs(x = "Original Date", y = "Date2 (Potentially Adjusted)", 
       title = "Scatter Plot Comparing Date vs. Date2") +
  theme_minimal()

# Histogram of Date Differences
ggplot(nba_data, aes(x = Date_Diff)) +
  geom_histogram(binwidth = 5, fill = "blue", color = "blue", alpha = 0.7) +
  labs(x = "Date Difference (Days)", 
       y = "Frequency", 
       title = "Histogram of Date Differences Between Date and Date2") +
  theme_minimal()

#Findings from the Visualizations

Scatter Plot (Date vs. Date2)
The scatter plot reveals a strong positive correlation between Date and Date2, indicating that Date2 is likely an adjusted version of Date while maintaining a similar trend. Most data points align along a linear pattern, though some deviations suggest potential outliers or modifications. .

Histogram (Differences Between Date and Date2)
 The histogram of date differences further supports this observation, with a significant peak at zero, showing that many entries remain unchanged. However, the presence of both positive and negative variations suggests that some dates were shifted forward or backward, potentially due to data adjustments or inconsistencies. The distribution of differences highlights the need to investigate extreme cases where modifications are substantial.

Why is This a Concern?
If Date2 was intended to represent a correction of Date, but no documentation explains why the shift happens, users may misinterpret trends or inaccurately analyze player performance over time.
If the difference follows a non-random pattern (e.g., consistently shifted by X days), it could indicate a data-processing issue rather than a meaningful adjustment.

Potential Risks
Incorrect conclusions: Analysts might unknowingly compare mismatched dates, leading to flawed insights.
Misinterpretation of player performance: If game logs are not correctly aligned with seasons, career progression, or game context, it could distort analysis.

Possible Fixes
Verify whether the Date2 column is needed and, if so, document its purpose explicitly.
Investigate whether the shifts in Date2 are due to timezone adjustments, data merging inconsistencies, or errors in data scraping.
Use only one date column for analysis (preferably the one validated as correct)

Why is This a Concern?
The variations between Date and Date2, as seen in the scatter plot and histogram, raise concerns about data consistency and integrity. If Date2 is meant to represent a corrected or adjusted version of Date, discrepancies in the data could indicate errors, inconsistencies in data entry, or incorrect transformations. Such issues can impact any time-sensitive analyses, leading to misleading conclusions, especially in studies that rely on event chronology or trend detection.

Potential Risks
Data Integrity Issues – If Date2 was modified without clear documentation, it could introduce inaccuracies in historical records.
Incorrect Analysis Results – If the differences in dates are significant, analyses that rely on time-based trends, sequences, or intervals could be distorted.
Outlier Impact – Large deviations in the histogram suggest extreme cases where dates have been altered significantly, which could skew statistical models.
Predictive Model Errors – If a machine learning model relies on these dates for forecasting, inconsistencies could lead to incorrect predictions.
Possible Fixes
Investigate the Cause – Determine whether Date2 adjustments were intentional or due to data entry errors.
Check Source Data – Compare the original dataset with any preprocessing steps to identify where the modifications occurred.
Standardize Date Adjustments – If Date2 is meant to be corrected, ensure a consistent method was used for adjustments, documenting the logic behind changes.
Handle Outliers – Identify and review cases with extreme date differences to decide whether they should be corrected or excluded from analysis.
Improve Data Collection Processes – If these discrepancies originate from data entry errors, refining data collection and validation steps can prevent similar issues in the future.


Question4:

I'll analyze two categorical columns from the dataset—"Tm" (Team) and "Opp" (Opponent)—to check for:

Explicitly Missing Rows (Rows where values are explicitly missing, i.e., NaN).
        If missing_counts returns nonzero values, it means some rows are explicitly missing (NA values are           present in Tm or Opp).

Implicitly Missing Rows (Expected values that are missing but not marked as NaN).
       If missing_teams is not empty, it means some teams appear in Tm but never as Opp.
If missing_opponents is not empty, it means some teams appear in Opp but never as Tm.

Empty Groups (Categories that exist but contain no data).
```     If empty_groups is not empty, it means there are teams that are never listed as both a team (Tm) and an opponent (Opp).

# Load the dataset
nba_data <- read.csv("C:/Statistics/nba.csv")

# 1. Check for Explicitly Missing Rows (NA values)
colSums(is.na(nba_data[, c("Tm", "Opp")]))  # Count NA values in 'Tm' and 'Opp'

##  Tm Opp 
##   0   0

# 2. Check for Implicitly Missing Rows (Missing Expected Teams/Opponents)
unique_teams <- unique(nba_data$Tm)   # List all unique teams
unique_opponents <- unique(nba_data$Opp)  # List all unique opponents

print(unique_teams)

##  [1] "BOS" "DEN" "SAC" "ATL" "OKC" "MIA" "ORL" "NYK" "PHO" "MIL" "DAL" "NOP"
## [13] "WSB" "LAC" "SAS" "WAS" "GSW" "UTA" "BRK" "IND" "NJN" "PHI" "VAN" "POR"
## [25] "TOR" "HOU" "CHA" "DET" "CHI" "NOH" "MEM" "SEA" "MIN" "CHO" "LAL" "CLE"
## [37] "CHH" "NOK"

print(unique_opponents)

##  [1] "GSW" "DAL" "VAN" "DET" "CHO" "PHI" "WSB" "MEM" "BOS" "BRK" "NOP" "ATL"
## [13] "LAL" "CHH" "UTA" "OKC" "ORL" "SAC" "LAC" "CLE" "MIA" "CHI" "DEN" "PHO"
## [25] "SAS" "MIN" "TOR" "NYK" "WAS" "HOU" "MIL" "NOH" "NJN" "CHA" "POR" "SEA"
## [37] "IND" "NOK"

# 3. Check for Empty Groups (Teams Expected but Not Present in Both Columns)
all_teams <- union(unique_teams, unique_opponents)  # All unique teams
empty_groups <- setdiff(all_teams, intersect(unique_teams, unique_opponents)) # Teams in only one column
print(empty_groups)  # Teams that appear in only one of 'Tm' or 'Opp'

## character(0)

``` Question5: The “PTS” (Points Scored) column in the nba_data dataset represents the number of points scored by a team in a game. This is a continuous numerical variable, making it a strong candidate for outlier detection.

Outliers in this column can indicate anomalous performances, such as record-breaking high-scoring games or extremely low scores due to poor performance, key player absences, or strong defensive play. Additionally, identifying outliers helps in assessing scoring trends and game flow, as unusually high or low scores can reveal variations in game pace, overtime impacts, or strategic shifts. Moreover, detecting outliers in the PTS column is essential for ensuring data integrity, as abnormally low scores (e.g., 2 points) or unrealistically high scores (e.g., 300 points) could signal data entry errors or missing information.

Finally, outlier analysis in points scored is valuable for historical and statistical insights, allowing us to identify record-setting performances, analyze team consistency, and evaluate the factors contributing to extreme scoring variations in NBA games.