1. Introduction

In today’s interconnected world, cyber security remains a big concern. As the digital footprint increases in every sector, malware infections and phishing attacks have become more common and sophisticated. Risks are plaguing organisations, who face the possibility of data breaches, financial loss and reputation damage.

This project performs Exploratory Data Analysis (EDA) on two cybersecurity datasets related to two problems: the first being to classify malware and the other being to detect phishing websites. We then use statistical methods, visualization techniques and clustering algorithms to derive insights into how we can support better detection strategies and better threat mitigation.

# Load map data and visualize major cities
data(world.cities)
leaflet(data = world.cities %>% filter(pop > 1000000)) %>%
  addTiles() %>%
  addCircles(~long, ~lat, popup = ~name, radius = 50000, color = 'red')

2. Literature Review

Data driven approaches have become the new workhorse for cyber security research, as cyber security researchers have endeavored to understand and mitigate cyber threats. Static and dynamic analysis, behavioral profiling, and machine learning classification models have all been used for malware detection. In their paper,present a detailed overview of all the data mining techniques that can be applied for malware detection focusing on the two major ones: pattern recognition and anomaly detection (Ye et al., 2017).

Phishing detection has also moved forward considerably.It Present the efficacy of URL based features, domain analysis, and content evaluation to detect a phishing attack (Sahoo et al., 2017) . Furthermore, recent threat reports indicate that phishing attacks are growing more sophisticated and are able to evade classical rulesets, and more, now often with special focus on mobile users and during a pandemic.

Cyber security still relies on the integration on data analytics and machine learning to detect threats in real time and to automate response and create security intelligence.

Key Reports and Studies:

Machine learning in malware detection (Ye et al., 2017).

Phishing detection using URL and content features by (Sahoo et al., 2017).

Threat report covering malware trends (Symantec, 2020).

The growth and techniques of phishing, (APWG, 2023).

3. Methods

Data Used

Two datasets were analyzed:

Malware Dataset: It consists of varied system level metrics such as CPU time, memory usage, and process states and contains over 100,000 rows. Classes are that malware or benign.

Phishing Dataset: More than 11,000 instances with over 30 features derived from website content, URLs, and server behaviors. Result is what we are after to find if it’s phishing or a legit website.

Representative of realistic cyber scenarios, these datasets are structured, to allow exploration of their structural properties, and pose patterns which help on threat classification.

Sources:
Malware: Kaggle
Phishing: UCI ML Repository

# Load and preprocess phishing dataset
phishing <- foreign::read.arff("Training Dataset.arff") %>%
  as_tibble()

# Rename columns
colnames(phishing) <- paste0("Var", 1:ncol(phishing))
colnames(phishing)[ncol(phishing)] <- "Result"
phishing$Result <- factor(phishing$Result)

# Remove near-zero variance predictors
nzv <- caret::nearZeroVar(phishing, saveMetrics = TRUE)
phishing <- phishing[, !nzv$nzv]

# Select only numeric columns
phishing_numeric <- phishing %>%
  select(where(is.numeric), -Result)

# Remove constant columns and missing values
phishing_numeric <- phishing_numeric[, apply(phishing_numeric, 2, var) != 0]
phishing_numeric <- na.omit(phishing_numeric)

# Load malware dataset
malware <- read_csv("Malware dataset.csv")
## Rows: 100000 Columns: 35
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (2): hash, classification
## dbl (33): millisecond, state, usage_counter, prio, static_prio, normal_prio,...
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
# Ensure 'classification' is a factor
malware$classification <- as.factor(malware$classification)

4. Results

Malware Class Distribution

ggplot(malware, aes(x = classification)) +
  geom_bar(fill = 'steelblue') +
  theme_minimal() +
  labs(title = "Malware Class Distribution", x = "Class", y = "Count")

We see from the class distribution that benign files dominate at orders of magnitude relative to number of malware which has further repercussions when it comes to training and evaluation of the models, especially in binary classification scenarios.

Malware Correlation Heatmap

malware_numeric <- malware %>%
  select(where(is.numeric)) %>%
  select(1:min(15, ncol(.)))  # Use up to 15 numeric features for clarity

corr_matrix <- cor(malware_numeric, use = "complete.obs")
## Warning in cor(malware_numeric, use = "complete.obs"): the standard deviation
## is zero
corrplot(corr_matrix, method = "color", type = "lower", tl.cex = 0.6)

Linear relationships among some of the system-level indicators (utime, stime, and memory consumption) are also hinted by this heatmap. In predictive modeling, you should work around such multicollinearity to overcome redundancy.

Top Malware-Affected Countries (Simulated Data)

malware_countries <- data.frame(
  country = c("United States", "India", "Russia", "China", "Brazil", "Germany", "Nigeria"),
  lat = c(37.6, 20.6, 55.7, 35.9, -14.2, 51.2, 9.1),
  lon = c(-95.7, 78.9, 37.6, 104.2, -51.9, 10.4, 8.7),
  infections = c(90000, 75000, 65000, 62000, 50000, 48000, 46000)
)

leaflet(data = malware_countries) %>%
  addTiles() %>%
  addCircleMarkers(~lon, ~lat, radius = ~infections/15000,
                   popup = ~paste(country, ":", infections, "cases"),
                   color = "red", fillOpacity = 0.7)

The countries with the largest numbers of malware infections are noted on this map. In each map, the size of the red circle represents its proportional volume of infections, where a larger circle corresponds to more infections. First, this visualization gives a holistic view of malware prevalence around the world, scheduling countries like the United States, India, and Russia as having a lot of activity. It’s a useful tool for helping to determine global malware hotspots and to inform threat intelligence priorities.

Global Phishing Hotspots (Simulated Data)

phishing_countries <- data.frame(
  country = c("United Kingdom", "South Africa", "Canada", "Australia", "Philippines"),
  lat = c(55.3, -30.6, 56.1, -25.3, 13.4),
  lon = c(-3.4, 22.9, -106.3, 133.8, 122.9),
  phishing_reports = c(22000, 18000, 15000, 12000, 10000)
)

leaflet(data = phishing_countries) %>%
  addTiles() %>%
  addCircleMarkers(~lon, ~lat, radius = ~phishing_reports / 4000,
                   popup = ~paste(country, ":", phishing_reports, "reports"),
                   color = "blue", fillOpacity = 0.6)

It shows regions with more than average number of phishing reports. Reported cases are shown as blue circle markers where circles are large for high incident counts. The map has prominent countries as the United Kingdom, South Africa, and Canada. With evaluation to this visualization comes the understanding of where there is a geographical spread of phishing attacks, and which regions are vulnerable where awareness and training could be implemented.

Malware vs Phishing Activity Comparison (Bubble Overlay)

combined_data <- data.frame(
  country = c("USA", "India", "China", "Brazil", "UK"),
  lat = c(37.6, 20.6, 35.9, -14.2, 55.3),
  lon = c(-95.7, 78.9, 104.2, -51.9, -3.4),
  malware = c(90000, 75000, 62000, 50000, 30000),
  phishing = c(30000, 20000, 15000, 10000, 22000)
)

leaflet(data = combined_data) %>%
  addTiles() %>%
  addCircleMarkers(~lon, ~lat, radius = ~malware/20000,
                   popup = ~paste("Malware:", malware),
                   color = "red", fillOpacity = 0.5, group = "Malware") %>%
  addCircleMarkers(~lon, ~lat, radius = ~phishing/5000,
                   popup = ~paste("Phishing:", phishing),
                   color = "blue", fillOpacity = 0.4, group = "Phishing") %>%
  addLayersControl(overlayGroups = c("Malware", "Phishing"), options = layersControlOptions(collapsed = FALSE))

This comparative map visualizes malware infections (red) and phishing incidents (blue) across some selected countries in a dual layered form. This map is able to compare attack patterns by region by overlaying both threat types on the same geographic coordinates. For example, the United States and India are major players in malware and phishing activity. Layered control lets you turn views on/ off so that you can focus on the threats that worry you the most.

K-means Clustering on Phishing Data

dim(phishing_numeric)
## [1] 11055     0
summary(phishing_numeric)
## < table of extent 0 x 0 >
distinct_rows <- nrow(unique(phishing_numeric))

if (distinct_rows >= 2) {
  set.seed(123)
  kmeans_result <- kmeans(phishing_numeric, centers = 2, nstart = 25)
  print(kmeans_result)
} else {
  cat("⚠️ Not enough distinct data points for clustering.\n")
}
## ⚠️ Not enough distinct data points for clustering.

To determine the natural groupings of phishing data points, we applied K-means clustering. The results confirm the formation of phishing websites into distinct clusters according to their attributes, validating the dataset can be used for supervised classification.

5. Discussion

Analysis shows that behavioral and system level features are also important for threat identification. Detailed process level indicators greatly aid in malware classification and phishing can be probed for by characteristics of URL length, domain age, and presence of HTTPS.

Challenges identified include:

Malware detection problem may involve class imbalance problem which may require SMOTE or cost sensitive learning.

This has implications of high dimensional which motivates the need for techniques like PCA for dimensional reduction.

No temporal context provided (e.g., we see output for each sample as an isolation; no output for x_i-x_(i+1), which could have been solved through sequential modelling , such as LSTM for malware behavior over time).

6. Summary

We cleaned and visualized real world cyber security datasets and extracted useful insights using data cleaning, data visualization, and clustering. Results show the usefulness of structured data to aid in anomaly identification and intelligent threat detection system design.

Feature clusters that can be used for predictive modeling were seen in both datasets. The features indicating malicious behavior were decomposed into malware features like CPU usage, memory patterns, and phishing features showed clear clusters of features correlating to fraud.

On this premise, we build machine learning pipelines for automating the detection of cyber threats.

7. Future Work

Future directions may include:

Supervised learning models (Random Forest, SVM, Neural Networks) were applied to assess predictive accuracy.

Temporal features characterizing time-series behavior for malware.

Building dashboards to visualize real time threat activity using Shiny or Dash.

Domain-based threat intelligence feeds enhancing feature engineering.

References

Ye, Y., Li, T., Adjeroh, D., & Iyengar, S. S. (2017). A survey on malware detection using data mining techniques. Information Sciences, 231, 1–12. https://doi.org/10.1016/j.ins.2017.03.037

Sahoo, D., Liu, C., & Hoi, S. C. H. (2017). Malicious URL Detection using Machine Learning: A Survey. ACM Computing Surveys, 51(2), 1–39. https://doi.org/10.1145/3131365

Symantec. (2020). Internet Security Threat Report. https://symantec-enterprise-blogs.security.com

APWG. (2023). Phishing Activity Trends Report. https://apwg.org/trendsreports/

Malware Dataset: https://www.kaggle.com/datasets/sid321axn/malware-detection-dataset

Phishing Dataset: https://archive.ics.uci.edu/ml/datasets/phishing+websites