Handling Data Streams for Real-Time Analytics

# Handling Data Streams for Real-Time Analytics
# In today’s data-driven world, real-time data processing has become essential, especially with the rise of cloud computing and big data technologies. Unlike traditional data processing, where data is often stored and retrieved from local files, real-time analytics frequently involves handling data streams—data that needs to be read from or written to a continuous source. In R, we can handle data streams by creating file connections, which allows us to work with data sources beyond our local system, such as streaming data from an API or cloud storage. This essay covers techniques for establishing file connections in R, reading data in various formats, and writing processed results back to these connections.

# Establishing File Connections for Real-Time Data
# To handle real-time data, we start by establishing a file connection using the file() function. This function sets up a logical link to a data source, whether it’s a local file or an external input stream. For reading data, we specify "r" (read mode), and for writing data, we specify "w" (write mode). Here’s an example of setting up a connection for both local files and standard input:

# Loading the readxl library
library(readxl)

# Establishing a connection to read from an Excel file
sample_data <- read_excel("R Studio Projects 2024/Datasets/sample_data.xlsx")

# View the dataset to confirm it loaded successfully
View(sample_data)

# Process the dataset as needed. Here’s an example of displaying a summary of the dataset.
summary(sample_data)

##        ID          Name               Score     
##  Min.   :1.0   Length:3           Min.   :85.0  
##  1st Qu.:1.5   Class :character   1st Qu.:87.5  
##  Median :2.0   Mode  :character   Median :90.0  
##  Mean   :2.0                      Mean   :90.0  
##  3rd Qu.:2.5                      3rd Qu.:92.5  
##  Max.   :3.0                      Max.   :95.0

# Writing Data to a New Excel File
# After processing the data, we can write it to a new Excel file for future reference or analysis. The write.xlsx function from the openxlsx package is one option for this task.

#install.packages("writexl")

# Load the writexl library
library(writexl)

# Define the output path for the Excel file
output_path <- "R Studio Projects 2024/Datasets/processed_sample_data.xlsx"

# Write the sample data to an Excel file
write_xlsx(sample_data, output_path)

# Confirm the file creation
cat("Data has been written to:", output_path, "\n")

## Data has been written to: R Studio Projects 2024/Datasets/processed_sample_data.xlsx

# Load necessary libraries
library(readxl)
library(writexl)
library(knitr)
library(kableExtra)

# Define the path to save the processed data
output_path <- "R Studio Projects 2024/Datasets/processed_sample_data.xlsx"

# Write the processed data to a new Excel file
write_xlsx(sample_data, output_path)

# Summary Table of File Connection Functions
# To summarize, here’s a table of the primary functions used for establishing, reading, and writing file connections in R.

# Create the summary table with descriptions and examples
connection_functions_table <- data.frame(
  Function = c("read_excel", "View", "summary", "write_xlsx"),
  Description = c("Reads data from an Excel file", 
                  "Displays the data in RStudio Viewer", 
                  "Provides a summary of the dataset", 
                  "Writes processed data to an Excel file")
)

# Display the table with colors and styling
kable(connection_functions_table, "html", col.names = c("Function", "Description")) %>%
  kable_styling(full_width = F, bootstrap_options = c("striped", "hover", "condensed")) %>%
  column_spec(1, bold = TRUE, color = "white", background = "#4CAF50") %>%
  column_spec(2, background = "#E0F7FA")

Function	Description
read_excel	Reads data from an Excel file
View	Displays the data in RStudio Viewer
summary	Provides a summary of the dataset
write_xlsx	Writes processed data to an Excel file

# This table provides a quick reference to key Excel-related functions in R, making it easier to read, view, summarize, and write Excel data efficiently for real-time data analytics.