Lab 05: Data Engineering & Lifecycle

🧠 Objective

Transform and version raw transactional data into a reproducible pipeline using lifecycle principles.

📥 Load Data

library(readr)
transactions <- read_csv("data/pipeline_input.csv")

Rows: 8 Columns: 7
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (5): transaction_id, user_id, currency, status, region
dbl  (1): amount
date (1): date

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

🧼 Stage 1: Cleaning

library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

cleaned <- transactions %>%
  filter(!is.na(amount), status == "Complete")

📊 Stage 2: Aggregation

regional_totals <- cleaned %>%
  group_by(region) %>%
  summarise(total = sum(amount))

##📝 Stage 3: Lifecycle Reflection ## Describe each stage: input → transformation → output ## Versioning options using Quarto or targets

📈 Optional Visualization

library(ggplot2)
ggplot(regional_totals, aes(x = region, y = total)) +
  geom_col(fill = "darkorange") +
  theme_minimal()