Course Details

• INSTRUCTOR: A.Dr. Robert P. Batzinger

• COURSE DESCRIPTION: Fundamental data mining concepts, data mining technique, followed by more advanced concepts and algorithms. (R-based)

• VENUE: PC301 Mon / Thu 9-12

• MIDTERM: 2 Jul Jun 2026 TIME 10:00 - 12:00

• FINAL: 24 Jul 2026 TIME 13:00 - 16:00 (C)

• There will be a 1-hr lab test where student will be expected to create an accurate assessment of a dataset and its potential

Schedule

• June

• July

Code Assessment

• Ignite Code Presentation: 5-minute with 20 slides that automatically advance class explaining key aspects of the program:
  • Describe what the program was designed to do
  • Decsribe the algorithm and key implementation choices
  • Highlight of the software
  • Describe any novel approaches taken
  • Sample input and output
  • Current limitations of the software
• A dynamic research notebook to provide the background information for a research paper
  • Introduction: Capture the rationale, key goals and objectives of the project
  • Methodology: Descriptions and citations about the data sources and methodologies used, research steps and working code broken into documented fragments
    
  • Results: Comparison of research outcomes to expectation
  • Discussion: Intepretation of the results and possible future research directions

Red Flags for AI-Generated Code

Automatic point deductions may apply for:

• Code that student cannot explain or modify
• Sophisticated techniques not covered in class without explanation
• Inconsistent coding style within the same assignment
• Comments that don’t match the actual code
• Over-engineering for simple problems
• Generic variable names throughout (e.g., data, result, output)
• Perfect code with no evidence of iteration or debugging
• Ignite presentation that goes over 5min or fails to cover the essentials

Ideal Software

Software Installation

• R 4.6.0
  • Download: https://cran.r-project.org/bin/
• RStudio
  • Download: https://posit.co/download/rstudio-desktop

Definitions:

• Data: Factual information (such as measurements, statistics, or observations) used as a basis for reasoning, discussion, or calculation.

• Fact: A piece of information presented as having objective reality; an event or item of information that can be verified through objective evidence, empirical observation, or rigorous proof.

• Opinion: A view, judgment, or appraisal formed in the mind about a particular matter; a belief stronger than impression but less strong than positive knowledge, which cannot be conclusively proven or verified by objective evidence.

• Belief: a perception that something seems true, genuine, or real, often on the basis of emotional conviction, authority, or faith, rather than on absolute empirical proof or demonstration.

• Hypothesis: A tentative, testable assumption or proposition advanced to explain a phenomenon or relationship. It serves as a starting point for further investigation, experimentation, or data collection, and has not yet been thoroughly proven.

• Theory: A well-substantiated, structurally sound explanation of some aspect of the natural or digital world. It incorporates facts, laws, inferences, and tested hypotheses, and is widely accepted within a field because it reliably predicts outcomes and survives repeated testing.

• Truth The property of being in accord with fact or reality. In logic and philosophy, a statement is considered a truth if it accurately describes the actual state of affairs or conforms to an established, verifiable reality.

Discussion of conceptions

• Truth
• Insight
• Information
• Data
• Mistake (glitch)
• Error
• Variation
• Outlier
• Falsehood
• Misrepresentation

Key Disciplines

• Data Mining: The process of discovering patterns, insights, and anomalies from large datasets. The information extracted can be used for dataset development, decision-making, prediction, and/or understanding.

• Data Science: An interdisciplinary field that combines statistical methods, data manipulation and analysis and domain expertise to extract knowledge and insights from data. Data scientists are involved in the entire data lifecycle, from collection and cleaning to analysis, modeling, and communication of results.

• Data Analytics: A multidisciplinary discipline focused on analyzing raw data to extract meaningful insights, identify patterns, and draw actionable conclusions. It combines elements of mathematics, statistics, computer programming, and business intelligence to transform disorganized, complex data streams into structured information that guides strategic decision-making.

• Artificial Intelligence (AI): A broad field of computer science that focuses on creating intelligent agents, to assess the situation and take actions to advance towards achieving defined goals. The goal of AI is to enable machines to simulate human intelligence.

• Machine Learning (ML): A subfield of Artificial Intelligence that focuses on developing algorithms that allow computers to “learn” from data without being explicitly programmed. Instead of following rigid rules, ML models learn patterns and make predictions or decisions based on the data they are trained on. AI can be either supervised, unsupervised, and reinforcement learning.

• Deep Learning (DL): A subfield of Machine Learning that uses artificial neural networks with multiple layers. Deep learning excels at learning complex patterns from large datasets, used in areas like image recognition, speech recognition, and natural language processing.

Similar and Related Terms:

• Big Data: Refers to extremely large datasets that may be analyzed computationally to reveal subtle patterns, trends, and associations, especially relating to human behavior and interactions. These datasets often exceed the size and speed of memory.

• Business Intelligence (BI): A set of strategies and technologies used for the data analysis of business information to provide actionable insights that help organizations make better business decisions. Reports and dashboards used to focused on descriptive analytics (what happened) rather than predictive or prescriptive analytics.

• Natural Language Processing (NLP): A subfield of AI that focuses on enabling computers to understand, interpret, and generate human language. NLP techniques are used in applications like spam detection, sentiment analysis, machine translation, and chatbots.

• Computer Vision: A subfield of AI that enables computers to “see” and interpret visual information such as images and videos. Applications include facial recognition, object detection, autonomous vehicles, and medical image analysis.

• Predictive Analytics: A type of data analytics that uses historical data, statistical algorithms, and machine learning techniques to identify the likelihood of future outcomes based on historical data. Its goal is to predict what will happen.

• Prescriptive Analytics: The next step beyond predictive analytics, focusing on determining the best course of action for a given situation. It not only predicts what will happen but also suggests why it will happen and recommends actions to take to achieve a desired outcome or mitigate a risk.

• Reinforcement Learning (RL): A type of machine learning where an agent learns to make decisions by performing actions in an environment and receiving rewards or penalties. The goal is to maximize the cumulative reward over time. It’s often used in robotics, game playing (like AlphaGo), and autonomous systems.

• Data Engineering: The aspect of data science that focuses on the practical applications of data collection, storage, processing, and transformation. Data engineers build and maintain data pipelines, databases, and data warehouses to allow convenient access and efficient use.

• Ethics in AI/Data Science: Moral implications, societal impact, responsible development and deployment of AI and data-driven systems in attempt to address concerns like bias, fairness, privacy, transparency, and accountability.

Measuring errors

• Variance: The range values in a normal distribution

• Overfitting: A modeling error that occurs when an algorithm captures the noise or random fluctuations in the training dataset rather than the underlying data distribution. An overfitted model performs exceptionally well on the training data but fails to generalize to new, unseen testing data.

Confusion Matrix:

A tabular layout used to visualize and evaluate the performance of a supervised learning algorithm.

It maps actual class labels against predicted class labels, breaking down results into:

Market Basket Transaction Associations

• Association Rule: An implication expression of the form \(X \rightarrow Y\), where \(X\) and \(Y\) are disjoint itemsets. It signifies that if the items in set \(X\) are present in a transaction, the items in set \(Y\) are also likely to be present.

• Support: A metric that measures how frequently an itemset appears in the entire database. Mathematically, for a rule \(X \rightarrow Y\):

\[\text{Support}(X \rightarrow Y) = \frac{\text{Number of transactions containing both } X \text{ and } Y}{\text{Total number of transactions}}\]

• Confidence: A metric that measures how often items in \(Y\) appear in transactions that already contain \(X\). It assesses the reliability of the inference made by the rule:

\[\text{Confidence}(X \rightarrow Y) = \frac{\text{Support}(X \cup Y)}{\text{Support}(X)}\]

• Lift: A metric used to measure the strength of an association rule by comparing the co-occurrence of \(X\) and \(Y\) against what would be expected if they were completely independent.

\[\text{Lift}(X \rightarrow Y) = \frac{\text{Confidence}(X \rightarrow Y)}{\text{Support}(Y)}\]

• A lift value \(> 1\) indicates that \(X\) and \(Y\) are positively correlated, meaning the presence of \(X\) significantly increases the likelihood of \(Y\) occurring.

Differences

More differences

[]

Software Installation

• R 4.6.0
  • Download: https://cran.r-project.org/bin/
• RStudio
  • Download: https://posit.co/download/rstudio-desktop

R Notebook