Master Index: Introduction to Data Analytics (CE KMITL)

Welcome to the comprehensive study guide series for Introduction to Data Analytics, Department of Computer Engineering, Faculty of Engineering, King Mongkut’s Institute of Technology Ladkrabang (CE KMITL).

This repository provides rigorous, token-efficient Markdown lecture notes synthesized directly from the course slide curriculum (materials/). All progressive animation sequences have been deduplicated into unified reference tables and worked examples, mathematical expressions are standardized in KaTeX/LaTeX, and process lifecycles are visualized with Mermaid diagrams.

1. Important Note: Dedicated Personal Study Note

In addition to the official lecture curriculum, review the standalone foundational note on model validation:

📘 K-Fold Cross-Validation
A dedicated personal guide covering the mathematical mechanics, fold partitioning logic, and variance reduction properties of $K$ -fold cross-validation.

Direct Application in Course: Crucial for model validation in Lecture 04: AI & ML Foundations and performance metric reliability in Lecture 07: Classification Evaluation.

2. Master Curriculum Table

The course curriculum systematically covers the complete end-to-end data analytics lifecycle across 7 primary lecture modules:

Lecture #	Module Title & Study Guide Link	Core Topics & Key Algorithms	Source Material Reference
01	01. Introduction to Data Analytics	• DIKW Hierarchy (Data, Information, Knowledge, Wisdom) • 4 Analytics Types: Descriptive, Diagnostic, Predictive, Prescriptive • CRISP-DM Lifecycle (Business Understanding to Deployment) • Data Roles, Tools (Python, SQL, BI), and Ethics	`materials/DA01 - Introduction.pdf`
02	02. Data Processing & Cleansing	• Data Quality Issues (Missing values, Noise, Redundancy) • Outlier Detection via Interquartile Range (IQR) Fences • Data Transformation: Min-Max Normalization, Z-Score Standardization • Feature Encoding: One-Hot Encoding, Label Encoding, Discretization	`materials/DA04 - Data Processing.pdf`
03	03. Data Exploration & Visualization	• 5-Number Summary: Min, $Q_1$ , Median ( $Q_2$ ), $Q_3$ , Max • Distribution Analysis: Histograms, Box Plots, Density Plots • Bivariate Relationships: Scatter Plots, Pearson Correlation Heatmaps • Matplotlib & Seaborn Visualization Workflows	`materials/DA05 - Data Exploration.pdf`
04	04. AI & Machine Learning Foundations	• AI vs. Machine Learning vs. Deep Learning Taxonomy • Supervised vs. Unsupervised vs. Reinforcement Paradigms • Simple & Multiple Linear Regression: $y = \mathbf{w}^T \mathbf{x} + b$ • Loss Functions: MAE, MSE, RMSE, $R^2$ Score • Optimization via Gradient Descent: $w := w - \alpha \frac{\partial J}{\partial w}$	`materials/DA06-07 AI-ML-Intro.pdf`
05	05. Decision Tree Classification	• Classification Concept & Tree Anatomy (Root, Decision, Leaf Nodes) • Shannon Entropy: $H(S) = -\sum p_i \log_2(p_i)$ • Conditional Entropy: $H(S \mid A)$ & Information Gain: $IG(S, A)$ • Complete 7-Sample Medical Diagnosis (Flu) Calculation Walkthrough • Splitting Criteria & Overfitting Prevention (Pruning)	`materials/DA07.1 Decision Tree.pdf`
06	06. Cluster Analysis (Unsupervised Learning)	• Unsupervised Clustering: Intra-cluster vs. Inter-cluster Distances • Distance Metrics: Euclidean ( $L_2$ ) and Manhattan ( $L_1$ ) Norms • K-Means Algorithm (Lloyd’s), Objective SSE, 1D 3-Round Iteration Table • Determining $K$ : The Elbow Method and Silhouette Coefficient ( $s(i)$ ) • DBSCAN: Density parameters ( $\varepsilon, \text{MinPts}$ ), Core/Border/Noise • Dimensionality Reduction: SVD and Principal Component Analysis (PCA)	`materials/DA08 - Clustering.pdf`
07	07. Classification Evaluation & Performance Metrics	• Train-Test Split & Validation Strategy • $2 \times 2$ Confusion Matrix: $TP, TN, FP, FN$ & 8-Scenario Evaluation Drill • Class Imbalance Paradox & The “We Need FISH” Intuition • Performance Metrics: Accuracy, Precision, Recall/TPR, Specificity/TNR, $F_1$ -Score • Consolidated 5-Exercise Evaluation Table • Receiver Operating Characteristic (ROC Curve) & Area Under Curve (AUC) • Multi-Class Evaluation: Macro, Micro, and Weighted Averaging	`materials/AI-Classification-Evaluation.pdf`

3. Course Concept Flow & Knowledge Graph

The diagram below illustrates the hierarchical dependencies and sequential learning path through the curriculum:

flowchart TD subgraph Phase1["Phase 1: Analytics Foundations & Preprocessing"] DA01["01. Introduction to Data Analytics\n(DIKW, 4 Analytics Types, CRISP-DM)"] DA04["02. Data Processing & Cleansing\n(Missing Values, IQR Outliers, Scaling)"] DA05["03. Data Exploration & Visualization\n(5-Number Summary, Boxplots, Heatmaps)"] DA01 --> DA04 --> DA05 end subgraph Phase2["Phase 2: Supervised Learning & Predictive Modeling"] DA06["04. AI & Machine Learning Foundations\n(Linear Regression, Cost Functions, Gradient Descent)"] DA07["05. Decision Tree Classification\n(Entropy, Information Gain, ID3 Flu Walkthrough)"] DA05 --> DA06 DA06 --> DA07 end subgraph Phase3["Phase 3: Unsupervised Pattern Discovery"] DA08["06. Cluster Analysis\n(K-Means SSE, DBSCAN Density, PCA)"] DA05 --> DA08 end subgraph Phase4["Phase 4: Model Evaluation & Validation"] EVAL["07. Classification Evaluation\n(Confusion Matrix, F1-Score, ROC/AUC)"] KFOLD["Dedicated Personal Note:\nK-Fold Cross-Validation\n(/notes/k-fold-cross-validation)"] DA06 --> EVAL DA07 --> EVAL EVAL -.->|Validation Methodology| KFOLD end DA08 -.->|Unsupervised Validation| EVAL

4. Course-Wide Quick Formula Reference Cheat Sheet

For fast exam review and implementation reference, key mathematical equations across all modules are indexed below:

Domain	Equation Name	Standard LaTeX Formula	Primary Module
Data Cleaning	IQR & Outlier Bounds	$\text{IQR} = Q_3 - Q_1$ $\text{Fences} = [Q_1 - 1.5 \cdot \text{IQR}, \; Q_3 + 1.5 \cdot \text{IQR}]$	Lecture 02
Data Scaling	Min-Max Scaling	$x_{\text{scaled}} = \frac{x - x_{\min}}{x_{\max} - x_{\min}}$	Lecture 02
Data Scaling	Z-Score Standardization	$z = \frac{x - \mu}{\sigma}$	Lecture 02
Regression Loss	Mean Absolute Error (MAE)	$\text{MAE} = \frac{1}{n} \sum_{i=1}^n \|y_i - \hat{y}_i\|$	Lecture 04
Regression Loss	Mean Squared Error (MSE)	$\text{MSE} = \frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2$	Lecture 04
Regression Loss	Root Mean Squared Error	$\text{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2}$	Lecture 04
Optimization	Gradient Descent Update	$w_j := w_j - \alpha \frac{\partial J(\mathbf{w})}{\partial w_j}$	Lecture 04
Information Theory	Shannon Entropy	$H(S) = - \sum_{i=1}^c p_i \log_2(p_i)$	Lecture 05
Information Theory	Conditional Entropy	$H(S \mid A) = \sum_{v \in \text{Values}(A)} \frac{\|S_v\|}{\|S\|} H(S_v)$	Lecture 05
Information Theory	Information Gain (IG)	$\text{IG}(S, A) = H(S) - H(S \mid A)$	Lecture 05
Distance Metric	Euclidean Distance ( $L_2$ )	$d(\mathbf{x}, \mathbf{y}) = \sqrt{\sum_{i=1}^d (x_i - y_i)^2}$	Lecture 06
Distance Metric	Manhattan Distance ( $L_1$ )	$d(\mathbf{x}, \mathbf{y}) = \sum_{i=1}^d \|x_i - y_i\|$	Lecture 06
Clustering Cost	Sum of Squared Errors (SSE)	$\text{SSE} = \sum_{i=1}^K \sum_{\mathbf{x} \in C_i} \|\mathbf{x} - \mathbf{c}_i\|_2^2$	Lecture 06
Cluster Validation	Silhouette Coefficient	$s(i) = \frac{b(i) - a(i)}{\max(a(i), b(i))}, \quad s(i) \in [-1, +1]$	Lecture 06
Evaluation	Accuracy	$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$	Lecture 07
Evaluation	Precision (PPV)	$\text{Precision} = \frac{TP}{TP + FP}$	Lecture 07
Evaluation	Recall / Sensitivity / TPR	$\text{Recall} = \frac{TP}{TP + FN}$	Lecture 07
Evaluation	Specificity / TNR	$\text{Specificity} = \frac{TN}{TN + FP}$	Lecture 07
Evaluation	False Positive Rate (FPR)	$\text{FPR} = \frac{FP}{FP + TN} = 1 - \text{Specificity}$	Lecture 07
Evaluation	$F_1$ -Score ( $F$ -Measure)	$F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2 TP}{2 TP + FP + FN}$	Lecture 07

5. Verification and Integrity Assurance

Deliverable integrity across this study suite is verified automatically by running the project verification harness:

# Execute the comprehensive automated test suite from the repository root:
python3 validate_notes.py

The test runner validates: