| wflow_id | .config | rank | Avg_Precision | F1_Score | PR_AUC | ROC_AUC |
|---|---|---|---|---|---|---|
| recip_xgboost | Preprocessor1_Model04 | 1 | 0.9838416 | 0.9551489 | 0.9838027 | 0.9953246 |
| recip_xgboost | Preprocessor1_Model13 | 2 | 0.9838328 | 0.9557932 | 0.9837956 | 0.9953189 |
| recip_ranger | Preprocessor1_Model16 | 3 | 0.9825481 | 0.9496355 | 0.9827583 | 0.9950036 |
| recip_ranger | Preprocessor1_Model01 | 4 | 0.9830797 | 0.9527746 | 0.9839425 | 0.9950032 |
| recip_ranger | Preprocessor1_Model14 | 5 | 0.9825972 | 0.9479444 | 0.9825780 | 0.9949584 |
| recip_log_reg | Preprocessor1_Model20 | 6 | 0.9322815 | 0.9188279 | 0.9319274 | 0.9767676 |
Introduction
With the rise of wearable fitness technology, tracking personal activity has become a significant trend. However, most tracking focuses on the quantity of activity rather than its quality. This project aims to address that by analyzing accelerometer data collected from various body parts while individuals performed barbell lifts. The goal is to predict the manner in which they executed the exercise using machine learning techniques.
Data Description
The dataset consists of sensor readings from accelerometers placed on the belt, forearm, arm, and dumbbell of six participants. The target variable, “classe,” indicates how correctly the exercise was performed, with five possible classes:
- Class A: Correct execution
- Class B–E: Various types of incorrect execution
The dataset contains multiple numerical and categorical variables, including raw sensor readings and derived metrics.
| Descriptions | Value |
|---|---|
| Sample size (nrow) | 19622 |
| No. of variables (ncol) | 159 |
| No. of numeric/interger variables | 155 |
| No. of factor variables | 0 |
| No. of text variables | 3 |
| No. of logical variables | 0 |
| No. of identifier variables | 0 |
| No. of date variables | 1 |
| No. of zero variance variables (uniform) | 3 |
| %. of variables having complete cases | 37.11% (59) |
| %. of variables having >0% and <50% missing cases | 0% (0) |
| %. of variables having >=50% and <90% missing cases | 0% (0) |
| %. of variables having >=90% missing cases | 62.89% (100) |
Summary of Dataset
Data Preprocessing
Data Cleaning
- Removing Missing Values: Many columns contained a high percentage of missing values, which were removed to prevent bias.
- Eliminating Irrelevant Features: Columns such as timestamps and user names were excluded as they do not contribute to classification.
- Normalization and Scaling: Sensor readings were normalized to ensure equal weighting across features.
- Partitioning the Data: The dataset was split into training (80%) and testing (20%) sets to evaluate model performance properly.
Exploratory Data Analysis (EDA)
Data Distribution
Boxplots and histograms revealed that some features exhibited significant variance across different classes.
Feature Correlation
A correlation matrix was computed to identify relationships between features. Strongly correlated features were retained, while redundant ones were removed.
The correlation matrix reveals strong relationships among motion sensor variables, particularly within the same body region. For instance, roll, yaw, and total acceleration of the belt exhibit high correlations (e.g., roll_belt & total_accel_belt: 0.98), indicating synchronized movement patterns. Similarly, acceleration and magnetometer readings, especially in the belt and arm regions, show strong dependencies. Gyroscope readings within the same body part also tend to be highly correlated, reflecting consistent rotational behavior.
Notably, inverse relationships exist, such as roll and acceleration along the belt’s Z-axis (-0.99), suggesting that as roll increases, vertical acceleration decreases. Another example is pitch_belt’s strong negative correlation with accel_belt_x (-0.97), implying opposing movement tendencies.
On the other hand, variables from different body parts, such as the dumbbell and forearm, generally show low correlations, suggesting independent motion between these segments. Some gyroscope and magnetometer readings also exhibit weak relationships, indicating that rotation does not always influence all sensor measurements equally. Overall, the data suggests that body movements are highly interconnected within the same region, while different body parts move more independently. The presence of strong negative correlations may indicate compensatory motion, where one movement counterbalances another.
Building on the previous correlation insights, the correlation funnel analysis provides deeper clarity on how different sensor variables relate to the target variable “classe.” The strongest correlation is seen with class A (1.0), indicating a clear separation in movement characteristics. Other classes (B, C, D, E) show moderate negative correlations (-0.31 to -0.28), suggesting varying but less distinct movement patterns.
Specific sensor bins reveal key movement dynamics. Higher values of magnet_arm_x (>289) show negative correlations with “classe,” while lower values (<-300) exhibit positive correlations, indicating that magnetic field intensity around the arm differentiates movements. Pitch_forearm follows a similar trend: lower values (<0) correlate positively, whereas higher values (>28.4) correlate negatively. This suggests that forearm pitch may be more stabilized or constrained in certain activities.
Acceleration and gyroscope data provide further insights. High acceleration along the arm’s X-axis (<-242) positively correlates with “classe,” while mid-range values (-44 to 84) show negative correlations, implying different exertion levels across activities.
Similarly, gyroscope readings in the forearm and dumbbell exhibit positive correlations at lower ranges but negative correlations at higher ranges, suggesting rotational motion difference. Window-based segmentation (num_window) highlights how movement frequency affects classification. Mid-range windows (222-424) correlate positively, while higher window values (>644) show negative correlations, suggesting that controlled, repetitive motions may be more indicative of certain classes.
Overall, the correlation funnel refines previous insights by revealing how specific sensor thresholds influence classification. Movements involving arm magnetism, forearm pitch, acceleration extremes, and gyroscopic patterns play a crucial role in distinguishing activity classes. These findings suggest that class differentiation is driven by distinct biomechanical signatures rather than uniform motion across body segments.
In this data science project, the top 10 variables with the highest correlation to the target variable were identified using a correlation funnel, binned, and one-hot encoded. The dataset was then processed through UMAP for dimensionality reduction, leveraging Hamming distance for one-hot encoded variables and correlation distance for numeric variables. This process reduced the dataset to 18 variables, optimizing it for modeling.
Model Selection and Training
Several machine learning models were tested to determine the most effective approach:
- Multinomial Regression: Simple and interpretable.
- Random Forests: An ensemble method that reduces overfitting and improves accuracy.
- Gradient Boosting Machines (GBM): A powerful technique that optimizes prediction iteratively.
Cross-Validation
To assess model performance and prevent overfitting, k-fold cross-validation (k=10) was applied and used to tune the parameter.
Model Evaluation
The models were compared using pr-auc, average precision, roc-auc, and F1-score. Results showed:
xgboost outperformed other models with 99% accuracy, making it the final choice.
Out-of-Sample Error Estimation
The expected out-of-sample error was estimated using cross-validation. The final model’s estimated error was approximately 1%, indicating high generalization performance.
| Metric | Estimate |
|---|---|
| Accuracy | 0.985 |
| Average Precision | 0.998 |
| F1 Measure | 0.984 |
| PR AUC | 0.998 |
| ROC AUC | 0.999 |
Conclusion
This project successfully leveraged machine learning to classify exercise execution based on accelerometer data. The xgboost model proved to be the most effective, achieving a near-perfect accuracy rate.
References
- Breiman, L. (2001). “Random Forests.” Machine Learning, 45(1), 5-32.
- Hastie, T., Tibshirani, R., & Friedman, J. (2009). “The Elements of Statistical Learning.” Springer.
- Kuhn, M., & Johnson, K. (2013). “Applied Predictive Modeling.” Springer.
- Velloso, E.; Bulling, A.; Gellersen, H.; Ugulino, W.; Fuks, H. Qualitative Activity Recognition of Weight Lifting Exercises. Proceedings of 4th International Conference in Cooperation with SIGCHI (Augmented Human ’13) . Stuttgart, Germany: ACM SIGCHI, 2013.