Prediction Assignment - Practical Machine Learning
Marcelo Tibau
11 de novembro de 2016
Executive Summary
Using devices such as Jawbone Up, Nike FuelBand, and Fitbit it is now possible to collect a large amount of data about personal activity relatively inexpensively. These type of devices are part of the quantified self movement - a group of enthusiasts who take measurements about themselves regularly to improve their health, to find patterns in their behavior, or because they are tech geeks. One thing that people regularly do is quantify how much of a particular activity they do, but they rarely quantify how well they do it. In this project, your goal will be to use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways. More information is available from the website here: http://groupware.les.inf.puc-rio.br/har (see the section on the Weight Lifting Exercise Dataset).
Data source
The data for this project come from the Human Activity Recognition study, conducted by Pontifícia Universidade Católica - Rio de Janeiro. Read more: http://groupware.les.inf.puc-rio.br/har.
Ugulino, W.; Cardador, D.; Vega, K.; Velloso, E.; Milidiu, R.; Fuks, H. Wearable Computing: Accelerometers’ Data Classification of Body Postures and Movements. Proceedings of 21st Brazilian Symposium on Artificial Intelligence. Advances in Artificial Intelligence - SBIA 2012. In: Lecture Notes in Computer Science. , pp. 52-61. Curitiba, PR: Springer Berlin / Heidelberg, 2012. ISBN 978-3-642-34458-9. DOI: 10.1007/978-3-642-34459-6_6.
The training data for this project are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv
The test data are available here:
https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv
Overview
This document contains two sections:
Analysis - where I provide the codes to download the cited training and testing datasets as well as clean them and also 3 machine learning algorithms - which regarding one different method of modeling.
Predicting Results - where I digress about what is the expected out of sample error and what model do I think is the best fit to predict the “classe” variable.
Analysis
Codes to load the libraries to be used:
library("caret")## Loading required package: lattice## Loading required package: ggplot2library("gbm")## Loading required package: survival##
## Attaching package: 'survival'## The following object is masked from 'package:caret':
##
## cluster## Loading required package: splines## Loading required package: parallel## Loaded gbm 2.1.1library("rpart")
library("rpart.plot")
library("RColorBrewer")
library("rattle")## Rattle: A free graphical interface for data mining with R.
## Version 4.1.0 Copyright (c) 2006-2015 Togaware Pty Ltd.
## Type 'rattle()' to shake, rattle, and roll your data.library("randomForest")## randomForest 4.6-12## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:ggplot2':
##
## marginCodes to download and read the train and test sets:
download.file("https://d396qusza40orc.cloudfront.net/predmachlearn/pml-training.csv", dest="pml-training.csv", mode="wb")
download.file("https://d396qusza40orc.cloudfront.net/predmachlearn/pml-testing.csv", dest="pml-testing.csv", mode="wb")
dataTrain <- read.csv("pml-training.csv")
dataTest <- read.csv("pml-testing.csv")
head(dataTrain)## X user_name raw_timestamp_part_1 raw_timestamp_part_2 cvtd_timestamp
## 1 1 carlitos 1323084231 788290 05/12/2011 11:23
## 2 2 carlitos 1323084231 808298 05/12/2011 11:23
## 3 3 carlitos 1323084231 820366 05/12/2011 11:23
## 4 4 carlitos 1323084232 120339 05/12/2011 11:23
## 5 5 carlitos 1323084232 196328 05/12/2011 11:23
## 6 6 carlitos 1323084232 304277 05/12/2011 11:23
## new_window num_window roll_belt pitch_belt yaw_belt total_accel_belt
## 1 no 11 1.41 8.07 -94.4 3
## 2 no 11 1.41 8.07 -94.4 3
## 3 no 11 1.42 8.07 -94.4 3
## 4 no 12 1.48 8.05 -94.4 3
## 5 no 12 1.48 8.07 -94.4 3
## 6 no 12 1.45 8.06 -94.4 3
## kurtosis_roll_belt kurtosis_picth_belt kurtosis_yaw_belt
## 1
## 2
## 3
## 4
## 5
## 6
## skewness_roll_belt skewness_roll_belt.1 skewness_yaw_belt max_roll_belt
## 1 NA
## 2 NA
## 3 NA
## 4 NA
## 5 NA
## 6 NA
## max_picth_belt max_yaw_belt min_roll_belt min_pitch_belt min_yaw_belt
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## amplitude_roll_belt amplitude_pitch_belt amplitude_yaw_belt
## 1 NA NA
## 2 NA NA
## 3 NA NA
## 4 NA NA
## 5 NA NA
## 6 NA NA
## var_total_accel_belt avg_roll_belt stddev_roll_belt var_roll_belt
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## avg_pitch_belt stddev_pitch_belt var_pitch_belt avg_yaw_belt
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## stddev_yaw_belt var_yaw_belt gyros_belt_x gyros_belt_y gyros_belt_z
## 1 NA NA 0.00 0.00 -0.02
## 2 NA NA 0.02 0.00 -0.02
## 3 NA NA 0.00 0.00 -0.02
## 4 NA NA 0.02 0.00 -0.03
## 5 NA NA 0.02 0.02 -0.02
## 6 NA NA 0.02 0.00 -0.02
## accel_belt_x accel_belt_y accel_belt_z magnet_belt_x magnet_belt_y
## 1 -21 4 22 -3 599
## 2 -22 4 22 -7 608
## 3 -20 5 23 -2 600
## 4 -22 3 21 -6 604
## 5 -21 2 24 -6 600
## 6 -21 4 21 0 603
## magnet_belt_z roll_arm pitch_arm yaw_arm total_accel_arm var_accel_arm
## 1 -313 -128 22.5 -161 34 NA
## 2 -311 -128 22.5 -161 34 NA
## 3 -305 -128 22.5 -161 34 NA
## 4 -310 -128 22.1 -161 34 NA
## 5 -302 -128 22.1 -161 34 NA
## 6 -312 -128 22.0 -161 34 NA
## avg_roll_arm stddev_roll_arm var_roll_arm avg_pitch_arm stddev_pitch_arm
## 1 NA NA NA NA NA
## 2 NA NA NA NA NA
## 3 NA NA NA NA NA
## 4 NA NA NA NA NA
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## var_pitch_arm avg_yaw_arm stddev_yaw_arm var_yaw_arm gyros_arm_x
## 1 NA NA NA NA 0.00
## 2 NA NA NA NA 0.02
## 3 NA NA NA NA 0.02
## 4 NA NA NA NA 0.02
## 5 NA NA NA NA 0.00
## 6 NA NA NA NA 0.02
## gyros_arm_y gyros_arm_z accel_arm_x accel_arm_y accel_arm_z magnet_arm_x
## 1 0.00 -0.02 -288 109 -123 -368
## 2 -0.02 -0.02 -290 110 -125 -369
## 3 -0.02 -0.02 -289 110 -126 -368
## 4 -0.03 0.02 -289 111 -123 -372
## 5 -0.03 0.00 -289 111 -123 -374
## 6 -0.03 0.00 -289 111 -122 -369
## magnet_arm_y magnet_arm_z kurtosis_roll_arm kurtosis_picth_arm
## 1 337 516
## 2 337 513
## 3 344 513
## 4 344 512
## 5 337 506
## 6 342 513
## kurtosis_yaw_arm skewness_roll_arm skewness_pitch_arm skewness_yaw_arm
## 1
## 2
## 3
## 4
## 5
## 6
## max_roll_arm max_picth_arm max_yaw_arm min_roll_arm min_pitch_arm
## 1 NA NA NA NA NA
## 2 NA NA NA NA NA
## 3 NA NA NA NA NA
## 4 NA NA NA NA NA
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## min_yaw_arm amplitude_roll_arm amplitude_pitch_arm amplitude_yaw_arm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## roll_dumbbell pitch_dumbbell yaw_dumbbell kurtosis_roll_dumbbell
## 1 13.05217 -70.49400 -84.87394
## 2 13.13074 -70.63751 -84.71065
## 3 12.85075 -70.27812 -85.14078
## 4 13.43120 -70.39379 -84.87363
## 5 13.37872 -70.42856 -84.85306
## 6 13.38246 -70.81759 -84.46500
## kurtosis_picth_dumbbell kurtosis_yaw_dumbbell skewness_roll_dumbbell
## 1
## 2
## 3
## 4
## 5
## 6
## skewness_pitch_dumbbell skewness_yaw_dumbbell max_roll_dumbbell
## 1 NA
## 2 NA
## 3 NA
## 4 NA
## 5 NA
## 6 NA
## max_picth_dumbbell max_yaw_dumbbell min_roll_dumbbell min_pitch_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## min_yaw_dumbbell amplitude_roll_dumbbell amplitude_pitch_dumbbell
## 1 NA NA
## 2 NA NA
## 3 NA NA
## 4 NA NA
## 5 NA NA
## 6 NA NA
## amplitude_yaw_dumbbell total_accel_dumbbell var_accel_dumbbell
## 1 37 NA
## 2 37 NA
## 3 37 NA
## 4 37 NA
## 5 37 NA
## 6 37 NA
## avg_roll_dumbbell stddev_roll_dumbbell var_roll_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## avg_pitch_dumbbell stddev_pitch_dumbbell var_pitch_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## avg_yaw_dumbbell stddev_yaw_dumbbell var_yaw_dumbbell gyros_dumbbell_x
## 1 NA NA NA 0
## 2 NA NA NA 0
## 3 NA NA NA 0
## 4 NA NA NA 0
## 5 NA NA NA 0
## 6 NA NA NA 0
## gyros_dumbbell_y gyros_dumbbell_z accel_dumbbell_x accel_dumbbell_y
## 1 -0.02 0.00 -234 47
## 2 -0.02 0.00 -233 47
## 3 -0.02 0.00 -232 46
## 4 -0.02 -0.02 -232 48
## 5 -0.02 0.00 -233 48
## 6 -0.02 0.00 -234 48
## accel_dumbbell_z magnet_dumbbell_x magnet_dumbbell_y magnet_dumbbell_z
## 1 -271 -559 293 -65
## 2 -269 -555 296 -64
## 3 -270 -561 298 -63
## 4 -269 -552 303 -60
## 5 -270 -554 292 -68
## 6 -269 -558 294 -66
## roll_forearm pitch_forearm yaw_forearm kurtosis_roll_forearm
## 1 28.4 -63.9 -153
## 2 28.3 -63.9 -153
## 3 28.3 -63.9 -152
## 4 28.1 -63.9 -152
## 5 28.0 -63.9 -152
## 6 27.9 -63.9 -152
## kurtosis_picth_forearm kurtosis_yaw_forearm skewness_roll_forearm
## 1
## 2
## 3
## 4
## 5
## 6
## skewness_pitch_forearm skewness_yaw_forearm max_roll_forearm
## 1 NA
## 2 NA
## 3 NA
## 4 NA
## 5 NA
## 6 NA
## max_picth_forearm max_yaw_forearm min_roll_forearm min_pitch_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## min_yaw_forearm amplitude_roll_forearm amplitude_pitch_forearm
## 1 NA NA
## 2 NA NA
## 3 NA NA
## 4 NA NA
## 5 NA NA
## 6 NA NA
## amplitude_yaw_forearm total_accel_forearm var_accel_forearm
## 1 36 NA
## 2 36 NA
## 3 36 NA
## 4 36 NA
## 5 36 NA
## 6 36 NA
## avg_roll_forearm stddev_roll_forearm var_roll_forearm avg_pitch_forearm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## stddev_pitch_forearm var_pitch_forearm avg_yaw_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## stddev_yaw_forearm var_yaw_forearm gyros_forearm_x gyros_forearm_y
## 1 NA NA 0.03 0.00
## 2 NA NA 0.02 0.00
## 3 NA NA 0.03 -0.02
## 4 NA NA 0.02 -0.02
## 5 NA NA 0.02 0.00
## 6 NA NA 0.02 -0.02
## gyros_forearm_z accel_forearm_x accel_forearm_y accel_forearm_z
## 1 -0.02 192 203 -215
## 2 -0.02 192 203 -216
## 3 0.00 196 204 -213
## 4 0.00 189 206 -214
## 5 -0.02 189 206 -214
## 6 -0.03 193 203 -215
## magnet_forearm_x magnet_forearm_y magnet_forearm_z classe
## 1 -17 654 476 A
## 2 -18 661 473 A
## 3 -18 658 469 A
## 4 -16 658 469 A
## 5 -17 655 473 A
## 6 -9 660 478 Ahead(dataTest)## X user_name raw_timestamp_part_1 raw_timestamp_part_2 cvtd_timestamp
## 1 1 pedro 1323095002 868349 05/12/2011 14:23
## 2 2 jeremy 1322673067 778725 30/11/2011 17:11
## 3 3 jeremy 1322673075 342967 30/11/2011 17:11
## 4 4 adelmo 1322832789 560311 02/12/2011 13:33
## 5 5 eurico 1322489635 814776 28/11/2011 14:13
## 6 6 jeremy 1322673149 510661 30/11/2011 17:12
## new_window num_window roll_belt pitch_belt yaw_belt total_accel_belt
## 1 no 74 123.00 27.00 -4.75 20
## 2 no 431 1.02 4.87 -88.90 4
## 3 no 439 0.87 1.82 -88.50 5
## 4 no 194 125.00 -41.60 162.00 17
## 5 no 235 1.35 3.33 -88.60 3
## 6 no 504 -5.92 1.59 -87.70 4
## kurtosis_roll_belt kurtosis_picth_belt kurtosis_yaw_belt
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## skewness_roll_belt skewness_roll_belt.1 skewness_yaw_belt max_roll_belt
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## max_picth_belt max_yaw_belt min_roll_belt min_pitch_belt min_yaw_belt
## 1 NA NA NA NA NA
## 2 NA NA NA NA NA
## 3 NA NA NA NA NA
## 4 NA NA NA NA NA
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## amplitude_roll_belt amplitude_pitch_belt amplitude_yaw_belt
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## var_total_accel_belt avg_roll_belt stddev_roll_belt var_roll_belt
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## avg_pitch_belt stddev_pitch_belt var_pitch_belt avg_yaw_belt
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## stddev_yaw_belt var_yaw_belt gyros_belt_x gyros_belt_y gyros_belt_z
## 1 NA NA -0.50 -0.02 -0.46
## 2 NA NA -0.06 -0.02 -0.07
## 3 NA NA 0.05 0.02 0.03
## 4 NA NA 0.11 0.11 -0.16
## 5 NA NA 0.03 0.02 0.00
## 6 NA NA 0.10 0.05 -0.13
## accel_belt_x accel_belt_y accel_belt_z magnet_belt_x magnet_belt_y
## 1 -38 69 -179 -13 581
## 2 -13 11 39 43 636
## 3 1 -1 49 29 631
## 4 46 45 -156 169 608
## 5 -8 4 27 33 566
## 6 -11 -16 38 31 638
## magnet_belt_z roll_arm pitch_arm yaw_arm total_accel_arm var_accel_arm
## 1 -382 40.7 -27.80 178 10 NA
## 2 -309 0.0 0.00 0 38 NA
## 3 -312 0.0 0.00 0 44 NA
## 4 -304 -109.0 55.00 -142 25 NA
## 5 -418 76.1 2.76 102 29 NA
## 6 -291 0.0 0.00 0 14 NA
## avg_roll_arm stddev_roll_arm var_roll_arm avg_pitch_arm stddev_pitch_arm
## 1 NA NA NA NA NA
## 2 NA NA NA NA NA
## 3 NA NA NA NA NA
## 4 NA NA NA NA NA
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## var_pitch_arm avg_yaw_arm stddev_yaw_arm var_yaw_arm gyros_arm_x
## 1 NA NA NA NA -1.65
## 2 NA NA NA NA -1.17
## 3 NA NA NA NA 2.10
## 4 NA NA NA NA 0.22
## 5 NA NA NA NA -1.96
## 6 NA NA NA NA 0.02
## gyros_arm_y gyros_arm_z accel_arm_x accel_arm_y accel_arm_z magnet_arm_x
## 1 0.48 -0.18 16 38 93 -326
## 2 0.85 -0.43 -290 215 -90 -325
## 3 -1.36 1.13 -341 245 -87 -264
## 4 -0.51 0.92 -238 -57 6 -173
## 5 0.79 -0.54 -197 200 -30 -170
## 6 0.05 -0.07 -26 130 -19 396
## magnet_arm_y magnet_arm_z kurtosis_roll_arm kurtosis_picth_arm
## 1 385 481 NA NA
## 2 447 434 NA NA
## 3 474 413 NA NA
## 4 257 633 NA NA
## 5 275 617 NA NA
## 6 176 516 NA NA
## kurtosis_yaw_arm skewness_roll_arm skewness_pitch_arm skewness_yaw_arm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## max_roll_arm max_picth_arm max_yaw_arm min_roll_arm min_pitch_arm
## 1 NA NA NA NA NA
## 2 NA NA NA NA NA
## 3 NA NA NA NA NA
## 4 NA NA NA NA NA
## 5 NA NA NA NA NA
## 6 NA NA NA NA NA
## min_yaw_arm amplitude_roll_arm amplitude_pitch_arm amplitude_yaw_arm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## roll_dumbbell pitch_dumbbell yaw_dumbbell kurtosis_roll_dumbbell
## 1 -17.73748 24.96085 126.23596 NA
## 2 54.47761 -53.69758 -75.51480 NA
## 3 57.07031 -51.37303 -75.20287 NA
## 4 43.10927 -30.04885 -103.32003 NA
## 5 -101.38396 -53.43952 -14.19542 NA
## 6 62.18750 -50.55595 -71.12063 NA
## kurtosis_picth_dumbbell kurtosis_yaw_dumbbell skewness_roll_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## skewness_pitch_dumbbell skewness_yaw_dumbbell max_roll_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## max_picth_dumbbell max_yaw_dumbbell min_roll_dumbbell min_pitch_dumbbell
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## min_yaw_dumbbell amplitude_roll_dumbbell amplitude_pitch_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## amplitude_yaw_dumbbell total_accel_dumbbell var_accel_dumbbell
## 1 NA 9 NA
## 2 NA 31 NA
## 3 NA 29 NA
## 4 NA 18 NA
## 5 NA 4 NA
## 6 NA 29 NA
## avg_roll_dumbbell stddev_roll_dumbbell var_roll_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## avg_pitch_dumbbell stddev_pitch_dumbbell var_pitch_dumbbell
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## avg_yaw_dumbbell stddev_yaw_dumbbell var_yaw_dumbbell gyros_dumbbell_x
## 1 NA NA NA 0.64
## 2 NA NA NA 0.34
## 3 NA NA NA 0.39
## 4 NA NA NA 0.10
## 5 NA NA NA 0.29
## 6 NA NA NA -0.59
## gyros_dumbbell_y gyros_dumbbell_z accel_dumbbell_x accel_dumbbell_y
## 1 0.06 -0.61 21 -15
## 2 0.05 -0.71 -153 155
## 3 0.14 -0.34 -141 155
## 4 -0.02 0.05 -51 72
## 5 -0.47 -0.46 -18 -30
## 6 0.80 1.10 -138 166
## accel_dumbbell_z magnet_dumbbell_x magnet_dumbbell_y magnet_dumbbell_z
## 1 81 523 -528 -56
## 2 -205 -502 388 -36
## 3 -196 -506 349 41
## 4 -148 -576 238 53
## 5 -5 -424 252 312
## 6 -186 -543 262 96
## roll_forearm pitch_forearm yaw_forearm kurtosis_roll_forearm
## 1 141 49.30 156.0 NA
## 2 109 -17.60 106.0 NA
## 3 131 -32.60 93.0 NA
## 4 0 0.00 0.0 NA
## 5 -176 -2.16 -47.9 NA
## 6 150 1.46 89.7 NA
## kurtosis_picth_forearm kurtosis_yaw_forearm skewness_roll_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## skewness_pitch_forearm skewness_yaw_forearm max_roll_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## max_picth_forearm max_yaw_forearm min_roll_forearm min_pitch_forearm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## min_yaw_forearm amplitude_roll_forearm amplitude_pitch_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## amplitude_yaw_forearm total_accel_forearm var_accel_forearm
## 1 NA 33 NA
## 2 NA 39 NA
## 3 NA 34 NA
## 4 NA 43 NA
## 5 NA 24 NA
## 6 NA 43 NA
## avg_roll_forearm stddev_roll_forearm var_roll_forearm avg_pitch_forearm
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## 5 NA NA NA NA
## 6 NA NA NA NA
## stddev_pitch_forearm var_pitch_forearm avg_yaw_forearm
## 1 NA NA NA
## 2 NA NA NA
## 3 NA NA NA
## 4 NA NA NA
## 5 NA NA NA
## 6 NA NA NA
## stddev_yaw_forearm var_yaw_forearm gyros_forearm_x gyros_forearm_y
## 1 NA NA 0.74 -3.34
## 2 NA NA 1.12 -2.78
## 3 NA NA 0.18 -0.79
## 4 NA NA 1.38 0.69
## 5 NA NA -0.75 3.10
## 6 NA NA -0.88 4.26
## gyros_forearm_z accel_forearm_x accel_forearm_y accel_forearm_z
## 1 -0.59 -110 267 -149
## 2 -0.18 212 297 -118
## 3 0.28 154 271 -129
## 4 1.80 -92 406 -39
## 5 0.80 131 -93 172
## 6 1.35 230 322 -144
## magnet_forearm_x magnet_forearm_y magnet_forearm_z problem_id
## 1 -714 419 617 1
## 2 -237 791 873 2
## 3 -51 698 783 3
## 4 -233 783 521 4
## 5 375 -787 91 5
## 6 -300 800 884 6As we can see above, there’s some NA data in both datasets. I intend to proceed a cleaning of the dataset, but first I will partion the training set into two and check possibles Near Zero Variance Variables (it’s a necessity regarding the subject dataset, which has a great amount of Not Available datas, which I will refer as NA, and zero values).
Codes to partioning the training dataset into two: 70% for training and 30% for testing. I set seed to 13563 for reproducibility purposes.
set.seed(13563)
inTrain <- createDataPartition(y=dataTrain$classe, p=0.7, list=FALSE)
training <- dataTrain[inTrain, ]
testing <- dataTrain[-inTrain, ]
dim(training)## [1] 13737 160dim(testing)## [1] 5885 160Code to cheack possibles Near Zero Variance Variables:
NZV_check <- nearZeroVar(training, saveMetrics = TRUE)
NZV_check## freqRatio percentUnique zeroVar nzv
## X 1.000000 100.00000000 FALSE FALSE
## user_name 1.104042 0.04367766 FALSE FALSE
## raw_timestamp_part_1 1.034483 6.09303341 FALSE FALSE
## raw_timestamp_part_2 1.000000 89.24073670 FALSE FALSE
## cvtd_timestamp 1.001914 0.14559220 FALSE FALSE
## new_window 49.135036 0.01455922 FALSE TRUE
## num_window 1.034483 6.23134600 FALSE FALSE
## roll_belt 1.077532 8.05124845 FALSE FALSE
## pitch_belt 1.029412 12.16422800 FALSE FALSE
## yaw_belt 1.064972 13.12513649 FALSE FALSE
## total_accel_belt 1.048561 0.20382907 FALSE FALSE
## kurtosis_roll_belt 1923.285714 1.95093543 FALSE TRUE
## kurtosis_picth_belt 498.629630 1.58695494 FALSE TRUE
## kurtosis_yaw_belt 49.135036 0.01455922 FALSE TRUE
## skewness_roll_belt 2243.833333 1.93637621 FALSE TRUE
## skewness_roll_belt.1 498.629630 1.68886948 FALSE TRUE
## skewness_yaw_belt 49.135036 0.01455922 FALSE TRUE
## max_roll_belt 1.125000 1.15017835 FALSE FALSE
## max_picth_belt 1.731707 0.15287181 FALSE FALSE
## max_yaw_belt 673.150000 0.41493776 FALSE TRUE
## min_roll_belt 1.375000 1.10650069 FALSE FALSE
## min_pitch_belt 2.102564 0.10919415 FALSE FALSE
## min_yaw_belt 673.150000 0.41493776 FALSE TRUE
## amplitude_roll_belt 1.120000 0.78619786 FALSE FALSE
## amplitude_pitch_belt 3.106383 0.09463493 FALSE FALSE
## amplitude_yaw_belt 51.385496 0.02911844 FALSE TRUE
## var_total_accel_belt 1.344828 0.32758244 FALSE FALSE
## avg_roll_belt 1.000000 1.01186576 FALSE FALSE
## stddev_roll_belt 1.081081 0.36398049 FALSE FALSE
## var_roll_belt 1.692308 0.46589503 FALSE FALSE
## avg_pitch_belt 1.333333 1.20113562 FALSE FALSE
## stddev_pitch_belt 1.139535 0.24022712 FALSE FALSE
## var_pitch_belt 1.200000 0.32758244 FALSE FALSE
## avg_yaw_belt 1.142857 1.31032977 FALSE FALSE
## stddev_yaw_belt 2.032258 0.32030283 FALSE FALSE
## var_yaw_belt 1.108108 0.76435903 FALSE FALSE
## gyros_belt_x 1.036269 0.92451045 FALSE FALSE
## gyros_belt_y 1.119946 0.48773386 FALSE FALSE
## gyros_belt_z 1.100083 1.19385601 FALSE FALSE
## accel_belt_x 1.044964 1.17929679 FALSE FALSE
## accel_belt_y 1.102683 0.99002693 FALSE FALSE
## accel_belt_z 1.046549 2.06740919 FALSE FALSE
## magnet_belt_x 1.048980 2.21300138 FALSE FALSE
## magnet_belt_y 1.085714 2.10380724 FALSE FALSE
## magnet_belt_z 1.021084 3.18846910 FALSE FALSE
## roll_arm 48.081633 17.34003057 FALSE FALSE
## pitch_arm 84.178571 20.23731528 FALSE FALSE
## yaw_arm 33.657143 19.13809420 FALSE FALSE
## total_accel_arm 1.006144 0.48045425 FALSE FALSE
## var_accel_arm 5.500000 1.91453738 FALSE FALSE
## avg_roll_arm 47.000000 1.65975104 FALSE TRUE
## stddev_roll_arm 47.000000 1.65975104 FALSE TRUE
## var_roll_arm 47.000000 1.65975104 FALSE TRUE
## avg_pitch_arm 47.000000 1.65975104 FALSE TRUE
## stddev_pitch_arm 47.000000 1.65975104 FALSE TRUE
## var_pitch_arm 47.000000 1.65975104 FALSE TRUE
## avg_yaw_arm 47.000000 1.65975104 FALSE TRUE
## stddev_yaw_arm 50.000000 1.63791221 FALSE TRUE
## var_yaw_arm 50.000000 1.63791221 FALSE TRUE
## gyros_arm_x 1.036517 4.59343379 FALSE FALSE
## gyros_arm_y 1.430137 2.69345563 FALSE FALSE
## gyros_arm_z 1.110193 1.71070831 FALSE FALSE
## accel_arm_x 1.076923 5.53978307 FALSE FALSE
## accel_arm_y 1.140940 3.81451554 FALSE FALSE
## accel_arm_z 1.081395 5.58346073 FALSE FALSE
## magnet_arm_x 1.000000 9.60180534 FALSE FALSE
## magnet_arm_y 1.062500 6.18038873 FALSE FALSE
## magnet_arm_z 1.135135 9.10679188 FALSE FALSE
## kurtosis_roll_arm 280.479167 1.65975104 FALSE TRUE
## kurtosis_picth_arm 269.260000 1.64519182 FALSE TRUE
## kurtosis_yaw_arm 1223.909091 1.92181699 FALSE TRUE
## skewness_roll_arm 286.446809 1.66703065 FALSE TRUE
## skewness_pitch_arm 269.260000 1.64519182 FALSE TRUE
## skewness_yaw_arm 1223.909091 1.91453738 FALSE TRUE
## max_roll_arm 15.666667 1.52871806 FALSE FALSE
## max_picth_arm 11.750000 1.35400743 FALSE FALSE
## max_yaw_arm 1.411765 0.35670088 FALSE FALSE
## min_roll_arm 15.666667 1.51415884 FALSE FALSE
## min_pitch_arm 15.666667 1.50687923 FALSE FALSE
## min_yaw_arm 1.000000 0.26934556 FALSE FALSE
## amplitude_roll_arm 15.666667 1.55783650 FALSE FALSE
## amplitude_pitch_arm 16.666667 1.51415884 FALSE FALSE
## amplitude_yaw_arm 1.105263 0.36398049 FALSE FALSE
## roll_dumbbell 1.032609 86.61279755 FALSE FALSE
## pitch_dumbbell 2.260870 84.22508554 FALSE FALSE
## yaw_dumbbell 1.057471 85.85571813 FALSE FALSE
## kurtosis_roll_dumbbell 4487.666667 1.96549465 FALSE TRUE
## kurtosis_picth_dumbbell 6731.500000 1.98733348 FALSE TRUE
## kurtosis_yaw_dumbbell 49.135036 0.01455922 FALSE TRUE
## skewness_roll_dumbbell 6731.500000 1.99461309 FALSE TRUE
## skewness_pitch_dumbbell 6731.500000 1.99461309 FALSE TRUE
## skewness_yaw_dumbbell 49.135036 0.01455922 FALSE TRUE
## max_roll_dumbbell 1.000000 1.72526753 FALSE FALSE
## max_picth_dumbbell 1.500000 1.74710636 FALSE FALSE
## max_yaw_dumbbell 897.533333 0.48773386 FALSE TRUE
## min_roll_dumbbell 1.000000 1.72526753 FALSE FALSE
## min_pitch_dumbbell 1.000000 1.84174128 FALSE FALSE
## min_yaw_dumbbell 897.533333 0.48773386 FALSE TRUE
## amplitude_roll_dumbbell 5.000000 1.91453738 FALSE FALSE
## amplitude_pitch_dumbbell 5.000000 1.88541894 FALSE FALSE
## amplitude_yaw_dumbbell 49.678967 0.02183883 FALSE TRUE
## total_accel_dumbbell 1.040000 0.31302322 FALSE FALSE
## var_accel_dumbbell 3.666667 1.88541894 FALSE FALSE
## avg_roll_dumbbell 1.000000 1.96549465 FALSE FALSE
## stddev_roll_dumbbell 10.000000 1.92909660 FALSE FALSE
## var_roll_dumbbell 10.000000 1.92909660 FALSE FALSE
## avg_pitch_dumbbell 1.000000 1.96549465 FALSE FALSE
## stddev_pitch_dumbbell 10.000000 1.92909660 FALSE FALSE
## var_pitch_dumbbell 10.000000 1.92909660 FALSE FALSE
## avg_yaw_dumbbell 1.000000 1.96549465 FALSE FALSE
## stddev_yaw_dumbbell 10.000000 1.92909660 FALSE FALSE
## var_yaw_dumbbell 10.000000 1.92909660 FALSE FALSE
## gyros_dumbbell_x 1.006977 1.68158987 FALSE FALSE
## gyros_dumbbell_y 1.248792 1.95093543 FALSE FALSE
## gyros_dumbbell_z 1.136919 1.40496469 FALSE FALSE
## accel_dumbbell_x 1.025532 2.98464002 FALSE FALSE
## accel_dumbbell_y 1.028090 3.32678168 FALSE FALSE
## accel_dumbbell_z 1.110465 2.89000510 FALSE FALSE
## magnet_dumbbell_x 1.105691 7.78918250 FALSE FALSE
## magnet_dumbbell_y 1.091603 6.00567810 FALSE FALSE
## magnet_dumbbell_z 1.000000 4.82638131 FALSE FALSE
## roll_forearm 12.782407 13.59831113 FALSE FALSE
## pitch_forearm 69.000000 19.18177186 FALSE FALSE
## yaw_forearm 16.134503 12.83395210 FALSE FALSE
## kurtosis_roll_forearm 224.383333 1.55783650 FALSE TRUE
## kurtosis_picth_forearm 224.383333 1.57239572 FALSE TRUE
## kurtosis_yaw_forearm 49.135036 0.01455922 FALSE TRUE
## skewness_roll_forearm 228.186441 1.56511611 FALSE TRUE
## skewness_pitch_forearm 224.383333 1.55055689 FALSE TRUE
## skewness_yaw_forearm 49.135036 0.01455922 FALSE TRUE
## max_roll_forearm 19.666667 1.39040547 FALSE TRUE
## max_picth_forearm 2.565217 0.85171435 FALSE FALSE
## max_yaw_forearm 224.383333 0.25478634 FALSE TRUE
## min_roll_forearm 29.500000 1.38312586 FALSE TRUE
## min_pitch_forearm 2.809524 0.93179006 FALSE FALSE
## min_yaw_forearm 224.383333 0.25478634 FALSE TRUE
## amplitude_roll_forearm 19.666667 1.46320157 FALSE TRUE
## amplitude_pitch_forearm 3.333333 0.95362889 FALSE FALSE
## amplitude_yaw_forearm 62.911215 0.02183883 FALSE TRUE
## total_accel_forearm 1.148276 0.48773386 FALSE FALSE
## var_accel_forearm 6.000000 1.95821504 FALSE FALSE
## avg_roll_forearm 29.500000 1.56511611 FALSE TRUE
## stddev_roll_forearm 62.000000 1.55055689 FALSE TRUE
## var_roll_forearm 62.000000 1.55055689 FALSE TRUE
## avg_pitch_forearm 59.000000 1.57239572 FALSE TRUE
## stddev_pitch_forearm 29.500000 1.56511611 FALSE TRUE
## var_pitch_forearm 59.000000 1.57239572 FALSE TRUE
## avg_yaw_forearm 59.000000 1.57239572 FALSE TRUE
## stddev_yaw_forearm 60.000000 1.56511611 FALSE TRUE
## var_yaw_forearm 60.000000 1.56511611 FALSE TRUE
## gyros_forearm_x 1.016043 2.06012958 FALSE FALSE
## gyros_forearm_y 1.062963 5.23403946 FALSE FALSE
## gyros_forearm_z 1.172205 2.16204411 FALSE FALSE
## accel_forearm_x 1.114754 5.68537526 FALSE FALSE
## accel_forearm_y 1.128571 7.11217879 FALSE FALSE
## accel_forearm_z 1.140000 4.03290384 FALSE FALSE
## magnet_forearm_x 1.019608 10.62823033 FALSE FALSE
## magnet_forearm_y 1.344262 13.21977142 FALSE FALSE
## magnet_forearm_z 1.113636 11.72017180 FALSE FALSE
## classe 1.469526 0.03639805 FALSE FALSECodes to reset both training and testing sets without NZV:
training <- training[, NZV_check$nzv==FALSE]
dim(training)## [1] 13737 105NZV_check2 <- nearZeroVar(testing, saveMetrics = TRUE)
testing <- testing[, NZV_check2$nzv==FALSE]
dim(testing)## [1] 5885 108Theres a second transformation worth doing, which is removing the ID variable (the first column) so that it won’t interfer with Machine Learning Algorithms:
training <- training[c(-1)]
testing <- testing[c(-1)]I decided to clean the variables with more than 60% NAs.The threshold was chosen based at the 60% method, which I considered the most appropriate Training Threshold method for this particular dataset.
I also think worth to explain what’s going on: first, I will create a temporary subset to iterate in loop, then I will check for NA in every column in the dataset, select the columns under the 60% method and remove them.For closers, I will set it back to the proper dataset and remove the temp dataset. As you will see the code involved is plain logical reasoning.
Cleaning the training dataset:
temp_train <- training
for (i in 1:length(training)) {
if(sum(is.na(training[, i]))/nrow(training)>=0.6) {
for (j in 1:length(temp_train)) {
if(length(grep(names(training[i]), names(temp_train)[j]))==1){
temp_train <- temp_train[, -j]
}
}
}
}
dim(temp_train) ## [1] 13737 58training <- temp_train
rm(temp_train)Cleaning the testing dataset:
temp_test <- testing
for (i in 1:length(testing)) {
if(sum(is.na(testing[, i]))/nrow(testing)>=0.6) {
for (j in 1:length(temp_test)) {
if(length(grep(names(testing[i]), names(temp_test)[j]))==1){
temp_test <- temp_test[, -j]
}
}
}
}
dim(temp_test) ## [1] 5885 58testing <- temp_test
rm(temp_test)I chose to build 3 models. One model, using a random forest (“rf”) algorithm, the other, using a decision trees algorithm and a third, using a boosted trees algorithm - also known as generalized boosted regression (“gbm”).
Then, I intend to cross validate it predicting the outcomes and checking the accuracy of each model at the testing set.
Codes to build the models:
Random Forest Algorithm
set.seed(13563)
modelFitRF <- randomForest(classe~., data = training)Cross validating the model:
predictFitRF <- predict(modelFitRF, testing, type = "class")To check the accuracy:
accuracy_FitRF <- confusionMatrix(predictFitRF, testing$classe)
accuracy_FitRF## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1674 3 0 0 0
## B 0 1136 0 0 0
## C 0 0 1026 5 0
## D 0 0 0 958 1
## E 0 0 0 1 1081
##
## Overall Statistics
##
## Accuracy : 0.9983
## 95% CI : (0.9969, 0.9992)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9979
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 0.9974 1.0000 0.9938 0.9991
## Specificity 0.9993 1.0000 0.9990 0.9998 0.9998
## Pos Pred Value 0.9982 1.0000 0.9952 0.9990 0.9991
## Neg Pred Value 1.0000 0.9994 1.0000 0.9988 0.9998
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2845 0.1930 0.1743 0.1628 0.1837
## Detection Prevalence 0.2850 0.1930 0.1752 0.1630 0.1839
## Balanced Accuracy 0.9996 0.9987 0.9995 0.9968 0.9994The accuracy of the Random Forest model is 0.9983, a very good one. To facilitate the visualization, I intend to plot it.
plot(modelFitRF, main = "Random Forest Algorithm")
plot(accuracy_FitRF$table, col = accuracy_FitRF$byClass, main = paste("Random Forest Algorithm Accuracy =", round(accuracy_FitRF$overall['Accuracy'], 4)))
Decision Tree Algorithm
set.seed(13563)
modelFitDT <- rpart(classe ~., data = training, method = "class")
fancyRpartPlot(modelFitDT)
Cross validating the model:
predictFitDT <- predict(modelFitDT, testing, type = "class")To check the accuracy:
accuracy_FitDT <- confusionMatrix(predictFitDT, testing$classe)
accuracy_FitDT## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1611 45 7 1 0
## B 45 964 53 47 0
## C 18 124 936 103 48
## D 0 6 17 667 74
## E 0 0 13 146 960
##
## Overall Statistics
##
## Accuracy : 0.8731
## 95% CI : (0.8643, 0.8815)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.8394
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 0.9624 0.8464 0.9123 0.6919 0.8872
## Specificity 0.9874 0.9694 0.9397 0.9803 0.9669
## Pos Pred Value 0.9681 0.8693 0.7616 0.8730 0.8579
## Neg Pred Value 0.9851 0.9634 0.9807 0.9420 0.9744
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2737 0.1638 0.1590 0.1133 0.1631
## Detection Prevalence 0.2828 0.1884 0.2088 0.1298 0.1901
## Balanced Accuracy 0.9749 0.9079 0.9260 0.8361 0.9271The model accuracy rate is 0.8731. Not a bad one, but less than Random Forest’s. Again, in order to facilitate the visualization, a plot is needed.
plot(accuracy_FitDT$table, col = accuracy_FitDT$byClass, main = paste("Decision Tree Algorithm Accuracy =", round(accuracy_FitDT$overall['Accuracy'], 4)))
Boosted Trees Algorithm
To load the library needed and set seed to reproducibility.
library(plyr)
set.seed(13563)I usually don’t use the trainControl function from the caret package because one of its uses is allow to perform a variety of cross validation. As the Confusion Matrix and the predict function allow us to do the same, I usually don’t see the point to trainControl the model. However, in this case the model took too long to fit and almost “hijacked” my computer memory, so I used it to cut it short.
FitControlGBM <- trainControl(method = "repeatedcv", number = 5, repeats = 1)
modelFitGBM <- train(classe~., data = training, method = "gbm", trControl = FitControlGBM, verbose = FALSE)
FinalmodelFitGBM <- modelFitGBM$finalModelCross validating the model:
predictFitGBM <- predict(modelFitGBM, newdata = testing)To check the accuracy:
accuracy_FitGBM <- confusionMatrix(predictFitGBM, testing$classe)
accuracy_FitGBM## Confusion Matrix and Statistics
##
## Reference
## Prediction A B C D E
## A 1674 3 0 0 0
## B 0 1133 0 0 0
## C 0 2 1022 7 0
## D 0 1 4 955 1
## E 0 0 0 2 1081
##
## Overall Statistics
##
## Accuracy : 0.9966
## 95% CI : (0.9948, 0.9979)
## No Information Rate : 0.2845
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.9957
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: A Class: B Class: C Class: D Class: E
## Sensitivity 1.0000 0.9947 0.9961 0.9907 0.9991
## Specificity 0.9993 1.0000 0.9981 0.9988 0.9996
## Pos Pred Value 0.9982 1.0000 0.9913 0.9938 0.9982
## Neg Pred Value 1.0000 0.9987 0.9992 0.9982 0.9998
## Prevalence 0.2845 0.1935 0.1743 0.1638 0.1839
## Detection Rate 0.2845 0.1925 0.1737 0.1623 0.1837
## Detection Prevalence 0.2850 0.1925 0.1752 0.1633 0.1840
## Balanced Accuracy 0.9996 0.9974 0.9971 0.9947 0.9993The accuracy of the model is rated at 0.9966. Although, comparing to Random Forest’s 0.9983 it’s not the best model. Once again, a plot to facilitate the visualization.
plot(modelFitGBM, ylim = c(0.9, 1))
Predicting Results
Random Forests gave an accuracy of 99.89%, this means that this model is more accurate than the Decision Trees or GBM models.
The expected out-of-sample error is 0.11% (100-99.89).
Code to proceed the prediction:
prediction_results <- predict(modelFitRF, testing, type = "class")
prediction_results## 5 10 12 13 15 21 22 23 30 36 48 51
## A A A A A A A A A A A A
## 57 59 63 65 67 69 72 83 85 89 92 93
## A A A A A A A A A A A A
## 96 97 102 105 107 109 112 114 116 117 118 119
## A A A A A A A A A A A A
## 120 124 130 133 134 138 139 141 142 144 146 152
## A A A A A A A A A A A A
## 156 157 163 165 167 169 174 176 179 188 196 197
## A A A A A A A A A A A A
## 198 203 205 210 211 212 213 214 217 220 223 224
## A A A A A A A A A A A A
## 225 227 229 230 231 232 235 238 241 243 249 252
## A A A A A A A A A A A A
## 253 266 273 277 281 282 284 291 292 298 300 306
## A A A A A A A A A A A A
## 311 312 315 317 318 324 326 330 331 332 333 337
## A A A A A A A A A A A A
## 339 345 348 352 356 358 360 361 364 366 372 376
## A A A A A A A A A A A A
## 377 378 389 390 394 397 398 404 405 407 411 413
## A A A A A A A A A A A A
## 414 416 417 420 426 428 430 438 439 449 454 456
## A A A A A A A A A A A A
## 457 466 467 469 471 477 484 486 490 491 499 500
## A A A A A A A A A A A A
## 501 504 505 506 516 517 519 520 530 533 535 543
## A A A A A A A A A A A A
## 546 552 559 560 566 567 569 570 572 578 580 581
## A A A A A A A A A A A A
## 583 591 598 601 605 607 608 609 611 617 618 620
## A A A A A A A A A A A A
## 623 626 629 631 636 638 639 640 641 644 645 648
## A A A A A A A A A A A A
## 662 664 666 670 671 673 674 681 682 687 688 691
## A A A A A A A A A A A A
## 697 698 700 712 719 721 724 725 727 730 733 734
## A A A A A A A A A A A A
## 735 736 739 740 744 748 750 754 755 763 769 772
## A A A A A A A A A A A A
## 774 775 776 777 778 779 781 788 791 792 797 799
## A A A A A A A A A A A A
## 800 817 818 819 823 826 840 845 846 848 850 851
## A A A A A A A A A A A A
## 855 856 861 862 864 868 871 875 882 885 886 890
## A A A A A A A A A A A A
## 894 895 897 898 899 906 909 910 911 919 921 925
## A A A A A A A A A A A A
## 929 936 939 945 948 955 960 961 966 968 970 972
## A A A A A A A A A A A A
## 977 979 982 983 984 987 988 989 996 997 1001 1003
## A A A A A A A A A A A A
## 1007 1014 1017 1021 1025 1035 1036 1037 1041 1044 1046 1047
## A A A A A A A A A A A A
## 1048 1049 1052 1053 1056 1057 1060 1065 1067 1069 1070 1072
## A A A A A A A A A A A A
## 1078 1084 1086 1087 1088 1092 1094 1097 1098 1099 1107 1115
## A A A A A A A A A A A A
## 1120 1127 1131 1132 1133 1142 1146 1153 1155 1161 1166 1168
## A A A A A A A A A A A A
## 1170 1173 1178 1179 1180 1186 1188 1190 1191 1192 1194 1199
## A A A A A A A A A A A A
## 1208 1210 1211 1212 1227 1230 1234 1236 1237 1239 1242 1243
## A A A A A A A A A A A A
## 1244 1246 1247 1248 1262 1268 1270 1271 1275 1276 1280 1281
## A A A A A A A A A A A A
## 1282 1283 1285 1286 1287 1288 1295 1298 1299 1301 1303 1309
## A A A A A A A A A A A A
## 1311 1315 1316 1324 1325 1328 1331 1332 1335 1339 1343 1347
## A A A A A A A A A A A A
## 1349 1363 1366 1369 1370 1372 1373 1377 1378 1380 1382 1383
## A A A A A A A A A A A A
## 1392 1393 1403 1404 1405 1407 1408 1410 1411 1416 1419 1422
## A A A A A A A A A A A A
## 1424 1427 1430 1432 1434 1436 1438 1441 1442 1443 1450 1453
## A A A A A A A A A A A A
## 1459 1460 1467 1469 1470 1471 1476 1483 1484 1488 1489 1491
## A A A A A A A A A A A A
## 1492 1493 1494 1495 1502 1503 1508 1509 1512 1519 1521 1522
## A A A A A A A A A A A A
## 1527 1530 1536 1540 1543 1544 1545 1546 1548 1556 1558 1560
## A A A A A A A A A A A A
## 1561 1566 1567 1569 1572 1578 1588 1595 1602 1603 1605 1611
## A A A A A A A A A A A A
## 1612 1618 1620 1621 1623 1624 1627 1633 1635 1639 1642 1645
## A A A A A A A A A A A A
## 1648 1649 1652 1656 1662 1665 1667 1677 1684 1690 1696 1705
## A A A A A A A A A A A A
## 1706 1707 1708 1712 1713 1714 1715 1717 1718 1721 1722 1726
## A A A A A A A A A A A A
## 1728 1731 1732 1736 1741 1743 1748 1751 1752 1753 1757 1758
## A A A A A A A A A A A A
## 1759 1760 1768 1774 1776 1780 1781 1786 1788 1789 1790 1791
## A A A A A A A A A A A A
## 1792 1796 1799 1800 1801 1806 1808 1809 1816 1817 1818 1819
## A A A A A A A A A A A A
## 1821 1826 1832 1841 1843 1846 1849 1851 1852 1860 1862 1863
## A A A A A A A A A A A A
## 1864 1865 1870 1871 1874 1879 1880 1884 1889 1890 1891 1893
## A A A A A A A A A A A A
## 1899 1900 1905 1909 1912 1914 1915 1917 1923 1928 1930 1931
## A A A A A A A A A A A A
## 1934 1944 1954 1956 1960 1963 1965 1966 1972 1976 1978 1979
## A A A A A A A A A A A A
## 1982 1988 1989 1992 1993 1998 1999 2000 2007 2013 2015 2019
## A A A A A A A A A A A A
## 2022 2023 2024 2030 2035 2037 2038 2039 2044 2049 2051 2053
## A A A A A A A A A A A A
## 2054 2055 2058 2070 2074 2075 2077 2080 2083 2084 2086 2087
## A A A A A A A A A A A A
## 2088 2094 2095 2106 2108 2114 2115 2116 2117 2118 2122 2127
## A A A A A A A A A A A A
## 2128 2129 2132 2137 2141 2146 2151 2152 2153 2156 2163 2164
## A A A A A A A A A A A A
## 2165 2170 2177 2178 2179 2182 2183 2187 2193 2195 2201 2206
## A A A A A A A A A A A A
## 2212 2215 2216 2218 2225 2226 2228 2231 2232 2235 2243 2245
## A A A A A A A A A A A A
## 2249 2263 2267 2269 2278 2280 2281 2282 2286 2287 2289 2291
## A A A A A A A A A A A A
## 2298 2301 2302 2304 2308 2314 2315 2317 2319 2320 2322 2334
## A A A A A A A A A A A A
## 2337 2341 2343 2345 2347 2349 2351 2352 2354 2355 2358 2368
## A A A A A A A A A A A A
## 2371 2373 2374 2378 2380 2381 2384 2388 2389 2394 2404 2409
## A A A A A A A A A A A A
## 2410 2417 2419 2420 2421 2425 2428 2429 2430 2434 2436 2442
## A A A A A A A A A A A A
## 2447 2448 2450 2454 2471 2473 2474 2475 2478 2490 2491 2500
## A A A A A A A A A A A A
## 2501 2502 2505 2508 2516 2519 2520 2522 2533 2538 2541 2542
## A A A A A A A A A A A A
## 2543 2545 2548 2551 2553 2556 2557 2558 2562 2563 2565 2566
## A A A A A A A A A A A A
## 2567 2568 2581 2582 2584 2585 2586 2588 2593 2596 2597 2600
## A A A A A A A A A A A A
## 2601 2605 2606 2608 2610 2612 2613 2617 2620 2622 2623 2636
## A A A A A A A A A A A A
## 2637 2643 2646 2651 2653 2656 2661 2662 2666 2667 2669 2676
## A A A A A A A A A A A A
## 2679 2682 2688 2690 2691 2693 2694 2701 2704 2706 2707 2709
## A A A A A A A A A A A A
## 2713 2721 2724 2725 2727 2728 2734 2738 2740 2751 2752 2761
## A A A A A A A A A A A A
## 2766 2769 2774 2775 2777 2779 2782 2792 2795 2801 2809 2811
## A A A A A A A A A A A A
## 2812 2813 2816 2817 2823 2825 2829 2830 2832 2833 2843 2844
## A A A A A A A A A A A A
## 2846 2849 2850 2858 2859 2862 2865 2867 2874 2876 2877 2881
## A A A A A A A A A A A A
## 2884 2888 2892 2895 2901 2903 2909 2914 2915 2920 2921 2923
## A A A A A A A A A A A A
## 2925 2926 2927 2936 2941 2942 2943 2951 2953 2961 2962 2964
## A A A A A A A A A A A A
## 2965 2966 2979 2986 2987 2989 2991 2992 2993 3002 3003 3004
## A A A A A A A A A A A A
## 3012 3017 3023 3025 3026 3028 3029 3035 3037 3040 3041 3042
## A A A A A A A A A A A A
## 3047 3048 3050 3052 3055 3061 3064 3073 3079 3080 3081 3082
## A A A A A A A A A A A A
## 3083 3086 3088 3094 3097 3099 3102 3103 3107 3108 3110 3119
## A A A A A A A A A A A A
## 3122 3123 3131 3132 3134 3137 3138 3143 3144 3148 3153 3154
## A A A A A A A A A A A A
## 3155 3158 3162 3169 3180 3184 3187 3192 3194 3195 3207 3210
## A A A A A A A A A A A A
## 3211 3215 3224 3228 3231 3236 3238 3240 3241 3243 3244 3252
## A A A A A A A A A A A A
## 3253 3263 3265 3269 3270 3272 3273 3274 3276 3278 3280 3283
## A A A A A A A A A A A A
## 3288 3292 3293 3296 3300 3303 3306 3310 3313 3319 3320 3323
## A A A A A A A A A A A A
## 3324 3325 3327 3328 3330 3332 3334 3337 3338 3339 3340 3341
## A A A A A A A A A A A A
## 3343 3346 3348 3349 3351 3352 3362 3375 3376 3377 3382 3383
## A A A A A A A A A A A A
## 3387 3391 3397 3400 3403 3405 3406 3408 3409 3410 3413 3417
## A A A A A A A A A A A A
## 3424 3425 3429 3431 3433 3438 3445 3450 3453 3464 3468 3471
## A A A A A A A A A A A A
## 3473 3474 3476 3484 3489 3492 3493 3497 3498 3500 3506 3514
## A A A A A A A A A A A A
## 3520 3527 3529 3531 3532 3539 3540 3544 3554 3557 3558 3561
## A A A A A A A A A A A A
## 3562 3565 3568 3575 3576 3579 3585 3588 3591 3592 3593 3595
## A A A A A A A A A A A A
## 3597 3598 3600 3602 3606 3610 3614 3615 3616 3621 3624 3627
## A A A A A A A A A A A A
## 3628 3633 3637 3638 3644 3645 3650 3653 3656 3657 3660 3667
## A A A A A A A A A A A A
## 3669 3670 3672 3673 3674 3676 3677 3678 3679 3682 3685 3686
## A A A A A A A A A A A A
## 3687 3688 3690 3692 3693 3697 3698 3699 3701 3704 3705 3712
## A A A A A A A A A A A A
## 3716 3718 3721 3722 3738 3743 3745 3748 3749 3753 3759 3760
## A A A A A A A A A A A A
## 3761 3766 3767 3769 3770 3774 3781 3782 3791 3794 3796 3798
## A A A A A A A A A A A A
## 3804 3807 3818 3819 3823 3826 3828 3830 3837 3838 3839 3840
## A A A A A A A A A A A A
## 3845 3846 3854 3865 3868 3871 3873 3877 3881 3889 3892 3893
## A A A A A A A A A A A A
## 3897 3898 3899 3901 3915 3923 3928 3929 3930 3931 3934 3940
## A A A A A A A A A A A A
## 3942 3945 3947 3949 3950 3952 3955 3962 3965 3968 3974 3976
## A A A A A A A A A A A A
## 3984 3987 3988 3989 3996 4000 4004 4014 4021 4024 4027 4028
## A A A A A A A A A A A A
## 4030 4034 4042 4043 4051 4053 4057 4060 4070 4077 4078 4080
## A A A A A A A A A A A A
## 4082 4083 4093 4097 4098 4106 4114 4115 4116 4122 4125 4130
## A A A A A A A A A A A A
## 4142 4143 4145 4147 4148 4149 4151 4154 4157 4162 4163 4170
## A A A A A A A A A A A A
## 4177 4181 4184 4186 4190 4192 4203 4206 4212 4213 4218 4223
## A A A A A A A A A A A A
## 4227 4234 4239 4240 4244 4251 4259 4263 4264 4265 4267 4268
## A A A A A A A A A A A A
## 4270 4272 4273 4282 4285 4289 4301 4308 4319 4320 4325 4330
## A A A A A A A A A A A A
## 4333 4336 4337 4339 4343 4344 4347 4354 4356 4357 4359 4362
## A A A A A A A A A A A A
## 4366 4368 4372 4373 4384 4386 4391 4393 4394 4397 4400 4404
## A A A A A A A A A A A A
## 4405 4407 4411 4412 4418 4422 4423 4429 4435 4436 4437 4438
## A A A A A A A A A A A A
## 4443 4444 4445 4448 4449 4450 4453 4454 4455 4458 4460 4461
## A A A A A A A A A A A A
## 4462 4470 4474 4475 4480 4482 4484 4487 4488 4492 4494 4496
## A A A A A A A A A A A A
## 4499 4509 4512 4514 4520 4522 4524 4525 4532 4534 4549 4550
## A A A A A A A A A A A A
## 4553 4555 4558 4560 4562 4565 4566 4582 4593 4594 4597 4599
## A A A A A A A A A A A A
## 4602 4604 4606 4607 4608 4610 4615 4618 4621 4623 4625 4627
## A A A A A A A A A A A A
## 4628 4629 4630 4633 4635 4640 4641 4643 4647 4649 4650 4651
## A A A A A A A A A A A A
## 4655 4656 4662 4663 4664 4671 4681 4686 4690 4691 4694 4695
## A A A A A A A A A A A A
## 4709 4711 4712 4713 4716 4717 4727 4737 4740 4742 4744 4756
## A A A A A A A A A A A A
## 4758 4764 4767 4771 4782 4790 4791 4799 4803 4804 4805 4806
## A A A A A A A A A A A A
## 4812 4817 4820 4823 4829 4831 4835 4836 4840 4845 4847 4850
## A A A A A A A A A A A A
## 4856 4857 4861 4862 4869 4874 4880 4883 4885 4905 4910 4911
## A A A A A A A A A A A A
## 4914 4922 4926 4928 4929 4932 4935 4943 4957 4958 4961 4970
## A A A A A A A A A A A A
## 4977 4978 4988 4990 4991 4993 4997 4999 5002 5004 5005 5010
## A A A A A A A A A A A A
## 5011 5015 5019 5026 5027 5028 5029 5030 5033 5036 5039 5041
## A A A A A A A A A A A A
## 5046 5049 5051 5055 5057 5065 5070 5072 5074 5075 5076 5082
## A A A A A A A A A A A A
## 5083 5084 5088 5094 5095 5102 5114 5115 5116 5118 5123 5129
## A A A A A A A A A A A A
## 5130 5131 5134 5143 5147 5148 5151 5157 5168 5170 5171 5179
## A A A A A A A A A A A A
## 5183 5184 5188 5189 5197 5200 5202 5210 5212 5213 5220 5224
## A A A A A A A A A A A A
## 5227 5229 5233 5239 5242 5245 5248 5250 5252 5253 5254 5255
## A A A A A A A A A A A A
## 5256 5257 5259 5266 5267 5269 5270 5271 5274 5277 5283 5286
## A A A A A A A A A A A A
## 5288 5291 5292 5298 5303 5304 5306 5311 5316 5319 5322 5325
## A A A A A A A A A A A A
## 5327 5328 5332 5333 5355 5358 5360 5363 5367 5377 5382 5386
## A A A A A A A A A A A A
## 5388 5389 5391 5399 5402 5403 5404 5412 5413 5414 5421 5422
## A A A A A A A A A A A A
## 5427 5429 5431 5436 5438 5442 5444 5446 5451 5455 5458 5461
## A A A A A A A A A A A A
## 5465 5468 5469 5472 5478 5481 5483 5486 5490 5499 5501 5505
## A A A A A A A A A A A A
## 5514 5519 5520 5525 5531 5532 5535 5537 5543 5545 5550 5552
## A A A A A A A A A A A A
## 5554 5562 5567 5571 5572 5578 5584 5585 5586 5588 5596 5599
## A A A A A A B B B B B B
## 5600 5601 5605 5611 5612 5614 5615 5617 5618 5619 5620 5624
## B B B B B B B B B B B B
## 5626 5629 5630 5631 5634 5637 5638 5640 5650 5652 5655 5656
## B B B B B B B B B B B B
## 5657 5662 5663 5664 5669 5671 5672 5682 5684 5686 5687 5691
## B B B B B B B B B B B B
## 5696 5699 5701 5711 5712 5714 5715 5717 5719 5721 5726 5728
## B B B B B B B B B B B B
## 5729 5735 5737 5742 5743 5744 5746 5747 5748 5752 5753 5760
## B B B B B B B B B B B B
## 5764 5765 5773 5781 5782 5787 5789 5804 5805 5810 5814 5819
## B B B B B B B B B B B B
## 5822 5825 5826 5830 5831 5834 5835 5836 5840 5843 5845 5847
## B B B B B B B B B B B B
## 5852 5853 5854 5857 5858 5859 5860 5862 5865 5868 5869 5870
## B B B B B B B B B B B B
## 5871 5873 5874 5876 5880 5887 5889 5890 5891 5897 5902 5903
## B B B B B B B B B B B B
## 5905 5911 5914 5920 5921 5925 5930 5933 5934 5935 5945 5953
## B B B B B B B B B B B B
## 5954 5957 5959 5965 5966 5967 5969 5972 5975 5976 5981 5989
## B B B B B B B B B B B B
## 5990 5994 5997 6000 6001 6005 6006 6009 6014 6018 6022 6031
## B B B B B B B B B B A B
## 6035 6036 6038 6039 6054 6057 6059 6064 6065 6071 6079 6080
## B B B B B B B B B B B B
## 6082 6084 6086 6087 6093 6107 6110 6113 6123 6125 6128 6129
## B B B B B B B B B B B B
## 6133 6138 6140 6141 6143 6145 6146 6149 6151 6152 6156 6158
## B B B B B B B B B B B B
## 6161 6165 6167 6168 6171 6173 6175 6177 6182 6185 6192 6193
## B B B B B B B B B B B B
## 6194 6196 6197 6199 6201 6205 6206 6209 6210 6211 6213 6217
## B B B B B B B B B B B B
## 6228 6230 6232 6245 6247 6251 6254 6261 6267 6272 6278 6279
## B B B B B B B B B B B B
## 6286 6292 6293 6297 6299 6302 6304 6308 6309 6312 6321 6329
## B B B B B B B B B B B B
## 6332 6333 6336 6339 6348 6354 6355 6359 6361 6365 6366 6368
## B B B B B B B B B B B B
## 6370 6375 6379 6381 6384 6389 6393 6396 6400 6401 6410 6411
## B B B B B B B B B B B B
## 6421 6429 6432 6437 6439 6440 6441 6447 6452 6454 6455 6457
## B B B B B B B B B B B B
## 6464 6465 6468 6470 6473 6474 6475 6480 6482 6486 6492 6494
## B B B B B B B B B B B B
## 6497 6498 6499 6500 6501 6503 6504 6506 6507 6509 6512 6523
## B B B B B B B B B B B B
## 6526 6532 6537 6541 6542 6548 6550 6552 6555 6559 6560 6565
## B B B B B B B B B B B B
## 6567 6574 6577 6578 6579 6582 6583 6591 6600 6606 6615 6616
## B B B B B B B B B B B B
## 6618 6624 6630 6632 6633 6635 6638 6639 6646 6648 6657 6663
## B B B B B B B B B B B B
## 6665 6667 6671 6675 6685 6686 6687 6689 6692 6696 6698 6701
## B B B B B B B B B B B B
## 6707 6712 6718 6724 6727 6731 6735 6739 6741 6746 6747 6750
## B B B B B B B B B B B B
## 6751 6754 6755 6756 6759 6760 6767 6769 6773 6776 6780 6781
## B B B B B B B B B B B B
## 6784 6786 6789 6792 6793 6794 6797 6803 6805 6807 6813 6815
## B B B B B B B B B B B B
## 6819 6820 6823 6826 6828 6830 6835 6837 6846 6847 6848 6849
## B B B B B B B B B B B B
## 6855 6856 6860 6861 6862 6863 6866 6868 6873 6878 6883 6887
## B B B B B B B B B B B B
## 6889 6899 6901 6905 6907 6908 6913 6914 6919 6920 6928 6933
## B B B B B B B B B B B B
## 6935 6936 6938 6940 6943 6950 6951 6953 6954 6957 6961 6962
## B B B B B B B B B B B B
## 6968 6969 6970 6971 6976 6980 6982 6988 6991 6995 6996 6998
## B B B B B B B B B B B B
## 7000 7007 7010 7013 7014 7018 7019 7021 7022 7028 7029 7031
## B B B B B B B B B B B B
## 7033 7048 7049 7051 7056 7061 7063 7064 7065 7066 7069 7076
## B B B B B B B B B B B B
## 7077 7078 7079 7082 7087 7088 7095 7103 7113 7115 7118 7119
## B B B B B B B B B B B B
## 7120 7122 7123 7126 7127 7129 7133 7135 7137 7138 7139 7142
## B B B B B B B B B B B B
## 7145 7147 7152 7155 7156 7162 7165 7167 7171 7172 7174 7178
## B B B B B B B B B B B B
## 7179 7180 7181 7183 7186 7188 7190 7199 7201 7205 7208 7215
## B B B B B B B B B B B B
## 7219 7220 7223 7224 7225 7241 7243 7246 7247 7248 7255 7259
## B B B B B B B B B B B B
## 7260 7263 7264 7267 7268 7272 7274 7276 7277 7278 7280 7284
## B B B B B B B B B B B B
## 7289 7290 7291 7292 7294 7295 7297 7298 7299 7300 7302 7304
## B B B B B B B B B B B B
## 7307 7309 7312 7313 7317 7319 7320 7323 7335 7337 7338 7341
## B B B B B B B B B B B B
## 7345 7351 7352 7353 7355 7361 7364 7368 7370 7371 7375 7378
## B B B B B B B B B B B B
## 7382 7383 7384 7394 7395 7396 7400 7407 7408 7411 7416 7417
## B B B B B B B B B B B B
## 7423 7425 7431 7433 7436 7441 7442 7445 7450 7451 7457 7458
## B B B B B B B B B B B B
## 7460 7468 7474 7477 7488 7490 7492 7496 7502 7507 7508 7509
## B B B B B B B B B B B B
## 7516 7520 7525 7526 7532 7538 7541 7543 7544 7547 7549 7552
## B B B B B B B B B B B B
## 7553 7557 7559 7561 7565 7567 7569 7573 7578 7591 7592 7597
## B B B B B B B B B B B B
## 7598 7600 7604 7608 7611 7618 7624 7627 7629 7632 7633 7636
## B B B B B B B B B B B B
## 7638 7641 7644 7649 7651 7652 7655 7658 7661 7664 7666 7667
## B B B B B B B B B B B B
## 7671 7674 7677 7682 7692 7693 7697 7699 7702 7703 7704 7705
## B B B B B B B B B B B B
## 7708 7712 7713 7726 7729 7731 7735 7736 7738 7739 7741 7742
## B B B B B B B B B B B B
## 7744 7745 7746 7747 7753 7755 7764 7771 7772 7777 7789 7791
## B B B B B B B B B B B B
## 7802 7804 7805 7808 7809 7813 7815 7821 7823 7827 7830 7836
## B B B B B B B B B B B B
## 7838 7846 7847 7852 7853 7855 7857 7858 7862 7864 7865 7872
## B B B B B B B B B B B B
## 7873 7874 7875 7878 7879 7886 7887 7898 7903 7904 7907 7912
## B B B B B B B B B B B B
## 7918 7919 7921 7922 7925 7930 7932 7935 7936 7938 7943 7952
## B B B B B B B B B B B B
## 7960 7961 7962 7963 7967 7973 7978 7979 7981 7982 7991 7995
## B B B B B B B B B B B B
## 7997 7998 7999 8002 8005 8017 8018 8019 8020 8022 8024 8030
## B B B B B B B B B B B B
## 8039 8053 8059 8062 8066 8070 8071 8079 8081 8082 8087 8089
## B B B B B B B B B B B B
## 8090 8091 8097 8100 8102 8103 8106 8107 8122 8124 8133 8134
## B B B B B B B B B B B B
## 8137 8141 8142 8145 8147 8149 8159 8161 8162 8163 8169 8174
## B B B B B B B B B B B B
## 8175 8176 8177 8179 8181 8182 8184 8186 8187 8190 8196 8198
## B B B B B B B B B B B B
## 8201 8203 8213 8214 8215 8217 8219 8223 8227 8232 8239 8240
## B B B B B B B B B B B B
## 8244 8248 8250 8251 8253 8260 8261 8272 8286 8288 8291 8292
## B B B B B B B B B B B B
## 8294 8299 8302 8306 8307 8309 8310 8311 8312 8316 8317 8318
## B B B B B B B B B B B B
## 8320 8322 8330 8331 8347 8349 8350 8352 8355 8361 8363 8380
## B B B B B B B B B B B B
## 8383 8389 8399 8407 8411 8412 8413 8420 8440 8446 8455 8459
## B B B B B B B B B B B B
## 8461 8462 8464 8465 8468 8473 8477 8478 8482 8483 8489 8491
## B B B B B B B B B B B B
## 8498 8501 8504 8506 8508 8509 8510 8512 8513 8514 8522 8523
## B B B B B B B B B B B B
## 8531 8533 8535 8538 8539 8540 8542 8544 8550 8552 8555 8559
## B B B B B B B B B B B B
## 8560 8561 8566 8568 8576 8577 8578 8580 8581 8583 8586 8587
## B B B B B B B B B B B B
## 8588 8590 8601 8603 8609 8612 8615 8619 8620 8622 8624 8627
## B B B B B B B A A B B B
## 8632 8641 8643 8646 8649 8655 8660 8662 8663 8664 8671 8672
## B B B B B B B B B B B B
## 8677 8679 8683 8684 8688 8706 8719 8721 8723 8724 8727 8732
## B B B B B B B B B B B B
## 8737 8738 8740 8745 8749 8750 8758 8763 8764 8767 8772 8774
## B B B B B B B B B B B B
## 8776 8777 8781 8782 8783 8788 8790 8794 8797 8804 8809 8810
## B B B B B B B B B B B B
## 8817 8818 8820 8821 8822 8827 8829 8832 8837 8838 8848 8850
## B B B B B B B B B B B B
## 8851 8852 8860 8862 8865 8867 8868 8873 8875 8879 8882 8888
## B B B B B B B B B B B B
## 8890 8895 8898 8903 8909 8919 8922 8927 8928 8934 8936 8937
## B B B B B B B B B B B B
## 8939 8942 8943 8948 8953 8957 8958 8972 8976 8978 8985 8992
## B B B B B B B B B B B B
## 8993 8994 8996 9004 9016 9018 9019 9022 9030 9031 9040 9041
## B B B B B B B B B B B B
## 9042 9044 9055 9057 9058 9059 9060 9063 9064 9065 9068 9069
## B B B B B B B B B B B B
## 9072 9073 9080 9082 9083 9084 9088 9089 9090 9099 9106 9117
## B B B B B B B B B B B B
## 9118 9122 9123 9125 9127 9129 9130 9131 9132 9133 9135 9138
## B B B B B B B B B B B B
## 9143 9145 9152 9153 9158 9162 9163 9178 9180 9185 9187 9191
## B B B B B B B B B B B B
## 9198 9200 9207 9210 9212 9213 9229 9231 9239 9240 9242 9243
## B B B B B B B B B B B B
## 9246 9247 9251 9252 9256 9262 9264 9272 9279 9281 9283 9287
## B B B B B B B B B B B B
## 9289 9296 9297 9303 9305 9306 9308 9316 9318 9319 9321 9327
## B B B B B B B B B B B B
## 9328 9329 9330 9332 9334 9342 9346 9352 9353 9355 9357 9358
## B B B B B B B B B B B B
## 9359 9364 9369 9371 9373 9379 9387 9388 9394 9395 9402 9408
## B B B B B C C C C C C C
## 9414 9429 9430 9434 9435 9446 9447 9452 9457 9462 9469 9470
## C C C C C C C C C C C C
## 9471 9473 9475 9481 9485 9487 9490 9492 9495 9499 9502 9503
## C C C C C C C C C C C C
## 9508 9510 9511 9518 9521 9527 9530 9531 9537 9539 9543 9561
## C C C C C C C C C C C C
## 9564 9566 9570 9576 9583 9585 9588 9591 9592 9600 9603 9605
## C C C C C C C C C C C C
## 9607 9608 9611 9614 9615 9616 9618 9620 9624 9625 9628 9633
## C C C C C C C C C C C C
## 9635 9637 9642 9644 9645 9646 9655 9660 9665 9666 9673 9676
## C C C C C C C C C C C C
## 9677 9681 9682 9684 9685 9689 9690 9691 9692 9693 9694 9702
## C C C C C C C C C C C C
## 9707 9708 9712 9714 9717 9718 9721 9734 9743 9745 9749 9750
## C C C C C C C C C C C C
## 9751 9752 9757 9759 9770 9778 9783 9784 9786 9789 9796 9797
## C C C C C C C C C C C C
## 9798 9802 9805 9810 9811 9812 9814 9816 9818 9819 9824 9825
## C C C C C C C C C C C C
## 9827 9829 9833 9835 9840 9841 9843 9844 9845 9849 9852 9861
## C C C C C C C C C C C C
## 9863 9868 9874 9879 9883 9885 9888 9889 9898 9901 9908 9910
## C C C C C C C C C C C C
## 9913 9924 9926 9929 9930 9931 9932 9934 9935 9938 9942 9944
## C C C C C C C C C C C C
## 9945 9946 9954 9956 9957 9959 9963 9966 9969 9970 9971 9972
## C C C C C C C C C C C C
## 9976 9982 9983 9984 9985 9986 9988 9990 9994 10002 10009 10013
## C C C C C C C C C C C C
## 10016 10022 10026 10027 10034 10036 10039 10046 10047 10051 10053 10054
## C C C C C C C C C C C C
## 10055 10062 10064 10067 10069 10071 10072 10074 10076 10078 10080 10082
## C C C C C C C C C C C C
## 10083 10086 10089 10091 10092 10094 10095 10096 10097 10098 10102 10105
## C C C C C C C C C C C C
## 10108 10114 10115 10120 10121 10123 10124 10131 10140 10141 10143 10144
## C C C C C C C C C C C C
## 10145 10149 10151 10153 10156 10157 10160 10164 10165 10169 10177 10180
## C C C C C C C C C C C C
## 10182 10183 10185 10187 10197 10199 10203 10210 10211 10212 10221 10224
## C C C C C C C C C C C C
## 10226 10228 10229 10230 10231 10232 10234 10236 10238 10244 10246 10249
## C C C C C C C C C C C C
## 10253 10258 10262 10267 10270 10274 10276 10281 10282 10283 10284 10285
## C C C C C C C C C C C C
## 10287 10296 10298 10300 10301 10302 10303 10305 10308 10309 10312 10313
## C C C C C C C C C C C C
## 10316 10317 10323 10324 10326 10327 10329 10332 10335 10340 10346 10349
## C C C C C C C C C C C C
## 10350 10352 10358 10363 10364 10368 10372 10376 10379 10385 10388 10391
## C C C C C C C C C C C C
## 10399 10401 10403 10404 10410 10413 10414 10415 10417 10418 10420 10421
## C C C C C C C C C C C C
## 10422 10424 10426 10427 10428 10432 10443 10447 10449 10455 10463 10468
## C C C C C C C C C C C C
## 10471 10478 10479 10480 10484 10485 10486 10488 10489 10491 10495 10497
## C C C C C C C C C C C C
## 10525 10531 10533 10536 10541 10543 10544 10548 10549 10550 10555 10556
## C C C C C C C C C C C C
## 10572 10577 10578 10581 10583 10584 10586 10587 10588 10590 10592 10597
## C C C C C C C C C C C C
## 10599 10604 10606 10608 10611 10614 10627 10633 10634 10636 10638 10639
## C C C C C C C C C C C C
## 10641 10642 10647 10657 10659 10664 10669 10672 10675 10676 10677 10682
## C C C C C C C C C C C C
## 10687 10693 10703 10708 10710 10713 10722 10726 10727 10730 10731 10732
## C C C C C C C C C C C C
## 10741 10744 10745 10747 10748 10751 10754 10760 10761 10762 10763 10764
## C C C C C C C C C C C C
## 10766 10769 10776 10779 10783 10789 10793 10795 10797 10803 10805 10808
## C C C C C C C C C C C C
## 10809 10810 10817 10821 10826 10831 10832 10834 10847 10848 10849 10855
## C C C C C C C C C C C C
## 10857 10859 10860 10868 10869 10874 10875 10877 10883 10885 10886 10888
## C C C C C C C C C C C C
## 10890 10893 10897 10900 10901 10906 10908 10910 10911 10916 10917 10922
## C C C C C C C C C C C C
## 10938 10939 10946 10947 10953 10958 10960 10962 10966 10967 10969 10970
## C C C C C C C C C C C C
## 10972 10974 10976 10977 10979 10980 10981 10984 10986 10988 10991 10992
## C C C C C C C C C C C C
## 10993 11003 11005 11013 11015 11016 11019 11025 11031 11037 11039 11042
## C C C C C C C C C C C C
## 11043 11046 11047 11048 11051 11053 11054 11057 11064 11065 11069 11072
## C C C C C C C C C C C C
## 11073 11074 11076 11088 11092 11102 11105 11106 11107 11113 11115 11126
## C C C C C C C C C C C C
## 11128 11130 11132 11133 11137 11138 11142 11145 11157 11158 11170 11175
## C C C C C C C C C C C C
## 11180 11182 11183 11187 11188 11195 11196 11199 11203 11205 11210 11211
## C C C C C C C C C C C C
## 11213 11216 11217 11221 11223 11228 11242 11249 11254 11257 11258 11260
## C C C C C C C C C C C C
## 11261 11268 11269 11271 11273 11274 11280 11281 11283 11284 11290 11293
## C C C C C C C C C C C C
## 11295 11296 11298 11299 11301 11307 11309 11310 11314 11315 11316 11320
## C C C C C C C C C C C C
## 11324 11325 11328 11333 11335 11342 11344 11347 11348 11352 11360 11362
## C C C C C C C C C C C C
## 11363 11371 11372 11375 11379 11385 11386 11387 11396 11397 11398 11410
## C C C C C C C C C C C C
## 11414 11416 11419 11426 11432 11440 11441 11447 11449 11453 11454 11460
## C C C C C C C C C C C C
## 11464 11467 11479 11481 11484 11487 11493 11504 11505 11506 11507 11512
## C C C C C C C C C C C C
## 11513 11514 11515 11518 11520 11521 11522 11528 11540 11543 11544 11547
## C C C C C C C C C C C C
## 11552 11553 11555 11556 11557 11561 11563 11569 11570 11572 11576 11577
## C C C C C C C C C C C C
## 11580 11583 11584 11591 11605 11606 11615 11619 11621 11623 11628 11639
## C C C C C C C C C C C C
## 11649 11663 11671 11673 11675 11678 11679 11689 11690 11697 11699 11706
## C C C C C C C C C C C C
## 11711 11722 11728 11729 11740 11742 11745 11747 11748 11750 11752 11755
## C C C C C C C C C C C C
## 11757 11759 11762 11764 11765 11769 11770 11771 11772 11774 11775 11778
## C C C C C C C C C C C C
## 11780 11782 11783 11784 11790 11792 11793 11796 11797 11800 11803 11812
## C C C C C C C C C C C C
## 11817 11825 11829 11838 11841 11847 11849 11850 11854 11855 11856 11857
## C C C C C C C C C C C C
## 11863 11864 11865 11871 11878 11886 11889 11894 11898 11900 11904 11907
## C C C C C C C C C C C C
## 11910 11911 11917 11922 11936 11940 11951 11956 11958 11961 11966 11970
## C C C C C C C C C C C C
## 11972 11973 11977 11978 11980 11982 11985 11986 11988 11990 11992 11993
## C C C C C C C C C C C C
## 11998 11999 12002 12003 12005 12006 12008 12009 12012 12014 12017 12018
## C C C C C C C C C C C C
## 12024 12042 12043 12049 12050 12054 12055 12056 12058 12065 12067 12070
## C C C C C C C C C C C C
## 12071 12073 12074 12076 12077 12080 12084 12085 12089 12090 12091 12096
## C C C C C C C C C C C C
## 12097 12100 12101 12106 12108 12110 12116 12117 12122 12130 12135 12137
## C C C C C C C C C C C C
## 12139 12150 12151 12155 12157 12159 12160 12165 12167 12168 12171 12172
## C C C C C C C C C C C C
## 12176 12178 12179 12184 12186 12192 12194 12196 12198 12200 12201 12202
## C C C C C C C C C C C C
## 12203 12204 12206 12208 12210 12219 12221 12222 12226 12227 12228 12235
## C C C C C C C C C C C C
## 12236 12239 12241 12246 12248 12249 12253 12255 12256 12259 12264 12265
## C C C C C C C C C C C C
## 12271 12275 12276 12278 12280 12282 12286 12289 12290 12298 12315 12317
## C C C C C C C C C C C C
## 12319 12320 12322 12324 12325 12334 12335 12337 12341 12348 12350 12351
## C C C C C C C C C C C C
## 12352 12353 12355 12359 12361 12367 12373 12378 12379 12380 12384 12387
## C C C C C C C C C C C C
## 12388 12395 12396 12399 12405 12410 12413 12416 12421 12423 12426 12428
## C C C C C C C C C C C C
## 12429 12433 12437 12438 12446 12447 12449 12457 12459 12462 12465 12466
## C C C C C C C C C C C C
## 12479 12480 12481 12482 12486 12491 12492 12496 12501 12502 12512 12513
## C C C C C C C C C C C C
## 12515 12517 12521 12523 12525 12527 12528 12533 12534 12537 12538 12549
## C C C C C C C C C C C C
## 12553 12559 12562 12576 12585 12587 12588 12590 12595 12598 12603 12604
## C C C C C C C C C C C C
## 12608 12610 12611 12612 12620 12623 12625 12627 12635 12636 12638 12643
## C C C C C C C C C C C C
## 12651 12660 12661 12662 12663 12668 12670 12676 12682 12684 12685 12689
## C C C C C C C C C C C C
## 12691 12697 12698 12700 12702 12707 12708 12712 12713 12715 12716 12720
## C C C C C C C C C C C C
## 12732 12733 12734 12739 12741 12748 12752 12754 12756 12757 12758 12760
## C C C C C C C C C C C C
## 12763 12768 12769 12775 12777 12787 12788 12790 12793 12796 12799 12801
## C C C C C C C C C C C D
## 12806 12808 12812 12814 12818 12820 12821 12825 12829 12835 12837 12841
## D D D D D D D D D D D D
## 12847 12848 12853 12855 12860 12868 12869 12871 12872 12874 12877 12880
## D D D D D D D D D D D D
## 12882 12887 12888 12893 12894 12895 12898 12910 12925 12927 12929 12931
## D D D D D D D D D D D D
## 12933 12934 12935 12942 12943 12946 12949 12957 12958 12961 12962 12967
## D D D D D D D D D D D D
## 12968 12969 12971 12973 12974 12979 12983 12985 12987 12990 12993 12995
## D D D D D D D D D D D D
## 12996 12997 12998 13000 13005 13011 13012 13013 13016 13017 13021 13025
## D D D D D D D D D D D D
## 13026 13031 13035 13050 13067 13069 13078 13081 13083 13085 13087 13089
## D D D D D D D D D D D D
## 13096 13098 13099 13106 13107 13108 13110 13111 13112 13113 13115 13118
## D D D D D D D D D D D D
## 13124 13125 13131 13133 13134 13138 13144 13146 13147 13149 13151 13153
## D D D D D D D D D D D D
## 13157 13163 13168 13170 13181 13202 13206 13213 13216 13218 13220 13222
## D D D D D D D D D D D D
## 13223 13226 13227 13230 13231 13232 13235 13239 13242 13243 13244 13255
## D D D D D D D D D D D D
## 13259 13262 13265 13268 13269 13271 13278 13282 13283 13284 13285 13293
## D D D D D D D D D D D D
## 13296 13299 13302 13303 13305 13313 13314 13316 13328 13329 13333 13334
## D D D D D D D D D D D D
## 13337 13339 13341 13347 13348 13349 13360 13363 13364 13367 13368 13375
## D D D D D D D D D D D D
## 13380 13384 13388 13391 13396 13397 13399 13404 13405 13411 13418 13421
## D D D D D D D D D D D D
## 13422 13423 13429 13430 13433 13442 13447 13449 13451 13452 13459 13460
## D D D D D D D D D D D D
## 13465 13466 13467 13471 13472 13473 13477 13478 13484 13493 13495 13497
## D D D D D D D D D D D D
## 13503 13506 13516 13518 13520 13521 13526 13527 13530 13532 13534 13539
## D D D D D D D D D D D D
## 13544 13545 13546 13549 13550 13551 13555 13557 13558 13561 13566 13567
## D D D D D D D D D D D D
## 13569 13577 13581 13582 13584 13586 13588 13589 13590 13593 13609 13610
## D D D D D D D D D D D D
## 13611 13614 13620 13623 13624 13625 13626 13629 13631 13632 13633 13637
## D D D D D D D E D D D D
## 13639 13643 13644 13653 13655 13663 13665 13676 13679 13682 13685 13689
## D D D D D D D D D D D D
## 13691 13693 13694 13696 13697 13702 13703 13709 13719 13722 13730 13733
## D D D D D D D D D D D D
## 13735 13737 13739 13740 13742 13744 13745 13748 13754 13756 13757 13758
## D D D D D D D D D D D D
## 13759 13761 13766 13769 13774 13780 13782 13785 13787 13789 13790 13794
## D D D D D D D D D D D D
## 13795 13797 13798 13799 13810 13814 13817 13822 13824 13826 13835 13839
## D D D D D D D D D D D D
## 13841 13847 13852 13855 13856 13859 13860 13861 13862 13864 13867 13874
## D D D D D D D D D D D D
## 13877 13878 13885 13886 13892 13895 13897 13901 13903 13908 13909 13912
## D D D D D D D D D D D D
## 13914 13920 13925 13927 13928 13938 13939 13940 13943 13947 13952 13953
## D D D D D D D D D D D D
## 13957 13958 13964 13971 13973 13984 13985 13989 13991 13992 13995 13997
## D D D D D D D D D D D D
## 13998 14000 14001 14006 14011 14012 14014 14017 14018 14019 14025 14028
## D D D D D D D D D D D D
## 14029 14032 14034 14037 14040 14049 14054 14057 14058 14059 14060 14061
## D D D D D D D D D D D D
## 14063 14067 14073 14078 14081 14085 14087 14089 14094 14099 14109 14111
## D D D D D D D D D D D D
## 14113 14121 14123 14124 14125 14129 14141 14153 14154 14155 14161 14163
## D D D D D D D D D D D D
## 14172 14174 14175 14177 14178 14182 14195 14203 14204 14206 14207 14208
## D D D D D D D D D D D D
## 14209 14210 14212 14213 14215 14217 14226 14233 14241 14242 14248 14249
## D D D D D D D D D D D D
## 14252 14258 14261 14264 14272 14276 14279 14281 14283 14284 14286 14287
## D D D D D D D D D D D D
## 14291 14304 14305 14307 14308 14311 14313 14314 14315 14316 14321 14324
## D D D D D D D D D D D D
## 14326 14331 14333 14334 14339 14344 14351 14352 14354 14360 14364 14367
## D D D D D D D D D D D D
## 14370 14371 14372 14373 14378 14380 14381 14382 14383 14397 14401 14402
## C C C D D D D D D D D D
## 14414 14422 14426 14428 14429 14433 14437 14443 14444 14447 14450 14451
## D D D D D D D D D D D D
## 14455 14456 14459 14460 14464 14465 14471 14473 14476 14477 14478 14482
## D D D D D D D D D D D D
## 14485 14495 14499 14502 14506 14510 14511 14512 14513 14515 14521 14522
## D D D D D D D D D D D D
## 14525 14526 14530 14532 14534 14535 14538 14542 14546 14552 14553 14557
## D D D D D D D D D D D D
## 14564 14566 14572 14573 14574 14575 14577 14578 14580 14581 14585 14587
## D D D D D D D D D D D D
## 14591 14598 14600 14603 14607 14609 14610 14611 14612 14615 14618 14623
## D D D D D D D D D D D D
## 14626 14627 14630 14632 14635 14644 14646 14647 14651 14653 14658 14661
## D D D D D D D D D D D D
## 14662 14673 14674 14678 14686 14692 14694 14696 14698 14699 14701 14702
## D D D D D D D D D D D D
## 14703 14705 14706 14707 14708 14712 14713 14715 14719 14722 14723 14726
## D D D D D D D D D D D D
## 14738 14739 14741 14745 14746 14748 14752 14754 14756 14758 14765 14767
## D D D D D D D D D D D D
## 14771 14781 14786 14791 14792 14794 14795 14799 14800 14803 14804 14809
## D D D D D D D D D D D D
## 14810 14822 14826 14839 14840 14843 14844 14855 14858 14866 14869 14871
## D D D D D D D D D D D D
## 14872 14879 14884 14890 14894 14898 14910 14914 14916 14920 14923 14926
## D D D D D D D D D D D D
## 14927 14929 14930 14932 14934 14940 14941 14943 14945 14947 14954 14956
## D D D D D D D D D D D D
## 14959 14963 14965 14966 14971 14981 14986 14988 14990 14994 14995 14997
## D D D D D D D D D D D D
## 15005 15006 15009 15010 15015 15016 15019 15020 15021 15025 15028 15033
## D D D D D D D D D D D D
## 15035 15037 15038 15040 15044 15051 15057 15060 15064 15066 15069 15072
## D D D D D D D D D D D D
## 15073 15078 15082 15083 15084 15086 15087 15088 15091 15094 15095 15096
## D D D D D D D D D D D D
## 15097 15108 15111 15117 15120 15124 15125 15128 15129 15132 15134 15135
## D D D D D D D D D D D D
## 15137 15141 15142 15147 15148 15164 15165 15171 15172 15173 15179 15180
## D D D D D D D D D D D D
## 15182 15186 15187 15190 15199 15205 15206 15207 15211 15212 15213 15214
## D D D D D D D D D D D D
## 15216 15217 15218 15224 15225 15226 15240 15241 15244 15245 15249 15253
## D D D D D D D D D D D D
## 15261 15265 15271 15272 15273 15285 15287 15289 15290 15293 15298 15302
## D D D D D D D D D D D D
## 15307 15310 15313 15314 15317 15319 15321 15322 15327 15329 15336 15342
## D D D D D D C C D D D D
## 15345 15353 15354 15355 15358 15374 15381 15384 15390 15391 15406 15407
## D D D D D D D D D D D D
## 15416 15426 15427 15433 15434 15446 15448 15451 15454 15458 15459 15463
## D D D D D D D D D D D D
## 15468 15475 15476 15480 15481 15485 15486 15490 15497 15499 15500 15501
## D D D D D D D D D D D D
## 15502 15505 15511 15518 15520 15521 15522 15524 15530 15534 15535 15536
## D D D D D D D D D D D D
## 15538 15539 15540 15545 15548 15550 15558 15561 15576 15577 15578 15587
## D D D D D D D D D D D D
## 15596 15597 15599 15600 15609 15611 15614 15617 15619 15620 15625 15626
## D D D D D D D D D D D D
## 15628 15632 15634 15635 15636 15638 15640 15642 15644 15647 15652 15654
## D D D D D D D D D D D D
## 15660 15662 15666 15669 15672 15674 15675 15677 15678 15679 15682 15689
## D D D D D D D D D D D D
## 15690 15691 15695 15696 15700 15706 15712 15713 15714 15719 15726 15728
## D D D D D D D D D D D D
## 15731 15733 15734 15738 15740 15743 15745 15747 15748 15751 15758 15763
## D D D D D D D D D D D D
## 15766 15768 15771 15774 15777 15778 15779 15780 15782 15790 15795 15798
## D D D D D D D D D D D D
## 15801 15802 15804 15805 15808 15811 15814 15817 15818 15823 15837 15839
## D D D D D D D D D D D D
## 15841 15842 15843 15845 15853 15854 15857 15859 15860 15863 15864 15874
## D D D D D D D D D D D D
## 15877 15880 15881 15882 15884 15887 15889 15890 15891 15893 15894 15906
## D D D D D D D D D D D D
## 15908 15920 15932 15935 15939 15943 15946 15956 15958 15959 15960 15962
## D D D D D D D D D D D D
## 15963 15967 15971 15972 15975 15979 15980 15987 15988 15991 15995 16000
## D D D D D D D D D D D D
## 16001 16004 16007 16022 16026 16027 16032 16034 16040 16047 16048 16050
## D D D E E E E E E E E E
## 16054 16056 16067 16072 16074 16075 16079 16083 16084 16088 16089 16092
## E E E E E E E E E E E E
## 16093 16094 16097 16099 16102 16106 16107 16108 16110 16113 16114 16116
## E E E E E E E E E E E E
## 16121 16125 16127 16128 16137 16138 16142 16147 16155 16156 16159 16161
## E E E E E E E E E E E E
## 16163 16165 16166 16169 16173 16174 16179 16180 16181 16182 16183 16185
## E E E E E E E E E E E E
## 16188 16190 16191 16193 16202 16205 16207 16209 16211 16214 16218 16219
## E E E E E E E E E E E E
## 16224 16233 16235 16239 16242 16244 16247 16250 16254 16258 16264 16267
## E E E E E E E E E E E E
## 16269 16270 16271 16276 16277 16284 16288 16290 16296 16300 16301 16306
## E E E E E E E E E E E E
## 16307 16314 16316 16322 16323 16329 16331 16334 16335 16336 16337 16338
## E E E E E E E E E E E E
## 16341 16343 16344 16353 16361 16367 16368 16372 16373 16376 16384 16388
## E E E E E E E E E E E E
## 16393 16399 16403 16407 16411 16414 16417 16420 16422 16426 16433 16434
## E E E E E E E E E E E E
## 16436 16437 16438 16440 16442 16444 16445 16446 16448 16449 16458 16459
## E E E E E E E E E E E E
## 16460 16467 16474 16481 16482 16483 16484 16485 16488 16489 16492 16494
## E E E E E E E E E E E E
## 16496 16500 16501 16504 16506 16507 16508 16509 16511 16515 16516 16517
## E E E E E E E E E E E E
## 16519 16520 16524 16526 16527 16531 16532 16535 16537 16538 16543 16544
## E E E E E E E E E E E E
## 16553 16556 16558 16564 16566 16569 16572 16579 16582 16587 16589 16592
## E E E E E E E E E E E E
## 16593 16604 16606 16609 16614 16615 16619 16621 16622 16624 16626 16627
## E E E E E E E E E E E E
## 16629 16634 16635 16640 16644 16650 16652 16653 16654 16656 16657 16661
## E E E E E E E E E E E E
## 16665 16674 16681 16682 16685 16687 16692 16695 16704 16705 16708 16710
## E E E E E E E E E E E E
## 16713 16714 16715 16721 16722 16726 16728 16730 16731 16733 16734 16738
## E E E D E E E E E E E E
## 16742 16744 16747 16748 16749 16755 16756 16762 16764 16769 16772 16773
## E E E E E E E E E E E E
## 16774 16775 16777 16778 16779 16781 16783 16784 16785 16786 16791 16793
## E E E E E E E E E E E E
## 16795 16798 16799 16804 16808 16811 16814 16815 16821 16826 16830 16834
## E E E E E E E E E E E E
## 16842 16850 16854 16865 16870 16871 16872 16873 16878 16887 16890 16891
## E E E E E E E E E E E E
## 16893 16895 16899 16903 16906 16909 16912 16913 16925 16926 16932 16939
## E E E E E E E E E E E E
## 16948 16949 16950 16951 16955 16956 16959 16962 16966 16973 16977 16979
## E E E E E E E E E E E E
## 16983 16984 16987 16991 16996 16997 16998 17002 17008 17011 17012 17014
## E E E E E E E E E E E E
## 17015 17020 17021 17022 17025 17027 17028 17029 17037 17040 17043 17046
## E E E E E E E E E E E E
## 17050 17051 17057 17060 17062 17070 17075 17077 17079 17080 17083 17084
## E E E E E E E E E E E E
## 17085 17091 17092 17093 17094 17095 17096 17101 17102 17104 17108 17109
## E E E E E E E E E E E E
## 17110 17118 17119 17120 17130 17132 17133 17135 17153 17155 17159 17163
## E E E E E E E E E E E E
## 17165 17166 17167 17172 17176 17177 17184 17187 17189 17195 17198 17202
## E E E E E E E E E E E E
## 17206 17213 17214 17217 17220 17224 17231 17236 17237 17240 17246 17249
## E E E E E E E E E E E E
## 17251 17259 17261 17265 17266 17274 17277 17282 17284 17291 17294 17296
## E E E E E E E E E E E E
## 17302 17310 17312 17313 17314 17316 17321 17340 17343 17344 17345 17346
## E E E E E E E E E E E E
## 17348 17350 17353 17358 17360 17366 17369 17371 17374 17375 17383 17389
## E E E E E E E E E E E E
## 17392 17395 17396 17400 17402 17406 17407 17409 17412 17415 17416 17417
## E E E E E E E E E E E E
## 17418 17419 17420 17421 17424 17433 17437 17442 17443 17451 17454 17455
## E E E E E E E E E E E E
## 17458 17462 17469 17470 17472 17479 17481 17484 17485 17489 17492 17493
## E E E E E E E E E E E E
## 17500 17501 17506 17509 17511 17514 17521 17523 17534 17536 17538 17543
## E E E E E E E E E E E E
## 17550 17554 17555 17561 17564 17567 17568 17569 17570 17571 17572 17575
## E E E E E E E E E E E E
## 17578 17579 17580 17582 17584 17585 17587 17589 17592 17598 17599 17600
## E E E E E E E E E E E E
## 17601 17605 17606 17610 17614 17617 17618 17621 17627 17629 17631 17653
## E E E E E E E E E E E E
## 17658 17659 17663 17665 17666 17669 17671 17676 17679 17683 17687 17692
## E E E E E E E E E E E E
## 17704 17705 17706 17721 17722 17723 17729 17736 17737 17747 17753 17755
## E E E E E E E E E E E E
## 17769 17776 17779 17782 17784 17785 17787 17792 17796 17797 17798 17801
## E E E E E E E E E E E E
## 17802 17805 17807 17813 17814 17819 17820 17821 17822 17826 17827 17829
## E E E E E E E E E E E E
## 17834 17842 17845 17847 17851 17857 17866 17870 17871 17873 17877 17882
## E E E E E E E E E E E E
## 17883 17886 17888 17891 17892 17894 17897 17899 17902 17905 17908 17913
## E E E E E E E E E E E E
## 17915 17916 17919 17921 17923 17926 17928 17936 17942 17943 17953 17958
## E E E E E E E E E E E E
## 17959 17966 17969 17970 17974 17979 17980 17981 17983 17987 17991 17992
## E E E E E E E E E E E E
## 17997 17998 18000 18002 18004 18005 18006 18008 18013 18015 18021 18024
## E E E E E E E E E E E E
## 18027 18028 18031 18038 18049 18054 18058 18059 18060 18061 18062 18070
## E E E E E E E E E E E E
## 18076 18077 18080 18081 18084 18086 18090 18091 18094 18101 18102 18104
## E E E E E E E E E E E E
## 18107 18109 18112 18113 18116 18118 18122 18123 18126 18128 18131 18132
## E E E E E E E E E E E E
## 18135 18136 18141 18142 18146 18149 18151 18158 18160 18164 18167 18170
## E E E E E E E E E E E E
## 18171 18180 18183 18188 18192 18195 18200 18206 18207 18208 18211 18213
## E E E E E E E E E E E E
## 18217 18219 18226 18228 18230 18233 18241 18242 18244 18245 18248 18249
## E E E E E E E E E E E E
## 18257 18260 18263 18264 18269 18272 18273 18278 18279 18286 18292 18298
## E E E E E E E E E E E E
## 18300 18305 18312 18315 18322 18323 18332 18335 18339 18341 18346 18349
## E E E E E E E E E E E E
## 18352 18353 18357 18361 18363 18368 18371 18374 18380 18389 18390 18393
## E E E E E E E E E E E E
## 18394 18395 18403 18405 18407 18411 18412 18422 18433 18440 18441 18445
## E E E E E E E E E E E E
## 18447 18451 18455 18456 18457 18460 18461 18462 18464 18493 18497 18498
## E E E E E E E E E E E E
## 18504 18508 18509 18510 18517 18523 18525 18531 18538 18544 18552 18555
## E E E E E E E E E E E E
## 18563 18564 18566 18570 18572 18573 18576 18584 18588 18592 18600 18603
## E E E E E E E E E E E E
## 18607 18610 18615 18617 18619 18623 18624 18629 18632 18634 18637 18638
## E E E E E E E E E E E E
## 18639 18640 18643 18646 18647 18653 18659 18662 18664 18666 18671 18676
## E E E E E E E E E E E E
## 18679 18680 18681 18683 18686 18687 18694 18695 18697 18705 18714 18717
## E E E E E E E E E E E E
## 18718 18722 18723 18724 18725 18731 18736 18741 18744 18746 18750 18756
## E E E E E E E E E E E E
## 18762 18767 18768 18773 18779 18780 18785 18790 18799 18800 18803 18804
## E E E E E E E E E E E E
## 18808 18809 18811 18813 18818 18821 18826 18829 18830 18831 18834 18837
## E E E E E E E E E E E E
## 18848 18862 18865 18868 18869 18871 18872 18875 18876 18877 18879 18881
## E E E E E E E E E E E E
## 18894 18903 18904 18908 18910 18911 18916 18921 18926 18931 18932 18933
## E E E E E E E E E E E E
## 18946 18948 18954 18955 18957 18960 18963 18968 18970 18971 18974 18975
## E E E E E E E E E E E E
## 18977 18981 18984 18985 18986 18987 18990 18991 18994 18995 18999 19000
## E E E E E E E E E E E E
## 19004 19008 19009 19011 19012 19018 19025 19026 19034 19036 19039 19041
## E E E E E E E E E E E E
## 19045 19053 19058 19062 19063 19068 19070 19080 19081 19084 19086 19088
## E E E E E E E E E E E E
## 19091 19093 19095 19104 19107 19109 19111 19114 19116 19120 19124 19129
## E E E E E E E E E E E E
## 19132 19136 19138 19139 19152 19155 19163 19165 19166 19168 19170 19173
## E E E E E E E E E E E E
## 19174 19182 19183 19184 19187 19191 19192 19204 19207 19211 19219 19228
## E E E E E E E E E E E E
## 19232 19234 19239 19242 19244 19253 19254 19255 19256 19260 19263 19268
## E E E E E E E E E E E E
## 19270 19271 19272 19274 19276 19278 19291 19294 19295 19297 19298 19304
## E E E E E E E E E E E E
## 19311 19313 19315 19316 19318 19319 19320 19326 19327 19328 19329 19337
## E E E E E E E E E E E E
## 19345 19352 19356 19366 19371 19374 19376 19379 19382 19384 19386 19389
## E E E E E E E E E E E E
## 19393 19398 19399 19405 19409 19414 19415 19416 19417 19419 19422 19424
## E E E E E E E E E E E E
## 19425 19429 19431 19432 19438 19441 19446 19447 19449 19457 19461 19462
## E E E E E E E E E E E E
## 19465 19467 19468 19470 19480 19482 19485 19487 19488 19490 19496 19499
## E E E E E E E E E E E E
## 19501 19503 19506 19507 19513 19516 19517 19519 19522 19523 19524 19527
## E E E E E E E E E E E E
## 19543 19546 19549 19551 19563 19565 19566 19568 19572 19574 19577 19578
## E E E E E E E E E E E E
## 19581 19586 19588 19594 19596 19597 19600 19601 19606 19611 19612 19614
## E E E E E E E E E E E E
## 19615 19616 19618 19620 19622
## E E E E E
## Levels: A B C D ETo generate file texts with predictions to submit for assignment. Here a warning is necessary, the code will create about 5,885 text files, so handle carefully.
file_to_assignment <- function(x){
n=length(x)
for (i in 1:n) {
filename = paste0("problem_id_", i, ".txt")
write.table(x[i], file = filename, quote = FALSE, row.names = FALSE, col.names = FALSE)
}
}To create the files, use the function: file_to_assignment(prediction_results)