LBB Neural Network
Fashion Classification
Meilinie
Jun 25, 2019
Artificial neural networks used in many fields such as image recognition and handwriting classification. In this NN, we will develop sophisticated models for classification task. From the fashion data, we will let the machine to learn the model and predict what the given picture. After that we will check the accuracy and try to improve our model.
Before we work with Neural Network, Let’s load the library needed:
## Loading required package: lattice## Loading required package: ggplot21 Fashionmnist
Fashionmnist is the data given to be classified with Neural Network method. To describe the data, in simply word, we try to learn our database and automatically give label to every fashion items which come to our warehouse.
We have two datasets already, there are train & test. Data train will be used as the resource for the machine to learn and create the model. Otherwise, data test will be used to check whether our model has been sophisticated enough or not.
2 Data Explaratory
Prepare the data & do the cross validation by using two datasets: train and test
fashionmnist_train <- fread("data/fashionmnist/train.csv")
fashionmnist_test <- fread("data/fashionmnist/test.csv")Check the column names
## [1] "label" "pixel1" "pixel2" "pixel3" "pixel4" "pixel779"
## [7] "pixel780" "pixel781" "pixel782" "pixel783"Check the label proportion
## label pixel1 pixel2 pixel3 pixel4 pixel5 pixel6 pixel7 pixel8 pixel9
## 1: 2 0 0 0 0 0 0 0 0 0
## 2: 9 0 0 0 0 0 0 0 0 0
## 3: 6 0 0 0 0 0 0 0 5 0
## 4: 0 0 0 0 1 2 0 0 0 0
## 5: 3 0 0 0 0 0 0 0 0 0
## 6: 4 0 0 0 5 4 5 5 3 5
## pixel10 pixel11 pixel12 pixel13 pixel14 pixel15 pixel16 pixel17 pixel18
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 0 105 92 101 107 100 132 0
## 4: 0 114 183 112 55 23 72 102 165
## 5: 0 0 0 46 0 21 68 0 0
## 6: 6 2 0 0 0 0 0 0 0
## pixel19 pixel20 pixel21 pixel22 pixel23 pixel24 pixel25 pixel26 pixel27
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 2 4 0 1 0 0 0 0
## 4: 160 28 0 0 0 1 0 0 0
## 5: 0 0 0 0 0 0 0 0 0
## 6: 2 3 7 4 4 5 5 0 0
## pixel28 pixel29 pixel30 pixel31 pixel32 pixel33 pixel34 pixel35 pixel36
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 0 0 0
## 4: 0 0 0 0 0 1 0 0 24
## 5: 0 0 0 0 0 0 0 0 0
## 6: 0 0 0 0 4 5 4 5 11
## pixel37 pixel38 pixel39 pixel40 pixel41 pixel42 pixel43 pixel44 pixel45
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 0 150 227 211 171 191 236 228
## 4: 188 163 93 136 153 168 252 174 136
## 5: 25 187 189 0 119 173 239 112 0
## 6: 2 0 0 21 231 156 134 177 194
## pixel46 pixel47 pixel48 pixel49 pixel50 pixel51 pixel52 pixel53 pixel54
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 172 0 0 0 0 1 0 0 0
## 4: 166 130 123 131 66 0 0 1 0
## 5: 189 125 0 0 0 0 0 0 0
## 6: 1 0 0 0 10 5 5 4 0
## pixel55 pixel56 pixel57 pixel58 pixel59 pixel60 pixel61 pixel62 pixel63
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 0 0 0
## 4: 0 0 0 0 0 2 0 10 157
## 5: 0 0 0 0 0 0 0 0 0
## 6: 0 0 0 0 0 4 5 3 7
## pixel64 pixel65 pixel66 pixel67 pixel68 pixel69 pixel70 pixel71 pixel72
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 48 153 209 210 187 199 142 157 209
## 4: 216 226 208 142 66 115 149 230 190
## 5: 14 219 224 211 92 52 227 224 127
## 6: 0 0 26 112 168 162 212 226 199
## pixel73 pixel74 pixel75 pixel76 pixel77 pixel78 pixel79 pixel80 pixel81
## 1: 0 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 179 199 233 138 44 0 0 0 0
## 4: 196 198 172 222 107 165 211 0 0
## 5: 92 196 237 136 0 0 0 0 0
## 6: 146 168 124 25 0 0 0 4 5
## pixel82 pixel83 pixel84 pixel85 pixel86 pixel87 pixel88 pixel89 pixel90
## 1: 0 0 0 0 0 0 0 4 0
## 2: 0 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 0 0 0
## 4: 0 0 0 0 0 0 0 0 118
## 5: 0 0 0 0 0 0 0 0 0
## 6: 0 0 0 0 0 0 4 5 8
## pixel91 pixel92 pixel93 pixel94 pixel95 pixel96 pixel97 pixel98 pixel99
## 1: 0 0 0 0 62 61 21 29 23
## 2: 0 0 0 0 0 0 0 0 0
## 3: 120 218 215 207 198 198 223 219 214
## 4: 214 174 168 109 200 124 150 143 58
## 5: 0 106 238 202 205 224 225 217 218
## 6: 0 15 95 142 170 144 123 156 172
## pixel100 pixel101 pixel102 pixel103 pixel104 pixel105 pixel106 pixel107
## 1: 51 136 61 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 225 190 177 195 209 204 139 0
## 4: 63 89 137 97 168 138 142 195
## 5: 222 227 202 206 218 0 0 0
## 6: 140 131 135 160 137 141 53 0
## pixel108 pixel109 pixel110 pixel111 pixel112 pixel113 pixel114 pixel115
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 1 0 0 0 0 0 0 0
## 4: 156 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 0 3 0 0 0 0 0 0
## pixel116 pixel117 pixel118 pixel119 pixel120 pixel121 pixel122 pixel123
## 1: 0 0 0 0 88 201 228 225
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 208 200 197 202 193
## 4: 0 25 140 70 80 43 71 96
## 5: 0 0 0 0 166 239 216 214
## 6: 5 6 0 0 165 130 120 130
## pixel124 pixel125 pixel126 pixel127 pixel128 pixel129 pixel130 pixel131
## 1: 255 115 62 137 255 235 222 255
## 2: 0 0 0 0 0 0 0 0
## 3: 203 207 228 245 196 197 202 193
## 4: 93 151 121 197 143 107 82 101
## 5: 192 181 205 204 180 187 205 208
## 6: 56 124 122 68 143 111 98 132
## pixel132 pixel133 pixel134 pixel135 pixel136 pixel137 pixel138 pixel139
## 1: 135 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 190 167 203 27 0 0 0 0
## 4: 111 80 137 193 208 6 0 0
## 5: 230 45 0 0 0 0 0 0
## 6: 125 111 164 60 0 6 0 0
## pixel140 pixel141 pixel142 pixel143 pixel144 pixel145 pixel146 pixel147
## 1: 0 0 0 0 0 0 47 252
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 89 217
## 4: 0 0 0 0 0 74 194 107
## 5: 0 0 0 0 0 0 0 0
## 6: 0 0 0 0 3 7 0 99
## pixel148 pixel149 pixel150 pixel151 pixel152 pixel153 pixel154 pixel155
## 1: 234 238 224 215 215 229 108 180
## 2: 0 0 0 0 0 1 1 0
## 3: 200 211 175 189 205 199 218 216
## 4: 146 178 185 182 77 185 218 210
## 5: 220 235 233 213 205 201 185 184
## 6: 165 135 154 98 66 130 141 114
## pixel156 pixel157 pixel158 pixel159 pixel160 pixel161 pixel162 pixel163
## 1: 207 214 224 231 249 254 45 0
## 2: 0 0 0 81 21 0 1 0
## 3: 181 179 185 192 198 184 198 121
## 4: 175 174 235 217 217 129 180 210
## 5: 200 199 223 224 232 101 0 0
## 6: 143 113 71 107 147 117 176 172
## pixel164 pixel165 pixel166 pixel167 pixel168 pixel169 pixel170 pixel171
## 1: 0 0 0 0 0 0 0 1
## 2: 1 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 0 0
## 4: 208 89 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 0 5 0 0 0 0 0 0
## pixel172 pixel173 pixel174 pixel175 pixel176 pixel177 pixel178 pixel179
## 1: 0 0 214 222 210 213 224 225
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 140 220 204 223 187 182
## 4: 0 179 213 203 177 228 192 193
## 5: 0 0 0 0 252 238 226 226
## 6: 4 0 0 197 188 169 148 146
## pixel180 pixel181 pixel182 pixel183 pixel184 pixel185 pixel186 pixel187
## 1: 217 220 254 233 219 221 217 223
## 2: 1 0 0 0 0 0 14 200
## 3: 200 207 217 213 210 207 180 161
## 4: 162 143 172 196 205 181 180 140
## 5: 221 224 195 198 223 229 205 202
## 6: 132 110 133 107 139 131 133 141
## pixel188 pixel189 pixel190 pixel191 pixel192 pixel193 pixel194 pixel195
## 1: 221 240 254 0 0 1 0 0
## 2: 49 0 0 0 0 0 0 0
## 3: 192 184 199 162 0 0 0 0
## 4: 134 176 194 171 170 65 0 0
## 5: 235 158 0 0 0 0 0 0
## 6: 156 143 170 186 0 0 0 0
## pixel196 pixel197 pixel198 pixel199 pixel200 pixel201 pixel202 pixel203
## 1: 0 1 0 0 0 128 237 207
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 166 218
## 4: 0 0 0 0 0 184 194 229
## 5: 0 0 0 0 0 0 0 9
## 6: 0 0 0 0 6 0 0 225
## pixel204 pixel205 pixel206 pixel207 pixel208 pixel209 pixel210 pixel211
## 1: 224 224 207 216 214 210 208 211
## 2: 0 2 1 1 1 0 1 0
## 3: 203 195 185 196 189 167 181 206
## 4: 209 176 198 129 227 225 140 196
## 5: 228 241 125 238 228 225 212 215
## 6: 130 140 169 148 110 45 117 137
## pixel212 pixel213 pixel214 pixel215 pixel216 pixel217 pixel218 pixel219
## 1: 221 208 219 213 226 211 237 150
## 2: 0 0 162 184 165 20 0 1
## 3: 177 176 181 165 187 179 189 183
## 4: 130 179 145 109 79 182 223 164
## 5: 222 250 168 141 243 197 0 0
## 6: 118 53 97 147 147 154 133 202
## pixel220 pixel221 pixel222 pixel223 pixel224 pixel225 pixel226 pixel227
## 1: 0 0 0 0 0 0 2 0
## 2: 1 3 4 1 0 0 0 0
## 3: 0 0 0 0 0 0 0 0
## 4: 195 233 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 18 0 0 0 0 0 0 0
## pixel228 pixel229 pixel230 pixel231 pixel232 pixel233 pixel234 pixel235
## 1: 0 237 222 215 207 210 212 213
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 196 215 211 206 201 196
## 4: 38 180 177 213 202 159 129 98
## 5: 0 0 0 51 232 247 34 225
## 6: 7 0 48 211 135 124 140 83
## pixel236 pixel237 pixel238 pixel239 pixel240 pixel241 pixel242 pixel243
## 1: 206 214 213 214 213 210 215 214
## 2: 1 2 0 2 0 120 183 154
## 3: 180 188 210 206 190 173 180 187
## 4: 179 149 90 187 211 61 134 91
## 5: 237 219 222 223 216 248 182 124
## 6: 158 145 160 126 151 136 156 113
## pixel244 pixel245 pixel246 pixel247 pixel248 pixel249 pixel250 pixel251
## 1: 206 199 218 255 13 0 2 0
## 2: 206 32 0 5 5 0 0 0
## 3: 192 184 192 199 0 0 0 0
## 4: 57 118 212 220 218 207 0 0
## 5: 246 200 0 0 0 0 0 0
## 6: 110 167 168 217 93 0 0 0
## pixel252 pixel253 pixel254 pixel255 pixel256 pixel257 pixel258 pixel259
## 1: 0 0 4 0 85 228 210 218
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 212 217
## 4: 0 0 0 0 114 154 142 182
## 5: 0 0 0 0 0 0 0 69
## 6: 0 0 0 0 6 0 104 219
## pixel260 pixel261 pixel262 pixel263 pixel264 pixel265 pixel266 pixel267
## 1: 200 211 208 203 215 210 209 209
## 2: 0 0 0 0 2 0 3 0
## 3: 224 211 216 201 189 184 202 210
## 4: 219 130 88 81 52 54 106 93
## 5: 236 255 0 218 240 219 221 222
## 6: 208 166 157 130 170 137 144 155
## pixel268 pixel269 pixel270 pixel271 pixel272 pixel273 pixel274 pixel275
## 1: 210 213 211 210 217 206 213 231
## 2: 76 227 151 139 158 0 0 0
## 3: 194 182 173 173 176 192 195 208
## 4: 110 159 222 227 83 117 253 218
## 5: 214 252 153 71 248 215 0 0
## 6: 160 150 160 158 120 194 220 213
## pixel276 pixel277 pixel278 pixel279 pixel280 pixel281 pixel282 pixel283
## 1: 175 0 0 0 0 0 0 0
## 2: 0 0 66 23 0 0 0 0
## 3: 0 0 0 0 0 0 0 0
## 4: 210 206 48 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 128 0 0 0 0 0 0 0
## pixel284 pixel285 pixel286 pixel287 pixel288 pixel289 pixel290 pixel291
## 1: 217 224 215 206 205 204 217 230
## 2: 0 0 0 0 0 1 0 1
## 3: 0 0 222 215 228 203 212 199
## 4: 18 127 208 228 185 172 240 91
## 5: 0 0 0 82 248 247 0 231
## 6: 6 0 133 213 235 206 169 194
## pixel292 pixel293 pixel294 pixel295 pixel296 pixel297 pixel298 pixel299
## 1: 222 215 224 233 228 232 228 224
## 2: 0 0 0 55 224 162 143 121
## 3: 198 191 186 228 195 191 161 178
## 4: 126 208 165 154 213 214 229 215
## 5: 236 219 221 221 212 253 109 0
## 6: 139 76 119 178 139 99 120 199
## pixel300 pixel301 pixel302 pixel303 pixel304 pixel305 pixel306 pixel307
## 1: 207 212 215 213 229 31 0 4
## 2: 204 236 144 21 74 90 190 8
## 3: 177 198 194 210 29 0 0 0
## 4: 175 222 204 153 130 125 39 0
## 5: 247 224 19 0 0 0 0 0
## 6: 167 190 226 210 152 0 0 0
## pixel308 pixel309 pixel310 pixel311 pixel312 pixel313 pixel314 pixel315
## 1: 0 1 0 21 225 212 212 203
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 244 215
## 4: 0 0 0 0 0 0 0 28
## 5: 0 0 0 0 0 0 0 106
## 6: 0 0 0 0 4 0 165 209
## pixel316 pixel317 pixel318 pixel319 pixel320 pixel321 pixel322 pixel323
## 1: 211 225 193 139 136 195 147 156
## 2: 0 0 1 0 0 0 42 228
## 3: 235 206 214 202 171 201 178 203
## 4: 0 212 228 170 221 205 225 228
## 5: 252 198 0 250 230 220 219 219
## 6: 233 237 131 158 151 124 147 157
## pixel324 pixel325 pixel326 pixel327 pixel328 pixel329 pixel330 pixel331
## 1: 139 128 162 197 223 207 220 213
## 2: 160 148 125 165 223 208 221 246
## 3: 190 180 187 196 183 206 195 206
## 4: 210 178 214 89 117 213 0 0
## 5: 212 252 129 0 255 232 59 0
## 6: 149 108 137 187 97 147 233 204
## pixel332 pixel333 pixel334 pixel335 pixel336 pixel337 pixel338 pixel339
## 1: 232 177 0 0 0 0 0 123
## 2: 211 174 151 0 0 0 0 0
## 3: 44 0 0 0 0 0 0 0
## 4: 0 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 184 0 0 0 0 0 0 0
## pixel340 pixel341 pixel342 pixel343 pixel344 pixel345 pixel346 pixel347
## 1: 226 207 211 209 205 228 158 90
## 2: 1 3 0 1 0 3 0 9
## 3: 0 0 233 221 237 182 206 207
## 4: 0 0 0 0 0 196 141 56
## 5: 0 0 0 134 252 150 0 254
## 6: 0 0 199 202 233 245 165 76
## pixel348 pixel349 pixel350 pixel351 pixel352 pixel353 pixel354 pixel355
## 1: 103 186 138 100 121 147 158 183
## 2: 0 23 230 168 147 119 163 227
## 3: 195 200 192 224 191 182 177 180
## 4: 176 210 164 162 91 135 196 158
## 5: 215 219 219 219 216 233 222 0
## 6: 138 174 160 148 159 155 163 125
## pixel356 pixel357 pixel358 pixel359 pixel360 pixel361 pixel362 pixel363
## 1: 226 208 214 209 216 255 13 0
## 2: 206 200 190 206 186 139 171 41
## 3: 186 178 196 207 60 0 0 0
## 4: 69 181 0 0 3 2 3 0
## 5: 253 237 89 0 0 0 0 0
## 6: 90 230 228 203 210 0 0 0
## pixel364 pixel365 pixel366 pixel367 pixel368 pixel369 pixel370 pixel371
## 1: 1 0 0 226 219 202 208 206
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 16 249 224
## 4: 0 0 0 0 0 0 1 0
## 5: 0 0 0 0 0 0 0 163
## 6: 0 0 0 0 0 0 218 196
## pixel372 pixel373 pixel374 pixel375 pixel376 pixel377 pixel378 pixel379
## 1: 205 216 184 156 150 193 170 164
## 2: 0 3 4 0 16 209 185 139
## 3: 229 192 215 207 198 198 192 226
## 4: 0 150 190 88 50 145 194 159
## 5: 253 126 115 253 211 222 220 219
## 6: 238 249 204 101 143 184 171 149
## pixel380 pixel381 pixel382 pixel383 pixel384 pixel385 pixel386 pixel387
## 1: 168 188 186 200 219 216 213 213
## 2: 118 181 227 209 204 210 199 225
## 3: 193 192 180 176 185 196 200 217
## 4: 120 136 207 230 144 171 4 0
## 5: 219 213 253 54 206 243 144 0
## 6: 163 173 158 103 132 255 217 205
## pixel388 pixel389 pixel390 pixel391 pixel392 pixel393 pixel394 pixel395
## 1: 211 233 148 0 0 0 45 227
## 2: 157 127 190 111 0 1 3 2
## 3: 87 0 0 0 0 0 0 0
## 4: 1 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 222 0 0 0 0 0 0 0
## pixel396 pixel397 pixel398 pixel399 pixel400 pixel401 pixel402 pixel403
## 1: 204 214 211 218 222 221 230 229
## 2: 1 3 2 3 0 0 0 67
## 3: 0 53 222 223 235 205 220 200
## 4: 0 0 0 0 0 187 220 182
## 5: 0 0 0 205 255 149 209 234
## 6: 0 0 234 193 237 255 169 139
## pixel404 pixel405 pixel406 pixel407 pixel408 pixel409 pixel410 pixel411
## 1: 221 213 224 233 226 220 219 221
## 2: 214 187 139 111 191 229 203 208
## 3: 151 206 198 214 199 171 161 197
## 4: 72 139 199 192 232 255 244 198
## 5: 219 221 220 219 219 213 244 164
## 6: 199 182 181 154 171 186 181 154
## pixel412 pixel413 pixel414 pixel415 pixel416 pixel417 pixel418 pixel419
## 1: 224 223 217 210 218 213 254 0
## 2: 208 196 223 188 100 128 155 112
## 3: 186 218 198 216 88 0 0 0
## 4: 170 189 2 0 3 0 0 0
## 5: 224 240 195 0 0 0 0 0
## 6: 148 242 216 208 220 0 0 0
## pixel420 pixel421 pixel422 pixel423 pixel424 pixel425 pixel426 pixel427
## 1: 0 0 157 226 203 207 211 209
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 75 223 223
## 4: 0 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 9
## 6: 0 0 0 0 0 7 239 190
## pixel428 pixel429 pixel430 pixel431 pixel432 pixel433 pixel434 pixel435
## 1: 215 205 198 207 208 201 201 197
## 2: 0 48 161 218 171 147 116 185
## 3: 237 199 219 203 193 204 201 220
## 4: 0 186 175 181 93 164 230 134
## 5: 0 20 253 219 221 219 219 219
## 6: 243 231 103 164 144 88 188 157
## pixel436 pixel437 pixel438 pixel439 pixel440 pixel441 pixel442 pixel443
## 1: 203 205 210 207 213 214 214 214
## 2: 235 201 209 203 204 220 208 126
## 3: 201 193 176 189 167 218 202 198
## 4: 153 142 137 79 143 183 0 0
## 5: 219 221 224 230 0 0 0 0
## 6: 177 154 118 181 105 226 223 210
## pixel444 pixel445 pixel446 pixel447 pixel448 pixel449 pixel450 pixel451
## 1: 213 208 234 107 0 0 235 213
## 2: 133 162 164 187 0 0 12 16
## 3: 109 0 0 0 0 0 0 0
## 4: 2 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 219 0 0 0 0 0 0 0
## pixel452 pixel453 pixel454 pixel455 pixel456 pixel457 pixel458 pixel459
## 1: 204 211 210 209 213 202 197 204
## 2: 9 4 19 73 170 211 188 145
## 3: 0 79 250 226 243 196 222 201
## 4: 0 0 0 0 0 93 72 144
## 5: 0 0 0 0 0 49 254 217
## 6: 0 30 242 187 246 196 93 162
## pixel460 pixel461 pixel462 pixel463 pixel464 pixel465 pixel466 pixel467
## 1: 215 217 213 212 210 206 212 203
## 2: 149 148 136 222 218 207 208 215
## 3: 206 205 207 222 185 194 186 187
## 4: 138 164 113 124 98 80 57 97
## 5: 221 219 219 219 220 221 216 253
## 6: 115 91 187 151 182 114 65 158
## pixel468 pixel469 pixel470 pixel471 pixel472 pixel473 pixel474 pixel475
## 1: 211 218 215 214 208 209 222 230
## 2: 224 193 140 146 162 156 118 203
## 3: 144 207 208 226 111 0 0 0
## 4: 138 124 4 0 3 0 0 0
## 5: 7 0 0 0 0 0 0 0
## 6: 116 176 220 216 223 0 0 0
## pixel476 pixel477 pixel478 pixel479 pixel480 pixel481 pixel482 pixel483
## 1: 0 52 255 207 200 208 213 210
## 2: 37 47 255 188 187 182 203 206
## 3: 0 0 0 0 0 87 249 222
## 4: 0 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 6
## 6: 0 0 0 0 0 46 206 187
## pixel484 pixel485 pixel486 pixel487 pixel488 pixel489 pixel490 pixel491
## 1: 210 208 207 202 201 209 216 216
## 2: 182 147 140 148 148 148 134 180
## 3: 240 226 193 187 205 206 198 212
## 4: 0 149 171 98 70 180 152 227
## 5: 0 94 229 216 220 218 218 219
## 6: 237 156 106 186 180 178 173 160
## pixel492 pixel493 pixel494 pixel495 pixel496 pixel497 pixel498 pixel499
## 1: 216 216 214 212 205 215 201 228
## 2: 218 221 225 204 156 134 139 144
## 3: 190 189 185 185 191 211 212 210
## 4: 173 143 180 183 127 206 38 0
## 5: 220 219 218 226 52 0 5 0
## 6: 176 180 161 188 136 133 226 219
## pixel500 pixel501 pixel502 pixel503 pixel504 pixel505 pixel506 pixel507
## 1: 208 214 212 218 25 118 217 201
## 2: 140 123 97 206 30 88 203 164
## 3: 113 0 0 0 0 0 0 0
## 4: 5 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 219 20 0 0 0 0 0 0
## pixel508 pixel509 pixel510 pixel511 pixel512 pixel513 pixel514 pixel515
## 1: 206 208 213 208 205 206 210 211
## 2: 167 170 161 147 147 156 159 153
## 3: 0 95 249 227 236 219 213 186
## 4: 0 1 0 0 0 195 210 226
## 5: 0 0 0 4 0 119 231 214
## 6: 0 76 208 185 249 152 149 171
## pixel516 pixel517 pixel518 pixel519 pixel520 pixel521 pixel522 pixel523
## 1: 202 199 207 208 209 210 207 210
## 2: 152 155 150 134 136 153 145 136
## 3: 203 219 198 206 218 194 177 181
## 4: 113 187 224 210 191 181 224 212
## 5: 220 217 218 219 220 221 217 224
## 6: 110 163 174 160 167 190 138 149
## pixel524 pixel525 pixel526 pixel527 pixel528 pixel529 pixel530 pixel531
## 1: 210 245 139 119 255 202 203 236
## 2: 151 164 168 170 166 153 130 200
## 3: 195 148 229 206 122 0 0 0
## 4: 198 172 36 0 6 0 0 0
## 5: 34 0 3 0 0 0 0 0
## 6: 154 147 225 216 206 45 0 0
## pixel532 pixel533 pixel534 pixel535 pixel536 pixel537 pixel538 pixel539
## 1: 114 171 238 212 203 220 216 217
## 2: 10 131 195 150 133 132 142 160
## 3: 0 0 0 0 0 106 248 228
## 4: 0 0 0 0 0 1 0 0
## 5: 0 0 0 0 0 0 0 3
## 6: 0 0 0 0 0 89 202 186
## pixel540 pixel541 pixel542 pixel543 pixel544 pixel545 pixel546 pixel547
## 1: 209 207 205 210 211 206 204 206
## 2: 166 162 153 152 153 151 154 163
## 3: 224 206 202 200 206 191 190 194
## 4: 0 153 197 171 175 161 171 199
## 5: 0 113 231 213 222 217 218 219
## 6: 226 152 167 122 74 151 187 149
## pixel548 pixel549 pixel550 pixel551 pixel552 pixel553 pixel554 pixel555
## 1: 209 211 215 210 206 221 242 0
## 2: 147 152 163 205 208 174 175 178
## 3: 203 196 180 180 185 148 228 214
## 4: 224 187 206 192 176 179 48 0
## 5: 220 219 219 219 10 0 2 0
## 6: 166 182 113 71 133 168 208 218
## pixel556 pixel557 pixel558 pixel559 pixel560 pixel561 pixel562 pixel563
## 1: 224 234 230 181 26 39 145 201
## 2: 181 168 150 196 1 155 228 197
## 3: 117 0 0 0 0 0 0 0
## 4: 6 1 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 179 25 0 0 0 0 0 0
## pixel564 pixel565 pixel566 pixel567 pixel568 pixel569 pixel570 pixel571
## 1: 255 157 115 250 200 207 206 207
## 2: 172 138 123 121 122 126 129 135
## 3: 0 110 242 225 239 218 204 201
## 4: 0 0 0 0 0 180 195 136
## 5: 0 0 0 3 0 51 227 215
## 6: 0 108 193 180 197 152 155 160
## pixel572 pixel573 pixel574 pixel575 pixel576 pixel577 pixel578 pixel579
## 1: 213 216 206 205 206 207 206 215
## 2: 139 144 158 177 192 239 244 178
## 3: 202 211 193 197 217 203 183 184
## 4: 230 240 148 204 140 173 199 193
## 5: 221 217 217 219 220 218 219 213
## 6: 115 168 192 154 161 193 141 122
## pixel580 pixel581 pixel582 pixel583 pixel584 pixel585 pixel586 pixel587
## 1: 207 221 238 0 0 188 85 0
## 2: 176 183 179 185 177 147 148 192
## 3: 175 212 219 206 122 0 0 0
## 4: 156 213 37 0 5 0 0 0
## 5: 0 0 1 0 0 0 0 0
## 6: 159 156 193 213 170 29 0 0
## pixel588 pixel589 pixel590 pixel591 pixel592 pixel593 pixel594 pixel595
## 1: 0 0 0 0 31 0 129 253
## 2: 0 0 80 196 199 212 209 190
## 3: 0 0 0 0 0 122 249 234
## 4: 0 0 0 0 0 0 1 0
## 5: 0 0 0 0 0 0 0 2
## 6: 0 0 0 0 0 141 193 170
## pixel596 pixel597 pixel598 pixel599 pixel600 pixel601 pixel602 pixel603
## 1: 190 207 208 208 208 209 211 211
## 2: 176 170 178 183 192 204 204 202
## 3: 241 216 215 196 200 222 208 195
## 4: 0 150 101 72 167 158 95 177
## 5: 0 0 216 217 219 218 218 219
## 6: 169 151 161 163 153 168 198 155
## pixel604 pixel605 pixel606 pixel607 pixel608 pixel609 pixel610 pixel611
## 1: 209 209 209 212 201 226 165 0
## 2: 144 59 3 0 67 217 184 188
## 3: 225 212 181 186 183 227 214 211
## 4: 234 113 142 112 59 152 22 0
## 5: 219 218 220 246 0 0 2 0
## 6: 153 188 164 161 170 148 195 210
## pixel612 pixel613 pixel614 pixel615 pixel616 pixel617 pixel618 pixel619
## 1: 0 0 0 0 0 2 0 0
## 2: 178 166 160 188 13 0 0 0
## 3: 121 0 0 0 0 0 0 0
## 4: 3 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 188 60 0 0 0 0 0 0
## pixel620 pixel621 pixel622 pixel623 pixel624 pixel625 pixel626 pixel627
## 1: 0 0 89 254 199 199 192 196
## 2: 3 51 105 179 217 235 227 208
## 3: 0 148 230 237 228 214 213 201
## 4: 0 0 0 0 0 197 178 82
## 5: 0 0 0 2 0 0 217 216
## 6: 0 178 184 188 163 148 160 163
## pixel628 pixel629 pixel630 pixel631 pixel632 pixel633 pixel634 pixel635
## 1: 198 199 201 202 203 204 203 203
## 2: 201 133 58 0 0 0 0 0
## 3: 207 214 201 204 235 191 161 185
## 4: 47 64 106 112 121 110 189 225
## 5: 216 219 218 219 219 217 222 235
## 6: 164 162 202 155 156 187 168 152
## pixel636 pixel637 pixel638 pixel639 pixel640 pixel641 pixel642 pixel643
## 1: 200 222 155 0 3 3 3 2
## 2: 16 187 194 184 185 175 181 131
## 3: 180 216 201 212 119 0 0 0
## 4: 121 98 58 0 4 0 0 0
## 5: 0 0 2 0 0 0 0 0
## 6: 172 148 173 207 224 100 0 0
## pixel644 pixel645 pixel646 pixel647 pixel648 pixel649 pixel650 pixel651
## 1: 0 0 0 1 5 0 0 255
## 2: 0 0 3 0 0 0 0 0
## 3: 0 0 0 0 0 151 230 235
## 4: 0 0 0 0 0 0 2 0
## 5: 0 0 0 0 0 0 0 2
## 6: 0 0 0 0 0 188 181 204
## pixel652 pixel653 pixel654 pixel655 pixel656 pixel657 pixel658 pixel659
## 1: 218 226 232 228 224 222 220 219
## 2: 0 0 0 0 0 0 0 0
## 3: 229 218 209 198 193 226 182 189
## 4: 0 202 219 161 135 205 200 156
## 5: 0 0 212 216 217 219 218 219
## 6: 194 146 158 161 169 177 185 154
## pixel660 pixel661 pixel662 pixel663 pixel664 pixel665 pixel666 pixel667
## 1: 219 217 221 220 212 236 95 0
## 2: 5 5 5 0 0 0 0 0
## 3: 240 200 186 188 188 218 216 209
## 4: 195 231 234 218 182 223 99 0
## 5: 219 216 224 204 0 0 2 0
## 6: 148 168 169 157 161 155 188 209
## pixel668 pixel669 pixel670 pixel671 pixel672 pixel673 pixel674 pixel675
## 1: 2 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 110 0 0 0 0 0 0 0
## 4: 6 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 190 62 0 0 0 0 0 0
## pixel676 pixel677 pixel678 pixel679 pixel680 pixel681 pixel682 pixel683
## 1: 0 0 0 155 194 168 170 171
## 2: 0 0 0 0 0 0 0 0
## 3: 0 96 237 233 237 207 200 203
## 4: 0 0 4 0 0 188 152 118
## 5: 0 0 0 4 0 3 217 216
## 6: 0 117 198 189 80 216 201 205
## pixel684 pixel685 pixel686 pixel687 pixel688 pixel689 pixel690 pixel691
## 1: 173 173 179 177 175 172 171 167
## 2: 0 0 0 0 0 0 0 0
## 3: 191 213 170 186 239 209 186 173
## 4: 222 214 203 233 226 193 200 173
## 5: 217 219 218 220 220 218 222 209
## 6: 205 209 233 189 197 211 202 200
## pixel692 pixel693 pixel694 pixel695 pixel696 pixel697 pixel698 pixel699
## 1: 161 180 0 0 1 0 1 0
## 2: 0 0 0 0 0 0 0 0
## 3: 186 156 199 217 84 0 0 0
## 4: 53 166 97 0 6 0 0 0
## 5: 0 0 2 0 0 0 0 0
## 6: 201 210 150 220 197 72 0 0
## pixel700 pixel701 pixel702 pixel703 pixel704 pixel705 pixel706 pixel707
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 89 237 207
## 4: 0 0 0 0 0 0 3 0
## 5: 0 0 0 0 0 0 0 3
## 6: 0 0 0 0 0 151 214 181
## pixel708 pixel709 pixel710 pixel711 pixel712 pixel713 pixel714 pixel715
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 121 255 193 183 193 209 224 208
## 4: 2 182 152 51 89 174 183 168
## 5: 0 4 214 211 213 213 215 213
## 6: 6 153 82 56 30 3 0 73
## pixel716 pixel717 pixel718 pixel719 pixel720 pixel721 pixel722 pixel723
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 224 203 160 184 223 0 186 216
## 4: 112 109 181 170 136 108 60 0
## 5: 211 214 220 200 0 0 1 0
## 6: 74 41 28 30 46 78 13 201
## pixel724 pixel725 pixel726 pixel727 pixel728 pixel729 pixel730 pixel731
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 102 0 0 0 0 0 0 0
## 4: 4 0 0 0 0 0 0 0
## 5: 0 0 0 0 0 0 0 0
## 6: 146 39 0 0 0 0 0 0
## pixel732 pixel733 pixel734 pixel735 pixel736 pixel737 pixel738 pixel739
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 0 108 242 253 0 112 255 217
## 4: 0 0 2 0 5 194 193 204
## 5: 0 0 0 3 0 0 214 232
## 6: 0 7 34 58 0 0 0 0
## pixel740 pixel741 pixel742 pixel743 pixel744 pixel745 pixel746 pixel747
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 211 227 245 195 194 211 220 214
## 4: 104 116 241 217 196 171 249 207
## 5: 229 249 245 248 252 230 237 229
## 6: 0 0 0 0 0 0 0 0
## pixel748 pixel749 pixel750 pixel751 pixel752 pixel753 pixel754 pixel755
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 74 0 255 222 128 0 0 0
## 4: 197 202 45 0 3 0 0 0
## 5: 0 0 1 0 0 0 0 0
## 6: 0 0 0 0 0 0 0 0
## pixel756 pixel757 pixel758 pixel759 pixel760 pixel761 pixel762 pixel763
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 0 0 0 0 44 12
## 4: 0 0 0 0 0 0 0 1
## 5: 0 0 0 0 0 0 0 0
## 6: 0 0 0 0 4 0 0 0
## pixel764 pixel765 pixel766 pixel767 pixel768 pixel769 pixel770 pixel771
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 0 0 40 134 162 191 214 163
## 4: 0 0 0 22 21 25 69 52
## 5: 0 0 68 116 112 136 147 144
## 6: 2 4 9 7 8 7 5 4
## pixel772 pixel773 pixel774 pixel775 pixel776 pixel777 pixel778 pixel779
## 1: 0 0 0 0 0 0 0 0
## 2: 0 0 0 0 0 0 0 0
## 3: 146 165 79 0 0 0 30 43
## 4: 45 74 39 3 0 0 0 0
## 5: 121 102 63 0 0 0 0 0
## 6: 8 12 5 7 8 7 4 3
## pixel780 pixel781 pixel782 pixel783 pixel784
## 1: 0 0 0 0 0
## 2: 0 0 0 0 0
## 3: 0 0 0 0 0
## 4: 1 0 0 0 0
## 5: 0 0 0 0 0
## 6: 7 5 0 0 0##
## 0 1 2 3 4 5 6 7 8 9
## 6000 6000 6000 6000 6000 6000 6000 6000 6000 6000
From the plot above, we know that the data has been balanced, the observation already has equal amount for each label.
We would like to manually show the data:
m1 <- matrix(fashionmnist_train[1,2:ncol(fashionmnist_train)], nrow = 28, ncol = 28, byrow = T)
m1 <- apply(m1,2,as.numeric)
m1 <- apply(m1, 2, rev)
m1 <- t(m1)
image(m1)
Rather than show the image one by one, the vizTrain function will be used to show the few images & its label.
vizTrain <- function(input) {
dimmax <- sqrt(ncol(fashionmnist_train[, -1]))
dimn <- ceiling(sqrt(nrow(input)))
par(mfrow = c(dimn, dimn), mar = c(0.1, 0.1, 0.1,
0.1))
for (i in 1:nrow(input)) {
m1 <- matrix(input[i, 2:ncol(input)], nrow = dimmax,
byrow = T)
m1 <- apply(m1, 2, as.numeric)
m1 <- t(apply(m1, 2, rev))
image(1:dimmax, 1:dimmax, m1, col = grey.colors(255),
xaxt = "n", yaxt = "n")
cat <- sapply(as.character(fashionmnist_train[i,1]), switch,
"0" = "T-shirt",
"1" = "Trouser",
"2" = "Pullover",
"3" = "Dress",
"4" = "Coat",
"5" = "Sandal",
"6" = "Shirt",
"7" = "Sneaker",
"8" = "Bag",
"9" = "Boot")
text(2, 20, col = "white", cex = 1.2, cat)
}
}Show the picture from data train and give the label
We will prepare the data to create the model using Keras & MXNet. 1. Change the data into matrix
- Separate the predictor (x) and target (y / label) in order to check, we also show the dimension of each matrixes.
## [1] 60000 784## NULL3 Keras
Keras is high level API for working with other deep learning frameworks. It could runs on top of multiple popular back-ends; e.g, TensorFlow, Theano, CNTK. In this course material, we will learning the basic workflow in using Keras with TensorFlow on the back-end for several deep learning tasks.
In Keras, we have to change the data matrix into array first.
Transform the data into array:
train_x_keras <- array_reshape(train_x,c(nrow(train_x), ncol(train_x)))
test_x_keras <- array_reshape(test_x, c(nrow(test_x), ncol(test_x)))We would like to standardize the picture
Change the label (y / target) into categorical. We have 10 label (0-9), so we defined 10 categorical:
3.1 1st Model
Create the model
model <- keras_model_sequential()
model %>% layer_dense(units = 128, activation = "relu", input_shape = c(784)) %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 10, activation = "softmax")
model %>% compile(loss = "categorical_crossentropy",
optimizer = optimizer_sgd(), metrics = c("accuracy"))## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## dense (Dense) (None, 128) 100480
## ___________________________________________________________________________
## dense_1 (Dense) (None, 64) 8256
## ___________________________________________________________________________
## dense_2 (Dense) (None, 10) 650
## ===========================================================================
## Total params: 109,386
## Trainable params: 109,386
## Non-trainable params: 0
## ___________________________________________________________________________# hist <- model %>% fit(train_x_keras, train_y_keras, epochs = 30, batch_size = 128)
# saveRDS(hist, "history.RDS")
hist <- readRDS("history.RDS")
plot(hist)
From the model above we got 88,45% accuracy
Epoch 30/30 60000/60000 [==============================] - 4s 59us/sample - loss: 0.3530 - acc: 0.8762
Epoch 30/30 60000/60000 [==============================] - 3s 51us/sample - loss: 0.3575 - acc: 0.8755
Epoch 30/30 60000/60000 [==============================] - 4s 59us/sample - loss: 0.3246 - acc: 0.8845
We also can get accuracy from command below:
## [1] 0.88453.2 2nd Model
The accurary of the model has been good, but Let’s try to improve the accuracy by adding 1 hidden layer: Create the model1
model1 <- keras_model_sequential()
model1 %>% layer_dense(units = 256, activation = "relu", input_shape = c(784)) %>%
layer_dense(units = 128, activation = "relu") %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 10, activation = "softmax")
model1 %>% compile(loss = "categorical_crossentropy",
optimizer = optimizer_sgd(), metrics = c("accuracy"))## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## dense_3 (Dense) (None, 256) 200960
## ___________________________________________________________________________
## dense_4 (Dense) (None, 128) 32896
## ___________________________________________________________________________
## dense_5 (Dense) (None, 64) 8256
## ___________________________________________________________________________
## dense_6 (Dense) (None, 10) 650
## ===========================================================================
## Total params: 242,762
## Trainable params: 242,762
## Non-trainable params: 0
## ___________________________________________________________________________# history1 <- model1 %>% fit(train_x_keras, train_y_keras, epochs = 30, batch_size = 128)
# saveRDS(history1, "history1.RDS")
history1 <- readRDS("history1.RDS")
plot(history1)
Epoch 30/30 60000/60000 [==============================] - 4s 60us/sample - loss: 0.3225 - acc: 0.8847
Epoch 30/30 60000/60000 [==============================] - 4s 63us/sample - loss: 0.3101 - acc: 0.8898
60000/60000 [==============================] - 4s 65us/sample - loss: 0.3172 - acc: 0.8865
The improved model accurary is 88,65%
## [1] 0.88651673.3 3rd Model
Create the model2
model2 <- keras_model_sequential()
model2 %>% layer_dense(units = 256, activation = "relu", input_shape = c(784)) %>%
layer_dense(units = 128, activation = "relu") %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 10, activation = "softmax")
model2 %>% compile(loss = "categorical_crossentropy",
optimizer = 'rmsprop', metrics = c("accuracy"))## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## dense_7 (Dense) (None, 256) 200960
## ___________________________________________________________________________
## dense_8 (Dense) (None, 128) 32896
## ___________________________________________________________________________
## dense_9 (Dense) (None, 64) 8256
## ___________________________________________________________________________
## dense_10 (Dense) (None, 10) 650
## ===========================================================================
## Total params: 242,762
## Trainable params: 242,762
## Non-trainable params: 0
## ___________________________________________________________________________# history2 <- model2 %>% fit(train_x_keras, train_y_keras, epochs = 30, batch_size = 128)
# saveRDS(history2, "history2.RDS")
history2 <- readRDS("history2.RDS")
plot(history2)
Epoch 30/30 60000/60000 [==============================] - 4s 72us/sample - loss: 0.1648 - acc: 0.9400
After we used different optimizer = optimizer = ‘rmsprop’. The improved model accurary is 94%
## [1] 0.943.4 Prediction
We will predict our data test:
## [1] 7 1 7 8 7 7 1 7 1 1vizTest <- function(input) {
dimmax <- sqrt(ncol(fashionmnist_test[ ,-1]))
dimn <- ceiling(sqrt(nrow(input)))
par(mfrow = c(dimn, dimn), mar = c(0.1, 0.1, 0.1, 0.1))
for (i in 1:nrow(input)) {
m1 <- matrix(input[i, 2:ncol(input)], nrow = dimmax, byrow = T)
m1 <- apply(m1, 2, as.numeric)
m1 <- t(apply(m1, 2, rev))
image(1:dimmax, 1:dimmax, m1, col = grey.colors(255),
xaxt = "n", yaxt = "n")
cat <- sapply(as.character(fashionmnist_test[i,1]), switch,
"0" = "T-shirt",
"1" = "Trouser",
"2" = "Pullover",
"3" = "Dress",
"4" = "Coat",
"5" = "Sandal",
"6" = "Shirt",
"7" = "Sneaker",
"8" = "Bag",
"9" = "Boot")
text(2, 20, col = "white", cex = 1.2, cat)
}
}Show the prediction in data test
## Warning in levels(reference) != levels(data): longer object length is not a
## multiple of shorter object length## Warning in confusionMatrix.default(as.factor(prediction_test),
## as.factor(test_y)): Levels are not in the same order for reference and
## data. Refactoring data to match.## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 9
## 0 0 0 0 1 0 22 1 2 0 9
## 1 356 386 151 266 120 507 164 133 557 190
## 2 1 0 1 0 3 18 0 12 0 1
## 3 0 0 1 1 3 6 0 10 0 1
## 4 51 2 3 7 1 141 15 282 21 664
## 5 0 0 0 0 0 0 0 0 0 0
## 6 0 0 0 0 0 0 0 0 0 0
## 7 331 453 804 508 720 255 653 438 404 105
## 8 260 158 40 215 152 39 166 117 17 30
## 9 1 1 0 2 1 12 1 6 1 0
##
## Overall Statistics
##
## Accuracy : 0.0844
## 95% CI : (0.079, 0.09)
## No Information Rate : 0.1
## P-Value [Acc > NIR] : 1
##
## Kappa : -0.0173
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4
## Sensitivity 0.0000 0.3860 0.00100 0.00100 0.0010000
## Specificity 0.9961 0.7284 0.99611 0.99767 0.8682222
## Pos Pred Value 0.0000 0.1364 0.02778 0.04545 0.0008425
## Neg Pred Value 0.8996 0.9144 0.89974 0.89988 0.8866447
## Prevalence 0.1000 0.1000 0.10000 0.10000 0.1000000
## Detection Rate 0.0000 0.0386 0.00010 0.00010 0.0001000
## Detection Prevalence 0.0035 0.2830 0.00360 0.00220 0.1187000
## Balanced Accuracy 0.4981 0.5572 0.49856 0.49933 0.4346111
## Class: 5 Class: 6 Class: 7 Class: 8 Class: 9
## Sensitivity 0.0 0.0 0.43800 0.01700 0.0000
## Specificity 1.0 1.0 0.52967 0.86922 0.9972
## Pos Pred Value NaN NaN 0.09377 0.01424 0.0000
## Neg Pred Value 0.9 0.9 0.89454 0.88837 0.8997
## Prevalence 0.1 0.1 0.10000 0.10000 0.1000
## Detection Rate 0.0 0.0 0.04380 0.00170 0.0000
## Detection Prevalence 0.0 0.0 0.46710 0.11940 0.0025
## Balanced Accuracy 0.5 0.5 0.48383 0.44311 0.4986Conclusion
The 1st model is good and not overfitting. In model, we used 2 hidden layer (128,64 neuron) and 1 output layer (10 neuron). The accuracy for data train 88% and test 87,45% only have different gap.
4 Mxnet
MXNet is another machine learning algorithm that is Bayesian in nature is not as suited from parallelism. In contrast to model parallelism is the notion of data parallelism, where each device is assigned an equal proportion of the data and the output from each device is aggregated to produce the final model.
To remind our data, let’s recall the column names in our data again:
## [1] "label" "pixel1" "pixel2" "pixel3" "pixel4" "pixel779"
## [7] "pixel780" "pixel781" "pixel782" "pixel783"Checking our data distribution whether the sample of the label has been balance or not
tr_lab <- table(fashionmnist_train$label)
barplot(tr_lab, main="Distribution of Digits in Training Sample", col=gray.colors(10))
4.1 Model
We design the layers using other method: MXNet
m1.data <- mx.symbol.Variable("data")
#Hidden Layer
m1.fc1 <- mx.symbol.FullyConnected(m1.data, name="fc1", num_hidden=256)
m1.act1 <- mx.symbol.Activation(m1.fc1, name="activation1", act_type="relu")
m1.fc2 <- mx.symbol.FullyConnected(m1.act1, name="fc2", num_hidden=128)
m1.act2 <- mx.symbol.Activation(m1.fc2, name="activation2", act_type="relu")
m1.fc3 <- mx.symbol.FullyConnected(m1.act2, name="fc3", num_hidden=64)
m1.act3 <- mx.symbol.Activation(m1.fc3, name="activation1", act_type="relu")
#Output Layer
m1.fc4 <- mx.symbol.FullyConnected(m1.act3, name="fc4", num_hidden=10)
m1.softmax <- mx.symbol.SoftmaxOutput(m1.fc4, name="softMax")Show the curve of Softmax function
The Model in graph:
We do cross validation by using 2 data and set it into matrixes.
Because we will learn & predict, we separate the predictor(x) and target (y/ label):
train_x <- train_mx[,-1]
train_y <- train_mx[,1]
test_x <- test_mx[,-1]
test_y <- test_mx[,1]
train_x <- t(train_x/255)
test_x <- t(test_x/255)
dim(train_x)## [1] 784 60000## [1] 784 100004.1.1 1st Model
log <- mx.metric.logger$new()
startime <- proc.time()
mx.set.seed(0)
m1 <- mx.model.FeedForward.create(m1.softmax, #the network configuration made above
X = train_x, #input (predictors),
y = train_y, #the labels
ctx = mx.cpu(),
num.round = 50,
array.batch.size = 80,
momentum = 0.95,
array.layout="colmajor",
learning.rate = 0.001,
eval.metric = mx.metric.accuracy,
epoch.end.callback = mx.callback.log.train.metric(1,log)
)## Start training with 1 devices## [1] Train-accuracy=0.0979166671509544## [2] Train-accuracy=0.0973666670918465## [3] Train-accuracy=0.0976000004708767## [4] Train-accuracy=0.097566667119662## [5] Train-accuracy=0.0978333337778846## [6] Train-accuracy=0.0979666671231389## [7] Train-accuracy=0.0980666671122114## [8] Train-accuracy=0.0986166671092312## [9] Train-accuracy=0.0995000004644195## [10] Train-accuracy=0.10051666715617## [11] Train-accuracy=0.105850000320623## [12] Train-accuracy=0.115750000435859## [13] Train-accuracy=0.152266666742663## [14] Train-accuracy=0.207199999948343## [15] Train-accuracy=0.470383331775665## [16] Train-accuracy=0.585783332506816## [17] Train-accuracy=0.628316666007042## [18] Train-accuracy=0.6525166670084## [19] Train-accuracy=0.67264999906222## [20] Train-accuracy=0.68975## [21] Train-accuracy=0.709566667159398## [22] Train-accuracy=0.732983332951864## [23] Train-accuracy=0.754066667079926## [24] Train-accuracy=0.770366665999095## [25] Train-accuracy=0.783983333428701## [26] Train-accuracy=0.801383334159851## [27] Train-accuracy=0.818666667461395## [28] Train-accuracy=0.829783334493637## [29] Train-accuracy=0.837666666825612## [30] Train-accuracy=0.845216666698456## [31] Train-accuracy=0.851216666777929## [32] Train-accuracy=0.856150000413259## [33] Train-accuracy=0.861000000317891## [34] Train-accuracy=0.864683332602183## [35] Train-accuracy=0.86906666636467## [36] Train-accuracy=0.873499999205271## [37] Train-accuracy=0.876683332840602## [38] Train-accuracy=0.879299999237061## [39] Train-accuracy=0.881516666889191## [40] Train-accuracy=0.883583333810171## [41] Train-accuracy=0.886216667811076## [42] Train-accuracy=0.88813333495458## [43] Train-accuracy=0.889966667413712## [44] Train-accuracy=0.891316667556763## [45] Train-accuracy=0.893700000524521## [46] Train-accuracy=0.895799999396006## [47] Train-accuracy=0.897183333158493## [48] Train-accuracy=0.898516666094462## [49] Train-accuracy=0.900850000063578## [50] Train-accuracy=0.902100000301997## [1] "Training took: 233.52 seconds"# saveRDS(m1print, "m1print.RDS")
# saveRDS(log, "log1.RDS")
# log1 <- readRDS("log1.RDS")
# m1 <- readRDS("m1.RDS")
# m1print <- readRDS("m1print.RDS")Result Analysis
We tried to add Layer in order to improve the Accuracy:
- Hidden Layer : 2 (128, 64), num.round = 40[40] Train-accuracy=0.895683333953222 [1] “Training took: 82.65 seconds”
[40] Train-accuracy=0.895683333953222 [1] “Training took: 129.46 seconds”
[40] Train-accuracy=0.895683333953222 [1] “Training took: 94.24 seconds”
The training time were different from time to time, it depends on Processor load when the chunk run.
- Hidden Layer : 3 (256, 128, 64), num.round = 40[40] Train-accuracy=0.883583333810171 [1] “Training took: 182 seconds”
- Hidden Layer : 3 (256, 128, 64), num.round = 50 (v)[50] Train-accuracy=0.902100000301997 [1] “Training took: 239.05 seconds”
[50] Train-accuracy=0.902100000301997 [1] “Training took: 295.5 seconds”
- Hidden Layer : 4 (512, 256, 128, 64), num.round = 50, learning.rate = 0.001[50] Train-accuracy=0.0975333338057001 [1] “Training took: 497.11 seconds”
By adding the hidden layer, make the result worse.
- Hidden Layer : 4 (512, 256, 128, 64), num.round = 40, learning.rate = 0.0005[40] Train-accuracy=0.0967000004624327 [1] “Training took: 414.36 seconds”
So, we can conclude the better model to describe our data is using:
1. use 3 hidden layer (each layer filled with 256, 128, 64 neuron)
2. use 10 output layer (as per total of the label)
3. the most efficient num.round or iteration is 50
4. learning rate = 0.001
5. Accuracy = 90,21%
We chose those model because in the Neural Network, prefer to use simple model and moderate time consumption.
Relation between Iteration and Accuracy.
Making prediction
Predict the unseen data-test
## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 0.75 0 0.00 0.00 0.00 0 0.25 0 0 0
## [2,] 0.00 1 0.00 0.00 0.00 0 0.00 0 0 0
## [3,] 0.06 0 0.55 0.00 0.00 0 0.39 0 0 0
## [4,] 0.25 0 0.59 0.00 0.00 0 0.16 0 0 0
## [5,] 0.00 0 0.01 0.21 0.78 0 0.00 0 0 0Predict the result:
## [1] 0 1 2 2 4 6 8 4 5 0Load the categories, so, we can understand the label:
We would like to compare the prediction and label
if the prediction is true, we will give the ‘darkgreen’ color, otherwise ‘red’.
plotResults <- function(images, preds){
x <- ceiling(sqrt(length(images)))
par(mfrow=c(x,x), mar=c(0, 0, 1.5, 0))
for (i in images){
m <- matrix(test[i,-1], nrow=28, byrow=TRUE)
m <- apply(m, 2, rev)
predicted_label <- preds[i]#m1_preds_result[i]
true_label <- test_y[i]
if (predicted_label == true_label) {
color <- 'darkgreen'
} else {
color <- 'red'
}
image(t(m), col=gray((0:255)/255), axes=FALSE,
main = paste0(categories[predicted_label + 1], " (",
categories[true_label + 1], ")"),
col.main = color)
}
}## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 9
## 0 827 5 9 28 0 0 129 0 0 0
## 1 0 975 1 14 0 0 1 0 0 0
## 2 11 4 759 8 38 0 53 0 1 0
## 3 15 13 8 866 16 1 16 0 2 0
## 4 0 2 136 51 895 0 85 0 4 0
## 5 1 0 0 1 0 937 0 27 5 6
## 6 135 1 86 30 47 1 710 0 20 0
## 7 0 0 0 0 0 20 0 903 1 21
## 8 11 0 1 2 4 7 6 0 965 0
## 9 0 0 0 0 0 34 0 70 2 973
##
## Overall Statistics
##
## Accuracy : 0.881
## 95% CI : (0.8745, 0.8873)
## No Information Rate : 0.1
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.8678
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity 0.8270 0.9750 0.7590 0.8660 0.8950 0.9370
## Specificity 0.9810 0.9982 0.9872 0.9921 0.9691 0.9956
## Pos Pred Value 0.8287 0.9839 0.8684 0.9242 0.7630 0.9591
## Neg Pred Value 0.9808 0.9972 0.9736 0.9852 0.9881 0.9930
## Prevalence 0.1000 0.1000 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0827 0.0975 0.0759 0.0866 0.0895 0.0937
## Detection Prevalence 0.0998 0.0991 0.0874 0.0937 0.1173 0.0977
## Balanced Accuracy 0.9040 0.9866 0.8731 0.9291 0.9321 0.9663
## Class: 6 Class: 7 Class: 8 Class: 9
## Sensitivity 0.7100 0.9030 0.9650 0.9730
## Specificity 0.9644 0.9953 0.9966 0.9882
## Pos Pred Value 0.6893 0.9556 0.9689 0.9018
## Neg Pred Value 0.9677 0.9893 0.9961 0.9970
## Prevalence 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0710 0.0903 0.0965 0.0973
## Detection Prevalence 0.1030 0.0945 0.0996 0.1079
## Balanced Accuracy 0.8372 0.9492 0.9808 0.9806The model is good in data train (accuracy = 90,21%) and data test (accuracy = 88,1%)
4.1.2 2nd Model
We’ll try to improve the model:
m2.data <- mx.symbol.Variable("data")
#Hidden Layer
m2.fc1 <- mx.symbol.FullyConnected(m2.data, name="fc1", num_hidden=256)
m2.act1 <- mx.symbol.Activation(m2.fc1, name="activation1", act_type="relu")
m2.fc2 <- mx.symbol.FullyConnected(m2.act1, name="fc2", num_hidden=128)
m2.act2 <- mx.symbol.Activation(m2.fc2, name="activation2", act_type="relu")
m2.fc3 <- mx.symbol.FullyConnected(m2.act2, name="fc3", num_hidden=64)
m2.act3 <- mx.symbol.Activation(m2.fc3, name="activation1", act_type="relu")
#Output Layer
m2.fc4 <- mx.symbol.FullyConnected(m2.act3, name="fc4", num_hidden=10)
m2.softmax <- mx.symbol.SoftmaxOutput(m2.fc4, name="softMax")log <- mx.metric.logger$new()
startime <- proc.time()
mx.set.seed(0)
m2 <- mx.model.FeedForward.create(m2.softmax, #the network configuration made above
X = train_x, #input (predictors),
y = train_y, #the labels
ctx = mx.cpu(),
num.round = 60,
array.batch.size = 80,
momentum = 0.95,
array.layout="colmajor",
learning.rate = 0.001,
eval.metric = mx.metric.accuracy,
epoch.end.callback = mx.callback.log.train.metric(1,log)
)## Start training with 1 devices## [1] Train-accuracy=0.0979166671509544## [2] Train-accuracy=0.0973666670918465## [3] Train-accuracy=0.0976000004708767## [4] Train-accuracy=0.097566667119662## [5] Train-accuracy=0.0978333337778846## [6] Train-accuracy=0.0979666671231389## [7] Train-accuracy=0.0980666671122114## [8] Train-accuracy=0.0986166671092312## [9] Train-accuracy=0.0995000004644195## [10] Train-accuracy=0.10051666715617## [11] Train-accuracy=0.105850000320623## [12] Train-accuracy=0.115750000435859## [13] Train-accuracy=0.152266666742663## [14] Train-accuracy=0.207199999948343## [15] Train-accuracy=0.470383331775665## [16] Train-accuracy=0.585783332506816## [17] Train-accuracy=0.628316666007042## [18] Train-accuracy=0.6525166670084## [19] Train-accuracy=0.67264999906222## [20] Train-accuracy=0.68975## [21] Train-accuracy=0.709566667159398## [22] Train-accuracy=0.732983332951864## [23] Train-accuracy=0.754066667079926## [24] Train-accuracy=0.770366665999095## [25] Train-accuracy=0.783983333428701## [26] Train-accuracy=0.801383334159851## [27] Train-accuracy=0.818666667461395## [28] Train-accuracy=0.829783334493637## [29] Train-accuracy=0.837666666825612## [30] Train-accuracy=0.845216666698456## [31] Train-accuracy=0.851216666777929## [32] Train-accuracy=0.856150000413259## [33] Train-accuracy=0.861000000317891## [34] Train-accuracy=0.864683332602183## [35] Train-accuracy=0.86906666636467## [36] Train-accuracy=0.873499999205271## [37] Train-accuracy=0.876683332840602## [38] Train-accuracy=0.879299999237061## [39] Train-accuracy=0.881516666889191## [40] Train-accuracy=0.883583333810171## [41] Train-accuracy=0.886216667811076## [42] Train-accuracy=0.88813333495458## [43] Train-accuracy=0.889966667413712## [44] Train-accuracy=0.891316667556763## [45] Train-accuracy=0.893700000524521## [46] Train-accuracy=0.895799999396006## [47] Train-accuracy=0.897183333158493## [48] Train-accuracy=0.898516666094462## [49] Train-accuracy=0.900850000063578## [50] Train-accuracy=0.902100000301997## [51] Train-accuracy=0.903533333063126## [52] Train-accuracy=0.905449999729792## [53] Train-accuracy=0.90641666730245## [54] Train-accuracy=0.907983333110809## [55] Train-accuracy=0.909466666936874## [56] Train-accuracy=0.910733332951864## [57] Train-accuracy=0.91168333307902## [58] Train-accuracy=0.912333333015442## [59] Train-accuracy=0.913566666285197## [60] Train-accuracy=0.915033332665761## [1] "Training took: 226.31 seconds"[60] Train-accuracy=0.915033332665761 [1] “Training took: 254.73 seconds”
## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 0.88 0 0.00 0.00 0.00 0 0.11 0 0 0
## [2,] 0.00 1 0.00 0.00 0.00 0 0.00 0 0 0
## [3,] 0.05 0 0.46 0.00 0.00 0 0.49 0 0 0
## [4,] 0.23 0 0.72 0.00 0.00 0 0.04 0 0 0
## [5,] 0.00 0 0.00 0.33 0.66 0 0.00 0 0 0## [1] 0 1 6 2 4 6 8 4 5 0## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 9
## 0 862 4 11 29 0 0 154 0 1 0
## 1 0 981 1 13 0 0 1 0 0 0
## 2 15 4 788 11 43 0 58 0 2 0
## 3 14 11 9 884 20 1 19 0 4 0
## 4 0 0 118 44 891 0 78 0 7 0
## 5 0 0 0 0 0 950 0 30 6 9
## 6 98 0 72 17 43 1 684 0 18 0
## 7 0 0 0 0 0 20 0 914 1 24
## 8 11 0 1 2 3 9 6 0 960 1
## 9 0 0 0 0 0 19 0 56 1 966
##
## Overall Statistics
##
## Accuracy : 0.888
## 95% CI : (0.8817, 0.8941)
## No Information Rate : 0.1
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.8756
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity 0.8620 0.9810 0.7880 0.8840 0.8910 0.9500
## Specificity 0.9779 0.9983 0.9852 0.9913 0.9726 0.9950
## Pos Pred Value 0.8124 0.9849 0.8556 0.9189 0.7830 0.9548
## Neg Pred Value 0.9846 0.9979 0.9766 0.9872 0.9877 0.9944
## Prevalence 0.1000 0.1000 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0862 0.0981 0.0788 0.0884 0.0891 0.0950
## Detection Prevalence 0.1061 0.0996 0.0921 0.0962 0.1138 0.0995
## Balanced Accuracy 0.9199 0.9897 0.8866 0.9377 0.9318 0.9725
## Class: 6 Class: 7 Class: 8 Class: 9
## Sensitivity 0.6840 0.9140 0.9600 0.9660
## Specificity 0.9723 0.9950 0.9963 0.9916
## Pos Pred Value 0.7331 0.9531 0.9668 0.9271
## Neg Pred Value 0.9651 0.9905 0.9956 0.9962
## Prevalence 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0684 0.0914 0.0960 0.0966
## Detection Prevalence 0.0933 0.0959 0.0993 0.1042
## Balanced Accuracy 0.8282 0.9545 0.9782 0.9788Accuracy in data test: 88,8%
4.1.3 3rd Model
We’ll try to improve the model:
m3.data <- mx.symbol.Variable("data")
#Hidden Layer
m3.fc0 <- mx.symbol.FullyConnected(m3.data, name="fc0", num_hidden= 512)
m3.act0 <- mx.symbol.Activation(m3.fc0, name="activation1", act_type="relu")
m3.fc1 <- mx.symbol.FullyConnected(m3.act0, name="fc1", num_hidden=256)
m3.act1 <- mx.symbol.Activation(m3.fc1, name="activation2", act_type="relu")
m3.fc2 <- mx.symbol.FullyConnected(m3.act1, name="fc2", num_hidden=128)
m3.act2 <- mx.symbol.Activation(m3.fc2, name="activation2", act_type="relu")
#Output Layer
#we're not using sigmoid, because its binary classification, now, we have to classify 10 classes
m3.fc3 <- mx.symbol.FullyConnected(m3.act2, name="fc4", num_hidden=10)
m3.softmax <- mx.symbol.SoftmaxOutput(m3.fc3, name="softMax")log <- mx.metric.logger$new()
startime <- proc.time()
mx.set.seed(0)
m3 <- mx.model.FeedForward.create(m3.softmax, #the network configuration made above
X = train_x, #input (predictors),
y = train_y, #the labels
ctx = mx.cpu(),
num.round = 80,
array.batch.size = 80,
momentum = 0.95,
array.layout="colmajor",
learning.rate = 0.001,
eval.metric = mx.metric.accuracy,
epoch.end.callback = mx.callback.log.train.metric(1,log)
)## Start training with 1 devices## [1] Train-accuracy=0.0980333337659637## [2] Train-accuracy=0.0981833337545395## [3] Train-accuracy=0.099050000431637## [4] Train-accuracy=0.100083333748082## [5] Train-accuracy=0.103150000344962## [6] Train-accuracy=0.11111666698133## [7] Train-accuracy=0.142966666626434## [8] Train-accuracy=0.199949999660254## [9] Train-accuracy=0.451900000194708## [10] Train-accuracy=0.584333332935969## [11] Train-accuracy=0.63233333269755## [12] Train-accuracy=0.664516668200493## [13] Train-accuracy=0.687716666698456## [14] Train-accuracy=0.707433332125346## [15] Train-accuracy=0.731649999936422## [16] Train-accuracy=0.755200000127157## [17] Train-accuracy=0.769600001255671## [18] Train-accuracy=0.78385000038147## [19] Train-accuracy=0.800466666380564## [20] Train-accuracy=0.816800000429153## [21] Train-accuracy=0.828683333555857## [22] Train-accuracy=0.837816665728887## [23] Train-accuracy=0.845216666857401## [24] Train-accuracy=0.851866666237513## [25] Train-accuracy=0.857150000651677## [26] Train-accuracy=0.86116666730245## [27] Train-accuracy=0.865583334048589## [28] Train-accuracy=0.869716667334239## [29] Train-accuracy=0.873700000524521## [30] Train-accuracy=0.876366666555405## [31] Train-accuracy=0.879250000158946## [32] Train-accuracy=0.88228333290418## [33] Train-accuracy=0.884350000222524## [34] Train-accuracy=0.886783332506816## [35] Train-accuracy=0.889083332777023## [36] Train-accuracy=0.890616665919622## [37] Train-accuracy=0.89281666636467## [38] Train-accuracy=0.894666666746139## [39] Train-accuracy=0.896649999221166## [40] Train-accuracy=0.898966667413712## [41] Train-accuracy=0.900849999189377## [42] Train-accuracy=0.903316666682561## [43] Train-accuracy=0.904616666475932## [44] Train-accuracy=0.906566666205724## [45] Train-accuracy=0.908316667000453## [46] Train-accuracy=0.910233333110809## [47] Train-accuracy=0.911233333587646## [48] Train-accuracy=0.913150000651677## [49] Train-accuracy=0.914549998919169## [50] Train-accuracy=0.916666666110357## [51] Train-accuracy=0.917933332920075## [52] Train-accuracy=0.919166666269302## [53] Train-accuracy=0.920399999141693## [54] Train-accuracy=0.921833334128062## [55] Train-accuracy=0.922583332777023## [56] Train-accuracy=0.924716666221619## [57] Train-accuracy=0.925366665840149## [58] Train-accuracy=0.925716665665309## [59] Train-accuracy=0.927533333222071## [60] Train-accuracy=0.928350000222524## [61] Train-accuracy=0.928733332554499## [62] Train-accuracy=0.930333333015442## [63] Train-accuracy=0.93033333269755## [64] Train-accuracy=0.92978333290418## [65] Train-accuracy=0.928933332920075## [66] Train-accuracy=0.927366666793823## [67] Train-accuracy=0.92833333214124## [68] Train-accuracy=0.931499999046326## [69] Train-accuracy=0.932083332697551## [70] Train-accuracy=0.934149998267492## [71] Train-accuracy=0.936016665379206## [72] Train-accuracy=0.935699998935064## [73] Train-accuracy=0.935666665792465## [74] Train-accuracy=0.936933332363764## [75] Train-accuracy=0.936933333476385## [76] Train-accuracy=0.935800000270208## [77] Train-accuracy=0.937083332220713## [78] Train-accuracy=0.939649999856949## [79] Train-accuracy=0.94115000017484## [80] Train-accuracy=0.941983331680298## [1] "Training took: 989.86 seconds"# saveRDS(log, "log3.RDS")
# saveRDS(m3, "m3.RDS")
# saveRDS(m3print, "m3print.RDS")
# log3 <- readRDS("log3.RDS")
# m3 <- readRDS("m3.RDS")
# m3print <- readRDS("m3print.RDS")Result Analysis
In this m3 - model, we tried few times using combination:
[40] Train-accuracy=0.895833333174388
[1] “Training took: 443.56 seconds”
[60] Train-accuracy=0.922000000953674
[1] “Training took: 551.75 seconds”
[60] Train-accuracy=0.928350000222524
[1] “Training took: 565.22 seconds”
- learning rate 0.0005:[60] Train-accuracy=0.886649999300639
[1] “Training took: 565.69 seconds”
- 80 iterasi:[80] Train-accuracy=0.941983331680298
[1] “Training took: 1004.59 seconds”
[80] Train-accuracy=0.941983331680298 [1] “Training took: 575.91 seconds”
Relation between Iteration and Accuracy.
m3_preds <- predict(m3, test_x, array.layout = "colmajor")
m3_preds_result <- max.col(t(m3_preds)) - 1
confusionMatrix(as.factor(m3_preds_result), as.factor(test_y))## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1 2 3 4 5 6 7 8 9
## 0 882 8 16 57 0 1 162 0 7 0
## 1 0 983 0 31 1 0 2 0 0 0
## 2 14 4 804 7 72 0 61 0 4 0
## 3 6 5 5 827 16 4 17 0 2 0
## 4 0 0 83 47 822 0 39 0 2 0
## 5 0 0 0 0 0 926 0 14 1 2
## 6 91 0 90 30 83 0 712 0 14 1
## 7 0 0 0 0 0 28 0 901 2 14
## 8 7 0 2 1 6 14 7 1 967 3
## 9 0 0 0 0 0 27 0 84 1 980
##
## Overall Statistics
##
## Accuracy : 0.8804
## 95% CI : (0.8739, 0.8867)
## No Information Rate : 0.1
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.8671
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity 0.8820 0.9830 0.8040 0.8270 0.8220 0.9260
## Specificity 0.9721 0.9962 0.9820 0.9939 0.9810 0.9981
## Pos Pred Value 0.7785 0.9666 0.8323 0.9376 0.8278 0.9820
## Neg Pred Value 0.9867 0.9981 0.9783 0.9810 0.9802 0.9918
## Prevalence 0.1000 0.1000 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0882 0.0983 0.0804 0.0827 0.0822 0.0926
## Detection Prevalence 0.1133 0.1017 0.0966 0.0882 0.0993 0.0943
## Balanced Accuracy 0.9271 0.9896 0.8930 0.9104 0.9015 0.9621
## Class: 6 Class: 7 Class: 8 Class: 9
## Sensitivity 0.7120 0.9010 0.9670 0.9800
## Specificity 0.9657 0.9951 0.9954 0.9876
## Pos Pred Value 0.6974 0.9534 0.9593 0.8974
## Neg Pred Value 0.9679 0.9891 0.9963 0.9978
## Prevalence 0.1000 0.1000 0.1000 0.1000
## Detection Rate 0.0712 0.0901 0.0967 0.0980
## Detection Prevalence 0.1021 0.0945 0.1008 0.1092
## Balanced Accuracy 0.8388 0.9481 0.9812 0.9838Accuracy for data Test: 0.8804
Conclusion
In choosing the best model for our Neural Network, we should consider few things: - choose the simplest model - time consumption - model is not overfit / underfit, because we need the model to be good in both data (train & test)
So, the best MXNet model is m2 based on time consumption and the difference between data Train & Test. The accuracy for data train is 91,5 % and data test is 88,8%.