Fraud Detection
The domain is sales data. We want to find “strange” transaction reports that may be fraudulent.
We want to provide a “fraud probability ranking” for events that have already occurred.
We will look at several data mining tasks including: (1) outlier or anomaly detection;(2) clustering; and (3) semi-supervised prediction modeling.
Data is transactions reported by salespeople.They sell products and report these sales with a regular periodicity.
The methodology Will need to load DMwR package
library(DMwR)## Warning: package 'DMwR' was built under R version 3.0.3## Loading required package: lattice## Warning: package 'lattice' was built under R version 3.0.3## Loading required package: grid
## KernSmooth 2.23 loaded
## Copyright M. P. Wand 1997-2009Then can load sales data:
data(sales)Take a look:
head(sales)## ID Prod Quant Val Insp
## 1 v1 p1 182 1665 unkn
## 2 v2 p1 3072 8780 unkn
## 3 v3 p1 20393 76990 unkn
## 4 v4 p1 112 1100 unkn
## 5 v3 p1 6164 20260 unkn
## 6 v5 p2 104 1155 unknHow many rows?:
nrow(sales)## [1] 401146How is data structured?
nrow(sales)## [1] 401146ID - a factor, ID of salesman Prod - a factor, ID of the sold product Quant - number of reported sold units of product Val - reported total monetary value of the sale Insp - Factor, 3 possible values: (1) ok, if transaction inspected and determined to be valid; (2) fraud, is found to be fraudulent;and (3) unk if unknown (not inspected yet)
Exploring the data set
Overview of statistical properties of data:
summary(sales)## ID Prod Quant Val
## v431 : 10159 p1125 : 3923 Min. : 100 Min. : 1005
## v54 : 6017 p3774 : 1824 1st Qu.: 107 1st Qu.: 1345
## v426 : 3902 p1437 : 1720 Median : 168 Median : 2675
## v1679 : 3016 p1917 : 1702 Mean : 8442 Mean : 14617
## v1085 : 3001 p4089 : 1598 3rd Qu.: 738 3rd Qu.: 8680
## v1183 : 2642 p2742 : 1519 Max. :473883883 Max. :4642955
## (Other):372409 (Other):388860 NA's :13842 NA's :1182
## Insp
## ok : 14462
## unkn :385414
## fraud: 1270
##
##
##
## have bunches of salespeople
nlevels(sales$ID)## [1] 6016and lots of unique products
nlevels(sales$Prod)## [1] 4548We check if Quant and Val are both missing together much (sum returns number 888)
sum(is.na(sales$Quant) & is.na(sales$Val))## [1] 888From summary() results, look at distribution of values in inspection column…proportion of frauds is relatively low, even if we only take into account the reports that were inspected (about 0.003166)
table(sales$Insp)/nrow(sales)*100##
## ok unkn fraud
## 3.605171 96.078236 0.316593We look at number of transactions per salesperson.If there is much variability, also if we look at number of transactions per product
Set up table for counts of transactions per salesperson:
(totS <- table(sales$ID))##
## v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12
## 96 50 26 549 33 61 34 89 86 683 43 246
## v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24
## 61 188 532 115 158 260 1355 48 118 409 391 176
## v25 v26 v27 v28 v29 v30 v31 v32 v33 v34 v35 v36
## 624 15 253 49 1068 44 8 8 9 6 105 4
## v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47 v48
## 29 6 2412 8 59 120 1604 1982 1572 105 57 255
## v49 v50 v51 v52 v53 v54 v55 v56 v57 v58 v59 v60
## 156 630 43 3 27 6017 52 53 10 52 104 11
## v61 v62 v63 v64 v65 v66 v67 v68 v69 v70 v71 v72
## 28 47 21 37 1 7 173 36 46 79 32 4
## v73 v74 v75 v76 v77 v78 v79 v80 v81 v82 v83 v84
## 2 27 395 154 264 197 214 348 60 107 42 71
## v85 v86 v87 v88 v89 v90 v91 v92 v93 v94 v95 v96
## 126 153 7 44 75 55 58 36 79 84 77 346
## v97 v98 v99 v100 v101 v102 v103 v104 v105 v106 v107 v108
## 24 126 14 17 40 26 59 20 33 82 38 6
## v109 v110 v111 v112 v113 v114 v115 v116 v117 v118 v119 v120
## 73 483 228 21 62 7 17 270 4 15 8 77
## v121 v122 v123 v124 v125 v126 v127 v128 v129 v130 v131 v132
## 38 53 92 22 79 55 40 37 110 388 1558 5
## v133 v134 v135 v136 v137 v138 v139 v140 v141 v142 v143 v144
## 32 68 37 32 12 16 49 60 52 21 112 67
## v145 v146 v147 v148 v149 v150 v151 v152 v153 v154 v155 v156
## 40 317 21 62 45 47 100 1900 5 309 116 89
## v157 v158 v159 v160 v161 v162 v163 v164 v165 v166 v167 v168
## 68 61 14 146 91 26 109 37 176 6 43 37
## v169 v170 v171 v172 v173 v174 v175 v176 v177 v178 v179 v180
## 525 13 53 7 40 28 15 119 49 19 25 59
## v181 v182 v183 v184 v185 v186 v187 v188 v189 v190 v191 v192
## 241 57 47 88 6 652 10 20 22 3 31 9
## v193 v194 v195 v196 v197 v198 v199 v200 v201 v202 v203 v204
## 19 129 32 64 3 89 169 59 100 56 168 36
## v205 v206 v207 v208 v209 v210 v211 v212 v213 v214 v215 v216
## 121 24 60 36 17 100 42 71 95 253 78 76
## v217 v218 v219 v220 v221 v222 v223 v224 v226 v227 v228 v229
## 14 57 4 4 7 30 59 5 69 43 18 32
## v230 v231 v232 v233 v234 v235 v236 v237 v238 v239 v240 v241
## 115 5 14 28 112 55 100 11 38 12 41 17
## v242 v243 v244 v245 v246 v247 v248 v249 v250 v251 v252 v253
## 62 114 13 87 20 7 20 9 650 643 519 1291
## v254 v255 v256 v257 v258 v259 v260 v261 v262 v263 v264 v265
## 176 55 50 14 15 21 865 6 64 15 507 135
## v266 v267 v268 v269 v270 v271 v272 v273 v274 v275 v276 v277
## 22 52 11 57 3 5 22 5 8 8 151 1126
## v278 v279 v280 v281 v282 v283 v284 v285 v286 v287 v288 v289
## 153 97 126 61 78 20 84 106 155 72 358 160
## v290 v291 v292 v293 v294 v295 v296 v297 v298 v299 v300 v301
## 29 52 9 31 121 120 51 43 59 52 62 9
## v302 v303 v304 v305 v306 v307 v308 v309 v310 v311 v312 v313
## 10 20 55 357 50 28 107 13 29 27 18 18
## v314 v315 v316 v317 v318 v319 v320 v321 v322 v324 v325 v326
## 236 37 10 92 33 74 22 13 35 3 11 13
## v327 v328 v329 v330 v331 v332 v333 v334 v335 v336 v337 v338
## 20 3 14 3 53 49 1875 11 31 7 11 1
## v339 v340 v341 v342 v343 v344 v345 v346 v347 v348 v349 v350
## 14 7 75 3 48 26 25 4 39 21 349 50
## v351 v352 v353 v354 v355 v356 v357 v358 v359 v360 v361 v362
## 1 12 100 112 102 1344 25 146 56 374 53 15
## v363 v364 v365 v366 v367 v368 v369 v370 v371 v372 v373 v375
## 34 97 115 7 44 127 163 484 74 33 199 367
## v376 v377 v378 v379 v380 v381 v383 v384 v385 v386 v387 v388
## 209 367 313 153 26 36 7 18 28 101 31 11
## v389 v390 v391 v392 v393 v394 v395 v396 v397 v398 v399 v400
## 71 7 8 269 140 26 83 25 1015 18 33 40
## v401 v402 v403 v404 v405 v406 v407 v408 v409 v410 v411 v412
## 25 20 29 67 25 77 62 27 178 70 14 131
## v413 v414 v415 v416 v417 v418 v419 v420 v421 v422 v423 v424
## 15 79 14 212 332 100 348 140 147 748 108 119
## v425 v426 v427 v428 v429 v430 v431 v432 v433 v434 v435 v436
## 284 3902 114 84 247 83 10159 902 72 454 29 147
## v437 v438 v439 v440 v441 v442 v443 v444 v445 v446 v447 v448
## 21 47 86 282 38 196 271 94 108 111 219 175
## v449 v450 v451 v452 v453 v454 v455 v456 v457 v458 v459 v460
## 38 85 75 51 600 357 1140 142 81 121 69 42
## v461 v462 v463 v464 v465 v466 v467 v468 v470 v471 v472 v473
## 51 179 44 268 157 42 139 157 113 102 1459 58
## v474 v475 v476 v477 v478 v479 v480 v481 v482 v483 v484 v485
## 124 189 24 312 109 358 7 21 113 302 174 54
## v486 v487 v488 v489 v490 v491 v492 v494 v495 v496 v497 v498
## 54 36 136 40 392 12 7 14 1183 255 116 128
## v499 v500 v501 v502 v503 v504 v505 v506 v507 v508 v509 v510
## 18 54 16 504 83 68 94 126 40 50 24 50
## v511 v512 v513 v514 v515 v516 v517 v518 v519 v520 v521 v522
## 81 44 91 22 188 121 49 108 25 43 43 160
## v523 v524 v525 v526 v527 v528 v529 v530 v531 v532 v533 v534
## 214 23 144 98 72 98 250 52 513 11 99 147
## v535 v536 v537 v538 v539 v540 v541 v542 v543 v544 v545 v546
## 7 187 30 166 542 42 94 30 167 57 94 19
## v547 v548 v549 v550 v551 v552 v553 v554 v555 v556 v557 v558
## 50 34 142 6 39 6 368 86 37 90 62 81
## v559 v560 v561 v562 v563 v564 v565 v566 v567 v568 v569 v570
## 159 77 77 142 39 123 5 105 84 43 116 21
## v571 v572 v573 v574 v575 v576 v577 v578 v579 v580 v581 v582
## 31 20 38 77 14 262 52 21 81 23 513 15
## v583 v584 v585 v586 v587 v588 v589 v590 v591 v592 v593 v594
## 244 22 5 30 13 49 70 582 61 5 182 361
## v595 v596 v597 v598 v599 v600 v601 v602 v603 v604 v605 v606
## 3 1168 80 83 12 365 171 131 7 96 54 88
## v607 v608 v609 v610 v611 v612 v613 v614 v615 v616 v617 v618
## 6 117 14 28 18 49 2 2478 407 8 181 48
## v619 v620 v621 v622 v623 v624 v625 v626 v627 v628 v629 v630
## 20 14 606 97 74 7 5 30 67 21 246 392
## v631 v632 v633 v634 v635 v636 v637 v638 v639 v640 v641 v642
## 35 48 56 20 31 19 10 4 327 295 76 12
## v643 v644 v645 v646 v647 v648 v649 v650 v651 v652 v653 v654
## 18 106 199 28 33 68 174 57 100 96 7 331
## v655 v656 v657 v658 v659 v660 v661 v662 v663 v664 v665 v666
## 98 325 32 11 8 288 572 991 391 466 29 47
## v667 v668 v669 v670 v671 v672 v673 v674 v675 v676 v677 v678
## 170 15 34 73 111 590 52 48 35 119 248 11
## v679 v680 v681 v682 v683 v684 v685 v686 v687 v688 v689 v690
## 36 21 82 62 55 161 24 69 291 36 60 24
## v691 v692 v693 v694 v695 v696 v697 v698 v699 v700 v701 v702
## 60 220 50 55 54 118 27 51 54 34 888 21
## v703 v704 v705 v706 v707 v708 v709 v710 v711 v712 v713 v714
## 182 64 151 54 105 7 44 180 140 115 97 54
## v715 v716 v717 v718 v719 v720 v721 v722 v723 v724 v725 v726
## 10 152 32 74 84 66 67 4 48 27 64 14
## v727 v728 v729 v730 v731 v732 v733 v734 v735 v736 v737 v738
## 555 92 44 26 119 10 495 110 23 20 70 34
## v739 v740 v741 v742 v743 v744 v745 v746 v747 v748 v749 v750
## 1412 70 63 58 11 17 114 40 556 20 593 212
## v751 v752 v753 v754 v755 v756 v757 v758 v759 v760 v761 v762
## 33 43 119 8 42 87 50 77 10 14 7 193
## v763 v764 v765 v766 v767 v768 v769 v770 v771 v772 v773 v774
## 111 427 250 417 117 138 142 36 501 19 149 110
## v775 v776 v777 v778 v779 v780 v781 v782 v783 v784 v785 v786
## 8 26 499 117 130 48 72 127 433 44 14 68
## v787 v788 v789 v790 v791 v792 v793 v794 v795 v796 v797 v798
## 98 3 716 71 15 15 30 108 187 92 91 269
## v799 v800 v801 v802 v803 v804 v805 v806 v807 v808 v809 v810
## 72 32 27 117 907 21 135 2 24 24 92 327
## v811 v812 v813 v814 v815 v816 v817 v818 v819 v820 v821 v822
## 111 128 50 33 15 193 189 366 840 54 32 53
## v823 v824 v825 v826 v827 v828 v829 v830 v831 v832 v833 v834
## 26 4 25 37 28 8 49 66 37 8 57 22
## v835 v836 v837 v838 v839 v840 v841 v842 v843 v844 v845 v846
## 66 80 92 233 71 88 8 27 10 82 37 145
## v847 v848 v849 v850 v851 v852 v853 v854 v855 v856 v857 v858
## 4 59 28 82 241 16 1 92 12 91 5 185
## v859 v860 v861 v862 v863 v864 v865 v866 v867 v868 v869 v870
## 10 6 437 19 130 4 30 123 92 6 156 46
## v871 v872 v873 v874 v875 v876 v877 v878 v879 v880 v881 v882
## 62 46 23 18 19 559 194 433 24 23 49 254
## v883 v884 v885 v886 v887 v888 v889 v890 v891 v892 v893 v894
## 10 156 117 72 107 31 43 48 234 74 110 30
## v895 v896 v897 v898 v899 v900 v901 v902 v903 v904 v905 v906
## 42 47 18 16 13 209 175 146 52 10 22 241
## v907 v908 v909 v910 v911 v912 v913 v914 v915 v916 v917 v918
## 9 60 71 91 50 26 143 35 6 95 6 64
## v919 v920 v921 v922 v923 v924 v925 v926 v927 v928 v929 v930
## 102 41 18 5 275 24 10 31 7 23 5 41
## v931 v932 v933 v934 v935 v936 v937 v938 v939 v940 v941 v942
## 36 127 34 12 88 34 28 28 9 14 101 27
## v943 v944 v945 v946 v947 v948 v949 v950 v951 v952 v953 v954
## 23 17 15 51 80 36 24 62 17 20 19 11
## v955 v956 v957 v958 v959 v960 v961 v962 v963 v964 v965 v966
## 1286 125 110 149 36 13 14 36 59 13 30 24
## v967 v968 v969 v970 v971 v972 v973 v974 v975 v976 v977 v978
## 45 9 15 2 119 183 21 69 94 203 184 79
## v979 v980 v981 v982 v983 v984 v985 v986 v987 v988 v989 v990
## 261 457 40 652 15 208 168 88 57 510 78 156
## v991 v992 v993 v994 v995 v996 v997 v998 v999 v1000 v1001 v1002
## 25 55 4 157 78 11 72 123 125 62 27 21
## v1003 v1004 v1005 v1006 v1007 v1008 v1009 v1010 v1011 v1012 v1013 v1014
## 58 265 118 16 56 33 844 90 6 16 36 968
## v1015 v1016 v1017 v1018 v1019 v1020 v1021 v1022 v1023 v1024 v1025 v1026
## 549 42 113 23 16 42 36 31 93 105 17 95
## v1027 v1028 v1029 v1030 v1031 v1032 v1033 v1034 v1035 v1036 v1037 v1038
## 55 68 13 18 136 41 384 380 47 65 90 45
## v1039 v1040 v1041 v1042 v1043 v1044 v1045 v1046 v1047 v1048 v1049 v1050
## 1248 125 86 21 228 69 9 9 27 18 10 36
## v1051 v1052 v1053 v1054 v1055 v1056 v1057 v1058 v1059 v1060 v1061 v1062
## 53 81 29 27 265 55 177 9 6 23 6 11
## v1063 v1064 v1065 v1066 v1067 v1068 v1069 v1070 v1071 v1072 v1073 v1074
## 168 10 4 8 8 22 42 204 80 31 18 83
## v1075 v1076 v1077 v1078 v1079 v1080 v1081 v1082 v1083 v1084 v1085 v1086
## 335 52 25 190 30 88 671 35 33 81 3001 75
## v1087 v1088 v1089 v1090 v1091 v1092 v1093 v1094 v1095 v1096 v1097 v1098
## 49 46 16 138 36 66 140 98 8 43 97 131
## v1099 v1100 v1101 v1102 v1103 v1104 v1105 v1106 v1107 v1108 v1109 v1110
## 68 32 96 29 44 45 50 73 129 15 665 13
## v1111 v1112 v1113 v1114 v1115 v1116 v1117 v1118 v1119 v1120 v1121 v1122
## 38 37 68 20 58 115 106 145 68 12 50 88
## v1123 v1124 v1125 v1126 v1127 v1128 v1129 v1130 v1131 v1132 v1133 v1134
## 7 86 40 35 337 600 15 123 221 124 174 356
## v1135 v1136 v1137 v1138 v1139 v1140 v1141 v1142 v1143 v1144 v1145 v1146
## 130 126 43 227 714 235 398 455 378 295 30 1094
## v1147 v1148 v1149 v1150 v1151 v1152 v1153 v1154 v1155 v1156 v1157 v1158
## 247 598 1729 431 98 505 246 285 583 235 62 450
## v1159 v1160 v1161 v1162 v1163 v1164 v1165 v1166 v1167 v1168 v1169 v1170
## 328 103 823 132 146 149 119 51 30 37 69 106
## v1171 v1172 v1173 v1174 v1175 v1176 v1177 v1178 v1179 v1180 v1181 v1182
## 572 1487 88 263 15 137 76 211 284 131 115 118
## v1183 v1184 v1185 v1186 v1187 v1188 v1189 v1190 v1191 v1192 v1193 v1194
## 2642 162 108 80 106 10 81 55 112 42 102 33
## v1195 v1196 v1197 v1198 v1199 v1200 v1201 v1202 v1203 v1204 v1205 v1206
## 113 217 43 64 4 196 152 34 66 27 57 64
## v1207 v1208 v1209 v1210 v1211 v1212 v1213 v1214 v1215 v1216 v1217 v1218
## 207 201 217 846 323 104 35 49 81 178 76 80
## v1219 v1220 v1221 v1222 v1223 v1224 v1225 v1226 v1227 v1228 v1229 v1230
## 24 176 75 32 174 30 7 230 110 75 30 58
## v1231 v1232 v1233 v1234 v1235 v1236 v1237 v1238 v1239 v1240 v1241 v1242
## 84 151 54 80 39 26 58 62 202 21 270 301
## v1243 v1244 v1245 v1246 v1247 v1248 v1249 v1250 v1251 v1252 v1253 v1254
## 736 162 46 26 408 172 102 87 32 256 27 42
## v1255 v1256 v1257 v1258 v1259 v1260 v1261 v1262 v1263 v1264 v1265 v1266
## 15 219 101 194 92 50 48 250 34 112 19 232
## v1267 v1268 v1269 v1270 v1271 v1272 v1273 v1274 v1275 v1276 v1277 v1278
## 77 44 18 68 51 47 95 137 38 233 41 95
## v1279 v1280 v1281 v1282 v1283 v1284 v1285 v1286 v1287 v1288 v1289 v1290
## 19 47 27 52 30 140 58 164 133 106 73 46
## v1291 v1292 v1293 v1294 v1295 v1296 v1297 v1298 v1299 v1300 v1301 v1302
## 14 275 106 173 196 110 289 179 266 269 264 102
## v1303 v1304 v1305 v1306 v1307 v1308 v1309 v1310 v1311 v1312 v1313 v1314
## 266 158 122 16 54 77 104 84 57 75 65 176
## v1315 v1316 v1317 v1318 v1319 v1320 v1321 v1322 v1323 v1324 v1325 v1326
## 260 117 423 27 28 194 19 85 166 32 302 69
## v1327 v1328 v1329 v1330 v1331 v1332 v1333 v1334 v1335 v1336 v1337 v1338
## 36 714 13 20 31 48 16 15 175 19 47 12
## v1339 v1340 v1341 v1342 v1343 v1344 v1345 v1346 v1347 v1348 v1349 v1350
## 38 100 37 36 7 26 133 281 42 48 28 187
## v1351 v1352 v1353 v1354 v1355 v1356 v1357 v1358 v1359 v1360 v1361 v1362
## 19 85 544 26 31 40 181 314 92 58 249 33
## v1363 v1364 v1365 v1366 v1367 v1368 v1369 v1370 v1371 v1372 v1373 v1374
## 48 48 23 24 41 132 22 166 294 72 14 65
## v1375 v1376 v1377 v1378 v1379 v1380 v1381 v1382 v1383 v1384 v1385 v1386
## 286 97 56 36 48 153 17 16 37 17 80 62
## v1387 v1388 v1389 v1390 v1391 v1392 v1393 v1394 v1395 v1396 v1397 v1398
## 88 6 15 28 27 14 81 7 97 110 13 81
## v1399 v1400 v1401 v1402 v1403 v1404 v1405 v1406 v1407 v1408 v1409 v1410
## 43 51 230 11 220 7 16 26 59 13 17 18
## v1411 v1412 v1413 v1414 v1415 v1416 v1417 v1418 v1419 v1420 v1421 v1422
## 61 332 70 103 39 140 14 17 19 25 10 22
## v1423 v1424 v1425 v1426 v1427 v1428 v1429 v1430 v1431 v1432 v1433 v1434
## 243 12 6 92 12 13 17 14 7 41 23 9
## v1435 v1436 v1437 v1438 v1439 v1440 v1441 v1442 v1443 v1444 v1445 v1446
## 49 54 800 23 24 71 241 64 79 276 50 156
## v1447 v1448 v1449 v1450 v1451 v1452 v1453 v1454 v1455 v1456 v1457 v1458
## 115 18 24 174 405 208 31 406 14 132 49 44
## v1459 v1460 v1461 v1462 v1463 v1464 v1465 v1466 v1467 v1468 v1469 v1470
## 23 11 10 43 33 47 33 69 120 92 39 82
## v1471 v1472 v1473 v1474 v1475 v1476 v1477 v1478 v1479 v1480 v1481 v1482
## 32 78 191 10 46 7 140 51 26 81 63 28
## v1483 v1485 v1486 v1487 v1488 v1489 v1490 v1491 v1492 v1493 v1494 v1495
## 14 17 19 31 33 55 98 14 49 17 8 63
## v1496 v1497 v1498 v1499 v1500 v1501 v1502 v1503 v1504 v1505 v1506 v1507
## 8 17 33 23 25 13 4 35 35 11 46 35
## v1508 v1509 v1510 v1511 v1512 v1513 v1514 v1515 v1516 v1517 v1518 v1519
## 28 19 31 40 18 29 65 11 90 36 29 2
## v1520 v1521 v1522 v1523 v1524 v1525 v1526 v1527 v1528 v1529 v1530 v1531
## 13 61 63 17 14 39 38 6 14 10 7 35
## v1532 v1533 v1534 v1535 v1536 v1537 v1538 v1539 v1540 v1541 v1542 v1543
## 30 68 5 18 43 35 51 21 25 69 24 51
## v1544 v1545 v1546 v1547 v1548 v1549 v1550 v1551 v1552 v1553 v1554 v1555
## 5 185 1 35 97 42 29 36 65 61 38 22
## v1556 v1557 v1558 v1559 v1560 v1561 v1562 v1563 v1564 v1565 v1566 v1567
## 18 22 80 79 63 11 63 7 20 12 6 4
## v1568 v1569 v1570 v1571 v1572 v1573 v1574 v1575 v1576 v1577 v1578 v1579
## 156 38 19 26 38 13 18 5 21 28 17 22
## v1580 v1581 v1582 v1583 v1584 v1585 v1586 v1587 v1588 v1589 v1590 v1591
## 14 82 20 48 2 115 48 17 28 23 13 24
## v1592 v1593 v1594 v1595 v1596 v1597 v1598 v1599 v1600 v1601 v1602 v1603
## 19 14 6 8 76 58 21 15 11 9 33 9
## v1604 v1605 v1606 v1607 v1608 v1609 v1610 v1611 v1612 v1613 v1614 v1615
## 7 8 8 20 8 67 16 39 38 3 9 59
## v1616 v1617 v1618 v1619 v1620 v1621 v1622 v1623 v1624 v1625 v1626 v1627
## 73 37 20 28 19 4 24 123 36 44 7 7
## v1628 v1629 v1630 v1631 v1632 v1633 v1634 v1635 v1636 v1637 v1638 v1639
## 46 39 17 19 12 14 1 30 18 13 9 64
## v1640 v1641 v1642 v1643 v1644 v1645 v1646 v1647 v1648 v1649 v1650 v1651
## 12 14 9 8 12 20 36 18 42 84 92 1
## v1652 v1653 v1654 v1655 v1656 v1657 v1658 v1659 v1660 v1661 v1662 v1663
## 44 43 11 11 19 19 40 82 205 28 10 36
## v1664 v1665 v1666 v1667 v1668 v1669 v1670 v1671 v1672 v1673 v1674 v1675
## 144 33 601 75 28 460 86 16 85 224 69 1544
## v1676 v1677 v1678 v1679 v1680 v1681 v1682 v1683 v1684 v1685 v1686 v1687
## 515 226 1490 3016 86 181 60 386 132 33 4 124
## v1688 v1689 v1690 v1691 v1692 v1693 v1694 v1695 v1696 v1697 v1698 v1699
## 46 19 146 39 102 91 148 65 113 206 204 57
## v1700 v1701 v1702 v1703 v1704 v1705 v1706 v1707 v1708 v1709 v1710 v1711
## 24 102 582 159 316 57 114 7 55 128 19 181
## v1712 v1713 v1714 v1715 v1716 v1717 v1718 v1719 v1720 v1721 v1722 v1723
## 496 12 32 322 94 75 227 317 460 89 323 142
## v1724 v1725 v1726 v1727 v1728 v1729 v1730 v1731 v1732 v1733 v1734 v1735
## 155 4 105 228 87 32 75 152 72 75 307 12
## v1736 v1737 v1738 v1739 v1740 v1741 v1743 v1744 v1746 v1747 v1748 v1749
## 76 476 89 88 37 7 121 83 34 15 41 22
## v1750 v1751 v1752 v1753 v1754 v1755 v1756 v1757 v1758 v1759 v1760 v1761
## 18 27 35 26 23 15 1 15 38 18 11 11
## v1762 v1763 v1764 v1765 v1766 v1767 v1768 v1769 v1770 v1771 v1772 v1773
## 19 5 11 1 23 23 3 15 20 110 12 24
## v1774 v1775 v1776 v1777 v1778 v1779 v1780 v1781 v1782 v1783 v1784 v1785
## 35 12 7 5 13 30 9 96 29 49 58 29
## v1786 v1787 v1788 v1789 v1790 v1791 v1792 v1793 v1794 v1795 v1796 v1797
## 5 12 40 488 12 20 11 14 18 23 54 46
## v1798 v1799 v1800 v1801 v1802 v1803 v1804 v1805 v1806 v1807 v1808 v1809
## 30 31 2 11 8 273 15 3 19 9 39 21
## v1810 v1811 v1812 v1813 v1814 v1815 v1816 v1817 v1818 v1819 v1820 v1821
## 19 106 21 57 30 61 44 15 11 96 9 7
## v1822 v1823 v1824 v1825 v1826 v1827 v1828 v1829 v1830 v1831 v1832 v1833
## 8 21 5 8 19 14 5 10 7 5 6 10
## v1834 v1835 v1836 v1837 v1838 v1839 v1840 v1841 v1842 v1843 v1844 v1845
## 8 8 1 3 84 4 1 8 5 6 8 6
## v1846 v1847 v1848 v1849 v1850 v1851 v1852 v1853 v1854 v1855 v1856 v1857
## 5 6 4 6 16 4 413 21 15 8 23 7
## v1858 v1859 v1860 v1861 v1862 v1863 v1864 v1865 v1866 v1867 v1868 v1869
## 6 38 7 151 20 53 4 29 14 66 60 38
## v1870 v1871 v1872 v1873 v1874 v1875 v1876 v1877 v1878 v1879 v1880 v1881
## 19 19 29 3 25 86 100 16 40 11 19 6
## v1882 v1883 v1884 v1885 v1886 v1887 v1888 v1889 v1890 v1891 v1892 v1893
## 8 8 27 17 21 33 11 5 12 5 8 8
## v1894 v1895 v1896 v1897 v1898 v1899 v1900 v1901 v1902 v1903 v1904 v1905
## 14 9 14 27 22 58 76 12 14 38 29 22
## v1906 v1907 v1908 v1909 v1910 v1911 v1912 v1913 v1914 v1915 v1916 v1917
## 16 112 148 1269 202 65 30 40 24 4 7 8
## v1918 v1919 v1920 v1921 v1922 v1923 v1924 v1925 v1926 v1927 v1928 v1929
## 22 34 53 13 33 11 8 71 48 134 10 87
## v1930 v1931 v1932 v1933 v1934 v1935 v1936 v1937 v1938 v1939 v1940 v1941
## 97 10 16 66 27 139 34 176 29 64 162 25
## v1942 v1943 v1944 v1945 v1946 v1947 v1948 v1949 v1950 v1951 v1952 v1953
## 64 26 314 118 30 52 8 33 1032 17 57 34
## v1954 v1955 v1956 v1957 v1958 v1959 v1960 v1961 v1962 v1963 v1964 v1965
## 8 130 27 52 24 56 15 23 36 44 28 35
## v1966 v1967 v1968 v1969 v1970 v1971 v1972 v1973 v1974 v1975 v1976 v1977
## 35 31 2 36 11 1156 24 23 219 76 26 1036
## v1978 v1979 v1980 v1981 v1982 v1983 v1984 v1985 v1986 v1987 v1988 v1989
## 58 78 55 375 27 39 131 281 58 85 92 168
## v1990 v1991 v1992 v1993 v1994 v1995 v1996 v1997 v1998 v1999 v2000 v2001
## 49 91 41 62 125 135 19 14 8 26 13 14
## v2002 v2003 v2004 v2005 v2006 v2007 v2008 v2009 v2010 v2011 v2012 v2013
## 711 25 317 7 93 87 61 10 44 22 25 92
## v2014 v2015 v2016 v2017 v2018 v2019 v2020 v2021 v2022 v2023 v2024 v2025
## 79 24 22 97 42 79 342 12 11 46 264 94
## v2026 v2027 v2028 v2029 v2030 v2031 v2032 v2033 v2034 v2035 v2036 v2037
## 80 13 7 174 71 122 192 50 39 10 5 90
## v2038 v2039 v2040 v2041 v2042 v2043 v2044 v2045 v2046 v2047 v2048 v2049
## 15 14 91 118 142 158 34 392 30 78 40 25
## v2050 v2051 v2052 v2053 v2054 v2055 v2056 v2057 v2058 v2059 v2060 v2061
## 331 53 93 40 24 23 8 22 61 44 18 12
## v2062 v2063 v2064 v2065 v2066 v2067 v2068 v2069 v2070 v2071 v2072 v2073
## 58 28 12 132 37 15 27 93 12 21 7 70
## v2074 v2075 v2076 v2077 v2078 v2079 v2080 v2081 v2082 v2083 v2084 v2085
## 105 107 56 95 251 86 22 157 190 14 20 44
## v2086 v2087 v2088 v2089 v2090 v2091 v2092 v2093 v2094 v2095 v2096 v2097
## 51 58 52 148 41 19 32 139 18 217 77 6
## v2098 v2099 v2100 v2101 v2102 v2103 v2104 v2105 v2106 v2107 v2108 v2109
## 9 8 14 34 31 94 44 9 81 3 58 23
## v2110 v2111 v2112 v2113 v2114 v2115 v2116 v2117 v2118 v2119 v2120 v2121
## 24 12 77 7 35 209 15 5 56 17 59 52
## v2122 v2123 v2124 v2125 v2126 v2127 v2128 v2129 v2130 v2131 v2132 v2133
## 68 27 68 27 65 18 65 6 56 11 15 43
## v2134 v2135 v2136 v2137 v2138 v2139 v2140 v2141 v2142 v2143 v2144 v2145
## 16 12 7 6 19 6 7 5 7 12 8 8
## v2146 v2147 v2148 v2149 v2150 v2151 v2152 v2153 v2154 v2155 v2156 v2157
## 36 58 5 17 3 32 19 51 83 27 12 26
## v2158 v2159 v2160 v2161 v2162 v2163 v2164 v2165 v2166 v2167 v2168 v2169
## 56 22 116 137 17 20 16 18 34 25 10 107
## v2170 v2171 v2172 v2173 v2174 v2175 v2176 v2177 v2178 v2179 v2180 v2181
## 50 19 13 22 16 13 45 18 23 27 59 31
## v2182 v2183 v2184 v2185 v2186 v2187 v2188 v2189 v2190 v2191 v2192 v2193
## 117 335 7 24 15 10 8 9 22 30 19 9
## v2194 v2195 v2196 v2197 v2198 v2199 v2200 v2201 v2202 v2203 v2204 v2205
## 18 13 10 15 4 8 5 15 6 30 2 7
## v2206 v2207 v2208 v2209 v2210 v2211 v2212 v2213 v2214 v2215 v2216 v2217
## 129 12 8 3 22 15 127 60 161 44 26 17
## v2218 v2219 v2220 v2221 v2222 v2223 v2224 v2225 v2226 v2227 v2228 v2229
## 42 65 9 43 611 70 48 23 42 135 84 32
## v2230 v2231 v2232 v2233 v2234 v2235 v2236 v2237 v2238 v2239 v2240 v2241
## 314 28 717 125 6 90 220 132 100 30 207 104
## v2242 v2243 v2244 v2245 v2246 v2247 v2248 v2249 v2250 v2251 v2252 v2253
## 24 113 80 59 158 32 216 16 48 15 125 32
## v2254 v2255 v2256 v2257 v2258 v2259 v2260 v2261 v2262 v2263 v2264 v2265
## 49 195 169 75 23 75 75 6 56 122 18 121
## v2266 v2267 v2268 v2269 v2270 v2271 v2272 v2273 v2274 v2275 v2276 v2277
## 2 41 28 170 94 51 48 88 43 117 24 79
## v2278 v2279 v2280 v2281 v2282 v2283 v2284 v2285 v2286 v2287 v2288 v2289
## 33 78 4 59 19 201 198 367 334 21 33 44
## v2290 v2291 v2292 v2293 v2294 v2295 v2296 v2297 v2298 v2299 v2300 v2301
## 126 58 314 12 148 106 38 11 15 79 53 67
## v2302 v2303 v2304 v2305 v2306 v2307 v2308 v2309 v2310 v2311 v2312 v2313
## 11 35 14 22 169 18 35 28 17 20 44 86
## v2314 v2315 v2316 v2317 v2318 v2319 v2320 v2321 v2322 v2323 v2324 v2325
## 93 43 11 74 60 6 15 12 14 3 9 26
## v2326 v2327 v2328 v2329 v2330 v2331 v2332 v2333 v2334 v2335 v2336 v2337
## 76 18 21 49 225 3 66 8 18 18 16 41
## v2338 v2339 v2340 v2341 v2342 v2343 v2344 v2345 v2346 v2347 v2348 v2349
## 10 35 7 42 58 79 143 55 41 60 13 35
## v2350 v2351 v2352 v2353 v2354 v2355 v2356 v2357 v2358 v2359 v2360 v2361
## 24 22 16 25 116 78 76 8 2 25 9 21
## v2362 v2363 v2364 v2365 v2366 v2367 v2368 v2370 v2371 v2372 v2373 v2374
## 21 1 6 33 6 65 17 13 196 27 102 8
## v2375 v2376 v2377 v2378 v2379 v2380 v2381 v2382 v2383 v2384 v2385 v2386
## 49 45 35 22 26 25 15 12 8 51 29 9
## v2387 v2388 v2389 v2390 v2391 v2392 v2393 v2394 v2395 v2396 v2397 v2398
## 4 17 16 7 31 46 20 11 53 43 1 11
## v2399 v2400 v2401 v2402 v2403 v2404 v2405 v2406 v2407 v2408 v2409 v2410
## 8 24 5 21 38 38 19 7 29 35 42 24
## v2411 v2412 v2413 v2414 v2415 v2416 v2417 v2418 v2419 v2420 v2421 v2422
## 10 104 7 23 7 17 9 31 54 10 10 88
## v2423 v2424 v2425 v2426 v2427 v2428 v2429 v2430 v2431 v2432 v2433 v2434
## 37 26 101 19 33 7 7 4 9 40 18 7
## v2436 v2437 v2438 v2439 v2440 v2441 v2442 v2443 v2444 v2445 v2446 v2447
## 20 6 43 60 8 290 13 17 22 115 15 26
## v2448 v2449 v2450 v2451 v2452 v2453 v2454 v2455 v2456 v2457 v2458 v2459
## 33 34 52 5 25 112 33 35 256 9 4 99
## v2460 v2461 v2462 v2463 v2464 v2465 v2466 v2467 v2468 v2469 v2470 v2471
## 21 23 8 43 36 32 35 71 45 1 40 18
## v2472 v2473 v2474 v2475 v2476 v2477 v2478 v2479 v2480 v2481 v2482 v2483
## 251 37 21 25 35 4 25 18 20 52 43 22
## v2484 v2485 v2486 v2487 v2488 v2489 v2490 v2491 v2492 v2493 v2494 v2495
## 54 99 41 91 22 44 62 36 21 2 46 29
## v2496 v2497 v2498 v2499 v2500 v2501 v2502 v2503 v2504 v2505 v2506 v2507
## 29 19 51 72 13 5 22 30 1 8 6 8
## v2508 v2509 v2510 v2511 v2512 v2513 v2514 v2515 v2516 v2517 v2518 v2519
## 42 46 14 13 10 91 3 8 6 7 3 53
## v2520 v2521 v2522 v2523 v2524 v2525 v2526 v2527 v2528 v2529 v2530 v2531
## 6 17 4 13 18 15 8 8 9 14 59 20
## v2532 v2533 v2534 v2535 v2536 v2537 v2538 v2539 v2540 v2541 v2542 v2543
## 15 11 20 7 26 11 17 59 152 153 34 18
## v2544 v2545 v2546 v2547 v2548 v2549 v2550 v2551 v2552 v2553 v2554 v2555
## 14 68 23 23 79 66 39 69 53 20 44 15
## v2556 v2557 v2558 v2559 v2560 v2561 v2562 v2563 v2564 v2565 v2566 v2567
## 45 20 18 49 15 46 62 38 7 13 10 9
## v2568 v2569 v2570 v2571 v2572 v2573 v2574 v2575 v2576 v2577 v2578 v2579
## 36 12 32 24 38 94 146 90 39 106 50 15
## v2580 v2581 v2582 v2583 v2584 v2585 v2586 v2587 v2588 v2589 v2590 v2591
## 18 19 38 18 16 10 22 22 24 12 15 16
## v2592 v2593 v2594 v2595 v2596 v2597 v2598 v2599 v2600 v2601 v2602 v2603
## 27 38 16 26 17 7 2 8 13 14 41 11
## v2604 v2605 v2606 v2607 v2608 v2609 v2610 v2611 v2612 v2613 v2614 v2615
## 45 5 140 30 59 6 4 15 29 8 44 7
## v2616 v2617 v2618 v2619 v2620 v2621 v2622 v2623 v2624 v2625 v2626 v2627
## 40 22 13 29 15 9 12 34 27 19 27 19
## v2628 v2629 v2630 v2631 v2632 v2633 v2634 v2635 v2636 v2637 v2638 v2639
## 24 39 34 121 10 26 20 5 929 21 14 85
## v2640 v2641 v2642 v2643 v2644 v2645 v2646 v2647 v2648 v2649 v2650 v2651
## 58 52 79 119 30 80 26 35 198 91 10 3
## v2652 v2653 v2654 v2655 v2656 v2657 v2658 v2659 v2660 v2661 v2662 v2663
## 10 12 235 6 23 70 72 227 170 37 12 4
## v2664 v2665 v2666 v2667 v2668 v2669 v2670 v2671 v2672 v2673 v2674 v2675
## 70 181 72 70 70 29 120 56 34 10 110 102
## v2676 v2677 v2678 v2679 v2680 v2681 v2682 v2683 v2684 v2685 v2686 v2687
## 80 47 3 35 67 16 73 43 46 132 10 20
## v2688 v2689 v2690 v2691 v2692 v2693 v2694 v2695 v2696 v2697 v2698 v2699
## 40 59 44 20 12 35 127 22 14 33 40 43
## v2700 v2701 v2702 v2703 v2704 v2705 v2706 v2707 v2708 v2709 v2710 v2711
## 16 61 69 52 6 11 57 40 46 10 3 2
## v2712 v2713 v2714 v2715 v2716 v2717 v2718 v2719 v2720 v2721 v2722 v2723
## 33 11 2 28 10 32 2 190 3 8 37 20
## v2724 v2725 v2726 v2727 v2728 v2729 v2730 v2731 v2732 v2733 v2734 v2735
## 38 13 81 103 28 5 8 10 19 19 9 27
## v2736 v2737 v2738 v2739 v2740 v2741 v2742 v2743 v2744 v2745 v2746 v2747
## 2 10 3 24 8 23 24 12 17 3 25 8
## v2748 v2749 v2750 v2751 v2752 v2753 v2754 v2755 v2756 v2757 v2758 v2759
## 29 57 32 34 23 29 18 2 6 62 14 13
## v2760 v2761 v2762 v2763 v2764 v2765 v2766 v2767 v2768 v2769 v2770 v2771
## 13 17 68 6 23 12 6 21 7 9 1 21
## v2772 v2773 v2774 v2775 v2776 v2777 v2778 v2779 v2780 v2781 v2782 v2783
## 9 4 8 5 75 30 20 29 16 16 15 12
## v2784 v2785 v2786 v2787 v2788 v2789 v2790 v2791 v2792 v2793 v2794 v2795
## 37 8 30 44 5 16 18 15 37 120 23 75
## v2796 v2797 v2798 v2799 v2800 v2801 v2802 v2803 v2804 v2805 v2806 v2807
## 337 93 106 19 10 69 600 16 69 17 292 28
## v2808 v2809 v2810 v2811 v2812 v2813 v2814 v2815 v2816 v2817 v2818 v2819
## 7 22 183 359 24 13 11 12 8 17 17 58
## v2820 v2821 v2822 v2823 v2824 v2825 v2826 v2827 v2828 v2829 v2830 v2831
## 55 32 7 16 16 23 14 32 20 49 19 30
## v2832 v2833 v2834 v2835 v2836 v2837 v2838 v2839 v2840 v2841 v2842 v2843
## 3 10 42 17 26 30 17 46 31 37 59 16
## v2844 v2845 v2846 v2847 v2848 v2849 v2850 v2851 v2852 v2853 v2854 v2855
## 13 38 28 16 16 28 31 20 19 27 12 21
## v2856 v2857 v2858 v2859 v2860 v2861 v2862 v2863 v2864 v2865 v2866 v2867
## 41 15 69 2 7 51 32 114 6 29 398 18
## v2868 v2869 v2870 v2871 v2872 v2873 v2874 v2875 v2876 v2877 v2878 v2879
## 62 10 37 6 13 14 47 128 15 3 36 14
## v2880 v2881 v2882 v2883 v2884 v2885 v2886 v2887 v2888 v2889 v2890 v2891
## 14 1 248 12 35 60 22 5 74 24 10 6
## v2892 v2893 v2894 v2895 v2896 v2897 v2898 v2899 v2900 v2901 v2902 v2903
## 11 133 80 45 102 11 56 13 78 289 197 11
## v2904 v2905 v2906 v2907 v2908 v2909 v2910 v2911 v2912 v2913 v2914 v2915
## 27 30 78 61 22 5 16 67 55 61 22 12
## v2916 v2917 v2918 v2919 v2920 v2921 v2922 v2923 v2924 v2925 v2926 v2927
## 66 16 24 1 7 7 55 10 68 18 8 39
## v2928 v2929 v2930 v2931 v2932 v2933 v2934 v2935 v2936 v2937 v2938 v2939
## 7 14 45 18 31 41 22 15 15 27 3 20
## v2940 v2941 v2942 v2943 v2944 v2945 v2946 v2947 v2948 v2949 v2950 v2951
## 12 51 18 24 66 7 2 11 5 1 21 8
## v2952 v2953 v2954 v2955 v2956 v2957 v2958 v2959 v2960 v2961 v2962 v2963
## 5 67 59 41 16 104 29 29 20 37 15 19
## v2964 v2965 v2966 v2967 v2968 v2969 v2970 v2971 v2972 v2973 v2974 v2975
## 1 7 17 12 47 7 7 105 10 9 5 21
## v2976 v2977 v2978 v2979 v2980 v2981 v2982 v2983 v2984 v2985 v2986 v2988
## 12 46 16 39 12 13 7 8 10 21 4 72
## v2989 v2990 v2991 v2992 v2993 v2994 v2995 v2996 v2997 v2998 v2999 v3000
## 171 34 77 72 199 44 48 147 250 113 111 119
## v3001 v3002 v3003 v3004 v3005 v3006 v3007 v3008 v3009 v3010 v3011 v3012
## 21 97 25 338 9 6 38 6 171 112 31 149
## v3013 v3014 v3015 v3016 v3017 v3018 v3019 v3020 v3021 v3022 v3023 v3024
## 113 34 43 69 47 40 133 26 35 16 26 39
## v3025 v3026 v3027 v3028 v3029 v3030 v3031 v3032 v3033 v3034 v3035 v3036
## 25 137 23 41 68 12 15 111 48 76 6 66
## v3037 v3038 v3039 v3040 v3041 v3042 v3043 v3044 v3045 v3046 v3047 v3048
## 32 70 22 20 27 130 32 23 8 28 198 92
## v3049 v3050 v3051 v3052 v3053 v3054 v3055 v3056 v3057 v3058 v3059 v3060
## 154 8 22 56 22 28 25 9 15 34 84 60
## v3061 v3062 v3063 v3064 v3065 v3066 v3067 v3068 v3069 v3070 v3071 v3072
## 29 31 37 101 12 9 12 32 14 320 12 5
## v3073 v3074 v3075 v3076 v3077 v3078 v3079 v3080 v3081 v3082 v3083 v3084
## 8 27 50 22 227 85 172 70 8 168 9 92
## v3085 v3086 v3087 v3088 v3089 v3090 v3091 v3092 v3093 v3094 v3095 v3096
## 11 37 23 107 9 57 87 48 50 12 26 25
## v3097 v3098 v3099 v3100 v3101 v3102 v3103 v3104 v3105 v3106 v3107 v3108
## 23 12 71 72 25 13 4 51 59 181 9 44
## v3109 v3110 v3111 v3112 v3113 v3114 v3115 v3116 v3117 v3118 v3119 v3120
## 5 65 12 11 54 7 45 89 70 21 13 11
## v3121 v3122 v3123 v3124 v3125 v3126 v3127 v3128 v3129 v3130 v3131 v3132
## 7 156 47 28 13 51 26 125 122 35 93 5
## v3133 v3134 v3135 v3136 v3137 v3138 v3139 v3140 v3141 v3142 v3143 v3144
## 67 31 213 50 57 18 26 111 3 121 62 25
## v3145 v3146 v3147 v3148 v3149 v3150 v3151 v3152 v3153 v3154 v3155 v3156
## 103 14 1 18 128 17 17 11 97 10 35 113
## v3157 v3158 v3159 v3160 v3161 v3162 v3163 v3164 v3165 v3166 v3167 v3168
## 17 112 92 13 60 30 17 7 17 22 5 90
## v3169 v3170 v3171 v3172 v3173 v3174 v3175 v3176 v3177 v3178 v3179 v3180
## 6 50 20 48 47 11 156 9 26 42 39 14
## v3181 v3182 v3183 v3184 v3185 v3186 v3187 v3188 v3189 v3190 v3191 v3192
## 7 2 23 16 6 53 15 15 50 36 16 70
## v3193 v3194 v3195 v3196 v3197 v3198 v3199 v3200 v3201 v3202 v3203 v3204
## 26 1 10 49 15 7 16 36 76 8 59 15
## v3205 v3206 v3207 v3208 v3209 v3210 v3211 v3212 v3213 v3214 v3215 v3216
## 36 15 9 35 9 18 18 8 18 35 14 33
## v3217 v3218 v3219 v3220 v3221 v3222 v3224 v3225 v3226 v3227 v3228 v3229
## 11 14 10 47 19 85 32 13 10 32 28 54
## v3230 v3231 v3232 v3233 v3234 v3235 v3236 v3237 v3238 v3239 v3240 v3241
## 7 22 25 18 13 4 7 52 11 29 134 88
## v3242 v3243 v3244 v3245 v3246 v3247 v3248 v3249 v3250 v3251 v3252 v3253
## 41 135 7 13 9 11 8 40 21 30 32 67
## v3254 v3255 v3256 v3257 v3258 v3259 v3260 v3261 v3262 v3263 v3264 v3265
## 192 205 39 22 22 10 25 57 100 26 97 126
## v3266 v3267 v3268 v3269 v3270 v3271 v3272 v3273 v3274 v3275 v3276 v3277
## 16 25 6 17 170 24 13 19 58 30 55 22
## v3278 v3279 v3280 v3281 v3282 v3283 v3284 v3285 v3286 v3287 v3288 v3289
## 22 16 14 13 31 16 17 45 19 34 10 15
## v3290 v3291 v3292 v3293 v3294 v3295 v3296 v3297 v3298 v3299 v3300 v3301
## 1 10 2 8 20 11 15 85 35 31 19 38
## v3302 v3303 v3304 v3305 v3306 v3307 v3308 v3309 v3310 v3311 v3312 v3313
## 7 356 43 117 33 18 24 15 27 248 13 25
## v3314 v3315 v3316 v3317 v3318 v3319 v3320 v3321 v3322 v3323 v3324 v3325
## 207 133 48 123 72 27 5 24 33 65 155 12
## v3326 v3327 v3328 v3329 v3330 v3331 v3332 v3333 v3334 v3335 v3336 v3337
## 69 10 28 126 19 53 29 15 36 64 12 43
## v3338 v3339 v3340 v3341 v3342 v3343 v3344 v3345 v3346 v3347 v3348 v3349
## 16 8 14 44 53 70 84 80 470 29 18 155
## v3350 v3351 v3352 v3353 v3354 v3355 v3356 v3357 v3358 v3359 v3360 v3361
## 182 24 40 44 11 1 29 36 41 117 17 35
## v3362 v3363 v3364 v3365 v3366 v3367 v3368 v3369 v3370 v3371 v3372 v3373
## 75 22 40 207 17 94 27 29 43 26 26 17
## v3374 v3375 v3376 v3377 v3378 v3379 v3380 v3381 v3382 v3383 v3384 v3385
## 53 42 16 38 12 5 17 70 16 8 66 102
## v3386 v3387 v3388 v3389 v3390 v3391 v3392 v3393 v3394 v3395 v3396 v3397
## 3 23 22 12 6 13 24 14 8 6 92 34
## v3398 v3399 v3400 v3401 v3402 v3403 v3404 v3405 v3406 v3408 v3409 v3410
## 18 35 20 21 18 87 29 2 98 11 59 47
## v3411 v3412 v3413 v3414 v3415 v3416 v3417 v3418 v3419 v3420 v3421 v3422
## 6 47 7 9 30 34 31 17 26 7 6 1
## v3423 v3424 v3425 v3426 v3427 v3428 v3429 v3430 v3431 v3432 v3433 v3434
## 94 66 50 55 2 8 38 7 22 52 8 13
## v3435 v3436 v3437 v3438 v3439 v3440 v3441 v3442 v3443 v3444 v3445 v3446
## 98 10 14 41 14 9 1 19 45 24 20 12
## v3447 v3448 v3449 v3450 v3451 v3452 v3453 v3454 v3455 v3456 v3457 v3458
## 15 9 17 30 43 11 13 21 99 24 7 11
## v3459 v3460 v3461 v3462 v3463 v3464 v3465 v3466 v3467 v3468 v3469 v3470
## 41 31 2 9 12 22 11 19 25 54 66 46
## v3471 v3472 v3473 v3474 v3475 v3476 v3477 v3478 v3479 v3480 v3481 v3482
## 47 40 37 4 17 77 41 28 16 47 17 20
## v3483 v3484 v3485 v3486 v3487 v3488 v3489 v3490 v3491 v3492 v3493 v3494
## 16 3 11 19 48 43 17 28 19 20 28 38
## v3495 v3496 v3497 v3498 v3499 v3500 v3501 v3502 v3503 v3504 v3505 v3506
## 11 46 26 24 12 5 18 11 10 12 22 22
## v3507 v3508 v3509 v3510 v3511 v3512 v3513 v3514 v3515 v3516 v3517 v3518
## 9 11 56 26 89 8 21 16 46 40 18 88
## v3519 v3520 v3521 v3522 v3523 v3524 v3525 v3526 v3527 v3528 v3529 v3530
## 11 37 4 28 42 1 6 8 7 14 39 31
## v3531 v3532 v3533 v3534 v3535 v3536 v3537 v3538 v3539 v3540 v3541 v3542
## 11 31 29 29 8 22 7 33 10 25 11 11
## v3543 v3544 v3545 v3546 v3547 v3548 v3549 v3550 v3551 v3552 v3553 v3554
## 14 56 16 5 25 14 66 31 7 28 25 67
## v3555 v3556 v3557 v3558 v3559 v3560 v3561 v3562 v3563 v3564 v3565 v3566
## 18 28 21 16 14 4 18 42 11 30 72 51
## v3567 v3568 v3569 v3570 v3571 v3572 v3573 v3574 v3575 v3576 v3577 v3578
## 7 51 15 49 28 13 29 12 8 57 182 12
## v3579 v3580 v3581 v3582 v3583 v3584 v3585 v3586 v3587 v3588 v3589 v3590
## 12 21 25 29 28 37 16 24 35 64 10 27
## v3591 v3592 v3593 v3594 v3595 v3596 v3597 v3598 v3599 v3600 v3601 v3602
## 33 10 32 47 30 21 30 39 30 37 45 82
## v3603 v3604 v3605 v3606 v3607 v3609 v3610 v3611 v3612 v3613 v3614 v3615
## 17 22 19 17 8 82 47 71 1 42 52 84
## v3616 v3617 v3618 v3619 v3620 v3621 v3622 v3623 v3624 v3625 v3626 v3627
## 31 13 13 33 67 105 20 28 17 9 35 21
## v3628 v3629 v3630 v3631 v3632 v3633 v3634 v3635 v3636 v3637 v3638 v3639
## 44 32 37 53 10 42 67 56 46 3 174 31
## v3640 v3641 v3642 v3643 v3644 v3645 v3646 v3647 v3648 v3649 v3650 v3651
## 29 104 23 71 18 25 19 45 36 56 29 36
## v3652 v3653 v3654 v3655 v3656 v3657 v3658 v3659 v3660 v3661 v3662 v3663
## 37 35 103 91 87 39 15 41 15 78 50 4
## v3664 v3665 v3666 v3667 v3668 v3669 v3670 v3671 v3672 v3673 v3674 v3675
## 216 17 15 3 22 56 78 31 45 22 17 2
## v3676 v3677 v3678 v3679 v3680 v3681 v3682 v3683 v3684 v3685 v3686 v3687
## 7 32 72 47 14 68 48 11 18 38 18 24
## v3689 v3690 v3691 v3692 v3693 v3694 v3695 v3696 v3697 v3698 v3699 v3700
## 8 20 18 37 15 22 15 28 24 10 11 1
## v3701 v3703 v3704 v3705 v3706 v3707 v3708 v3709 v3710 v3711 v3712 v3713
## 7 3 36 10 32 13 15 9 8 9 65 26
## v3714 v3715 v3716 v3717 v3718 v3719 v3720 v3721 v3722 v3723 v3724 v3725
## 6 7 8 8 9 26 7 13 9 39 11 57
## v3726 v3727 v3728 v3729 v3730 v3731 v3732 v3735 v3736 v3737 v3738 v3739
## 15 33 36 61 20 21 32 54 18 15 15 12
## v3740 v3741 v3742 v3743 v3744 v3745 v3746 v3747 v3748 v3749 v3750 v3751
## 7 18 8 6 11 32 12 61 55 89 55 5
## v3752 v3753 v3754 v3755 v3756 v3757 v3758 v3759 v3760 v3761 v3762 v3763
## 53 12 30 10 50 27 71 11 7 20 50 8
## v3764 v3765 v3766 v3767 v3768 v3769 v3770 v3771 v3772 v3773 v3774 v3775
## 7 17 15 19 53 28 26 34 56 34 46 45
## v3776 v3777 v3778 v3779 v3780 v3781 v3782 v3783 v3784 v3785 v3786 v3787
## 37 47 22 27 18 55 84 30 70 7 24 17
## v3788 v3789 v3790 v3791 v3792 v3793 v3794 v3795 v3796 v3797 v3798 v3799
## 9 6 22 8 23 7 19 1 17 40 8 7
## v3800 v3801 v3802 v3803 v3804 v3805 v3806 v3807 v3808 v3809 v3810 v3811
## 79 12 7 10 15 36 33 39 7 22 19 20
## v3812 v3813 v3814 v3815 v3816 v3817 v3818 v3819 v3820 v3821 v3822 v3823
## 47 16 22 14 39 32 12 9 17 9 8 30
## v3824 v3825 v3826 v3827 v3828 v3829 v3830 v3831 v3832 v3833 v3834 v3835
## 24 34 4 42 54 27 12 47 48 16 11 23
## v3836 v3837 v3838 v3839 v3840 v3841 v3842 v3843 v3844 v3845 v3846 v3847
## 22 4 9 13 9 10 45 32 119 2 22 23
## v3848 v3849 v3850 v3851 v3852 v3853 v3854 v3855 v3856 v3857 v3858 v3859
## 28 24 34 6 5 26 31 11 33 12 10 2
## v3860 v3861 v3862 v3863 v3864 v3865 v3866 v3867 v3868 v3869 v3870 v3871
## 34 24 17 2 21 5 17 6 112 5 27 44
## v3872 v3873 v3874 v3875 v3876 v3877 v3878 v3879 v3880 v3881 v3882 v3883
## 29 46 41 43 679 5 1 355 6 4 49 11
## v3884 v3885 v3886 v3887 v3888 v3889 v3890 v3891 v3892 v3893 v3894 v3895
## 52 7 8 13 47 1 10 41 76 28 198 5
## v3896 v3897 v3898 v3899 v3900 v3901 v3902 v3903 v3904 v3905 v3906 v3907
## 5 4 22 40 11 89 8 3 44 28 12 52
## v3908 v3909 v3910 v3911 v3912 v3913 v3914 v3915 v3916 v3917 v3918 v3919
## 12 31 7 13 48 6 404 3 20 17 4 1
## v3920 v3921 v3922 v3923 v3924 v3925 v3926 v3927 v3928 v3929 v3930 v3931
## 19 7 41 6 9 18 30 20 105 108 15 18
## v3932 v3933 v3934 v3935 v3936 v3937 v3938 v3939 v3940 v3941 v3942 v3943
## 46 5 4 23 2 8 60 100 18 37 197 15
## v3944 v3945 v3946 v3947 v3948 v3949 v3950 v3951 v3952 v3953 v3954 v3955
## 105 138 8 20 50 7 129 23 210 17 12 236
## v3956 v3957 v3958 v3959 v3960 v3961 v3962 v3963 v3964 v3965 v3966 v3967
## 18 7 42 42 10 85 143 2 20 27 105 125
## v3968 v3969 v3970 v3971 v3972 v3973 v3974 v3975 v3976 v3977 v3978 v3979
## 136 35 26 2 41 138 22 5 7 139 73 22
## v3980 v3981 v3982 v3983 v3984 v3985 v3986 v3987 v3988 v3989 v3990 v3991
## 54 28 84 33 104 16 40 33 21 44 11 21
## v3992 v3993 v3994 v3995 v3996 v3997 v3998 v3999 v4000 v4001 v4002 v4003
## 18 55 8 13 25 26 7 9 13 6 9 1
## v4004 v4005 v4006 v4007 v4008 v4009 v4010 v4011 v4012 v4013 v4014 v4015
## 158 10 22 428 10 16 114 276 42 9 66 10
## v4016 v4017 v4018 v4019 v4020 v4021 v4022 v4023 v4024 v4025 v4026 v4027
## 30 3 3 34 173 40 43 39 3 18 60 19
## v4028 v4029 v4030 v4031 v4032 v4033 v4034 v4035 v4036 v4037 v4038 v4039
## 10 46 47 37 10 16 7 134 83 55 12 3
## v4040 v4041 v4042 v4043 v4044 v4045 v4046 v4047 v4048 v4049 v4050 v4051
## 21 33 86 23 187 13 30 113 11 15 68 3
## v4052 v4053 v4054 v4055 v4056 v4057 v4058 v4059 v4060 v4061 v4062 v4063
## 29 3 4 9 11 23 6 4 13 3 15 147
## v4064 v4065 v4066 v4067 v4068 v4069 v4070 v4071 v4072 v4073 v4074 v4075
## 25 12 12 10 11 88 18 5 19 37 41 4
## v4076 v4077 v4078 v4079 v4080 v4081 v4082 v4083 v4084 v4085 v4086 v4087
## 18 11 5 2 9 8 3 1 25 11 158 373
## v4088 v4089 v4090 v4091 v4092 v4093 v4094 v4095 v4096 v4097 v4098 v4099
## 669 22 459 17 5 98 79 150 92 280 23 5
## v4100 v4101 v4102 v4103 v4104 v4105 v4106 v4107 v4108 v4109 v4110 v4112
## 4 4 23 77 80 10 29 8 9 6 13 4
## v4113 v4114 v4115 v4116 v4117 v4118 v4119 v4120 v4121 v4122 v4123 v4124
## 6 22 11 13 49 1 16 17 11 3 7 6
## v4125 v4126 v4127 v4128 v4129 v4130 v4131 v4132 v4133 v4134 v4135 v4136
## 10 4 5 9 2 2 2 8 6 3 2 8
## v4137 v4140 v4141 v4142 v4143 v4144 v4145 v4146 v4147 v4148 v4149 v4150
## 3 9 2 11 4 10 5 31 125 4 65 15
## v4151 v4152 v4153 v4154 v4155 v4156 v4157 v4158 v4159 v4160 v4161 v4162
## 4 5 12 10 9 6 29 7 22 48 22 5
## v4163 v4164 v4165 v4166 v4167 v4168 v4169 v4170 v4171 v4172 v4173 v4174
## 41 6 30 10 33 69 7 36 14 49 16 44
## v4175 v4176 v4177 v4178 v4179 v4180 v4181 v4182 v4183 v4184 v4185 v4186
## 90 13 11 5 27 4 27 6 37 14 7 21
## v4187 v4188 v4189 v4190 v4191 v4192 v4193 v4194 v4195 v4196 v4197 v4198
## 7 5 8 6 6 6 6 25 8 453 44 13
## v4199 v4200 v4201 v4202 v4203 v4204 v4205 v4206 v4207 v4208 v4209 v4210
## 48 5 2 4 87 13 21 11 40 32 63 6
## v4211 v4212 v4213 v4214 v4215 v4216 v4217 v4218 v4219 v4220 v4221 v4222
## 38 110 18 2 2 20 65 6 10 6 21 44
## v4223 v4224 v4225 v4226 v4227 v4228 v4229 v4230 v4231 v4232 v4233 v4234
## 11 10 7 24 4 6 16 5 8 9 6 5
## v4235 v4236 v4237 v4238 v4239 v4240 v4241 v4242 v4243 v4244 v4245 v4246
## 6 9 6 7 6 8 4 3 20 51 30 102
## v4247 v4248 v4249 v4250 v4251 v4252 v4253 v4254 v4255 v4256 v4257 v4258
## 191 5 8 26 8 3 11 21 12 5 4 23
## v4259 v4260 v4261 v4262 v4263 v4264 v4265 v4266 v4267 v4268 v4269 v4270
## 1 15 11 11 8 7 7 4 13 28 47 2
## v4271 v4272 v4273 v4274 v4275 v4276 v4277 v4278 v4279 v4280 v4281 v4282
## 17 8 14 51 15 6 11 17 12 10 11 14
## v4283 v4284 v4285 v4286 v4287 v4288 v4289 v4290 v4291 v4292 v4293 v4294
## 13 8 7 3 5 20 11 20 5 12 44 54
## v4295 v4296 v4297 v4298 v4299 v4300 v4301 v4302 v4303 v4304 v4305 v4306
## 46 66 16 43 4 33 29 3 12 43 37 69
## v4307 v4308 v4309 v4310 v4311 v4312 v4313 v4314 v4315 v4316 v4317 v4318
## 72 72 55 158 30 55 27 6 4 8 9 112
## v4319 v4320 v4321 v4322 v4323 v4324 v4325 v4326 v4327 v4328 v4329 v4330
## 32 74 37 15 71 13 13 12 8 8 19 11
## v4331 v4332 v4333 v4334 v4335 v4336 v4337 v4338 v4339 v4340 v4341 v4342
## 15 13 11 10 22 12 1 43 5 25 11 11
## v4343 v4344 v4345 v4346 v4347 v4348 v4349 v4350 v4351 v4352 v4353 v4354
## 6 38 7 44 13 30 12 41 1 11 42 5
## v4355 v4356 v4357 v4358 v4359 v4360 v4361 v4362 v4363 v4364 v4365 v4366
## 25 5 16 2 4 12 15 7 9 3 1 8
## v4367 v4368 v4369 v4370 v4371 v4372 v4373 v4374 v4375 v4376 v4377 v4378
## 7 18 21 3 20 6 14 7 5 36 3 10
## v4379 v4380 v4381 v4382 v4383 v4384 v4385 v4386 v4387 v4388 v4389 v4390
## 18 21 15 21 40 62 3 31 17 5 10 13
## v4391 v4392 v4393 v4394 v4395 v4396 v4397 v4398 v4399 v4400 v4401 v4402
## 10 7 120 21 55 32 1 103 27 7 17 13
## v4403 v4404 v4405 v4406 v4407 v4408 v4409 v4410 v4411 v4412 v4413 v4414
## 22 39 3 93 5 64 10 10 42 7 65 35
## v4415 v4416 v4417 v4418 v4419 v4420 v4421 v4422 v4423 v4424 v4425 v4426
## 32 52 39 4 4 3 72 4 9 5 15 3
## v4427 v4428 v4429 v4430 v4431 v4432 v4433 v4434 v4435 v4436 v4437 v4438
## 2 17 93 21 12 34 16 8 5 6 10 24
## v4439 v4440 v4441 v4442 v4443 v4444 v4445 v4446 v4447 v4448 v4449 v4450
## 2 48 9 18 15 47 70 7 10 118 4 12
## v4452 v4453 v4454 v4455 v4456 v4457 v4458 v4459 v4460 v4461 v4462 v4463
## 11 51 11 14 4 34 9 9 10 7 3 12
## v4464 v4465 v4466 v4467 v4468 v4469 v4470 v4471 v4472 v4473 v4474 v4475
## 32 62 35 7 11 8 14 2 8 4 10 3
## v4476 v4477 v4478 v4479 v4480 v4481 v4482 v4483 v4484 v4485 v4486 v4487
## 11 20 44 15 11 4 8 7 11 10 5 11
## v4488 v4489 v4490 v4491 v4492 v4493 v4494 v4495 v4496 v4497 v4498 v4499
## 49 1 5 2 2 8 21 7 3 32 8 19
## v4500 v4501 v4502 v4503 v4504 v4505 v4506 v4507 v4508 v4509 v4510 v4511
## 23 10 3 3 10 19 17 6 5 34 11 6
## v4512 v4513 v4514 v4515 v4516 v4517 v4518 v4519 v4520 v4521 v4522 v4523
## 21 6 16 1 18 9 10 9 586 2 12 1
## v4524 v4525 v4526 v4527 v4528 v4529 v4530 v4531 v4532 v4533 v4534 v4535
## 36 68 12 8 7 8 3 23 5 51 6 8
## v4536 v4537 v4538 v4539 v4540 v4541 v4542 v4543 v4544 v4545 v4546 v4547
## 22 11 8 25 24 36 21 58 2 49 23 6
## v4548 v4549 v4550 v4551 v4552 v4553 v4554 v4555 v4556 v4557 v4558 v4559
## 13 7 3 9 9 10 5 12 15 12 9 13
## v4560 v4561 v4562 v4563 v4564 v4565 v4566 v4567 v4568 v4569 v4570 v4571
## 28 13 3 19 9 8 5 3 24 177 6 13
## v4572 v4573 v4574 v4575 v4576 v4577 v4578 v4579 v4580 v4581 v4582 v4583
## 12 14 89 1 301 1749 21 60 64 30 8 237
## v4584 v4585 v4586 v4587 v4588 v4589 v4590 v4591 v4592 v4593 v4594 v4595
## 111 15 69 4 30 3 3 3 3 1 6 1
## v4596 v4597 v4598 v4599 v4600 v4601 v4602 v4603 v4604 v4605 v4606 v4607
## 14 14 5 23 9 14 65 150 9 547 17 12
## v4608 v4609 v4610 v4611 v4612 v4613 v4614 v4615 v4616 v4617 v4618 v4619
## 6 510 7 55 10 5 79 21 5 9 70 71
## v4620 v4621 v4622 v4623 v4624 v4625 v4626 v4627 v4628 v4629 v4630 v4631
## 11 10 48 2 535 19 14 13 6 27 62 2
## v4632 v4633 v4634 v4635 v4636 v4637 v4638 v4639 v4640 v4641 v4642 v4643
## 8 4 2 16 3 13 6 18 24 100 42 2
## v4644 v4645 v4646 v4647 v4648 v4649 v4650 v4651 v4652 v4653 v4654 v4655
## 30 78 7 8 1 83 4 43 2 43 6 2
## v4656 v4657 v4658 v4659 v4660 v4661 v4662 v4663 v4664 v4665 v4666 v4667
## 81 146 69 6 34 36 64 84 91 41 12 20
## v4668 v4669 v4670 v4671 v4672 v4673 v4674 v4675 v4676 v4677 v4678 v4679
## 11 57 73 7 64 27 2 30 10 17 3 2
## v4680 v4681 v4682 v4683 v4684 v4685 v4686 v4687 v4688 v4689 v4690 v4691
## 3 53 42 17 27 34 12 34 26 40 501 422
## v4692 v4693 v4694 v4695 v4696 v4697 v4698 v4699 v4700 v4701 v4702 v4703
## 37 13 7 7 6 1 1 4 31 8 3 4
## v4704 v4705 v4706 v4707 v4708 v4709 v4710 v4711 v4712 v4713 v4714 v4715
## 3 2 7 1 13 4 20 30 35 20 26 45
## v4716 v4717 v4719 v4720 v4721 v4722 v4723 v4724 v4725 v4726 v4727 v4728
## 48 3 70 24 52 18 11 9 31 85 1 4
## v4729 v4730 v4731 v4732 v4733 v4734 v4735 v4736 v4737 v4738 v4739 v4740
## 4 13 23 9 6 2 3 4 11 9 4 1
## v4741 v4742 v4743 v4744 v4745 v4746 v4747 v4748 v4749 v4750 v4751 v4752
## 9 1 6 5 8 10 1 59 32 192 11 24
## v4753 v4754 v4755 v4756 v4757 v4758 v4759 v4760 v4761 v4762 v4763 v4764
## 179 18 13 78 53 3 11 68 1 4 1 2
## v4765 v4766 v4767 v4768 v4769 v4770 v4771 v4772 v4773 v4774 v4775 v4776
## 9 6 5 16 25 44 15 5 16 3 25 4
## v4777 v4778 v4779 v4780 v4781 v4782 v4783 v4784 v4785 v4786 v4787 v4788
## 11 9 15 8 12 27 2 25 11 9 25 2
## v4789 v4790 v4791 v4792 v4793 v4794 v4795 v4796 v4797 v4798 v4799 v4800
## 128 5 15 7 24 35 17 16 16 26 21 9
## v4801 v4802 v4803 v4804 v4805 v4806 v4807 v4808 v4809 v4810 v4811 v4813
## 74 9 6 13 17 3 27 9 4 8 6 69
## v4814 v4815 v4816 v4817 v4818 v4819 v4820 v4821 v4822 v4823 v4824 v4825
## 1 8 22 43 4 3 17 4 42 337 542 6
## v4826 v4827 v4828 v4829 v4830 v4831 v4832 v4833 v4834 v4835 v4836 v4838
## 33 116 41 125 78 663 956 112 977 106 14 49
## v4839 v4840 v4841 v4842 v4843 v4844 v4845 v4846 v4847 v4848 v4850 v4851
## 3 147 5 19 9 6 5 3 12 6 15 13
## v4852 v4853 v4854 v4855 v4856 v4858 v4859 v4860 v4861 v4862 v4863 v4864
## 5 6 1 5 5 5 3 71 5 66 42 6
## v4865 v4866 v4867 v4868 v4869 v4870 v4871 v4872 v4873 v4874 v4875 v4876
## 2 32 10 9 4 30 2 6 2 10 2 9
## v4877 v4878 v4879 v4880 v4881 v4882 v4883 v4885 v4886 v4887 v4888 v4890
## 30 31 4 17 11 8 37 33 6 7 6 2
## v4891 v4892 v4893 v4894 v4895 v4896 v4897 v4898 v4899 v4900 v4901 v4902
## 4 16 6 7 1 6 33 4 4 1 6 7
## v4903 v4904 v4905 v4906 v4907 v4908 v4909 v4910 v4911 v4912 v4913 v4914
## 10 21 3 9 5 4 18 6 37 5 3 6
## v4915 v4916 v4917 v4918 v4919 v4920 v4921 v4922 v4923 v4924 v4925 v4926
## 5 3 2 9 32 18 24 5 12 31 11 18
## v4927 v4928 v4929 v4930 v4931 v4932 v4933 v4934 v4935 v4936 v4937 v4938
## 8 14 5 6 4 15 8 4 1 56 11 17
## v4939 v4940 v4941 v4942 v4943 v4946 v4947 v4948 v4949 v4950 v4951 v4952
## 6 14 3 2 9 1 29 55 24 15 38 92
## v4953 v4954 v4955 v4956 v4957 v4958 v4959 v4960 v4961 v4962 v4963 v4964
## 205 48 5 17 4 11 87 429 75 9 4 8
## v4965 v4966 v4967 v4969 v4970 v4971 v4972 v4973 v4974 v4975 v4976 v4977
## 39 40 26 13 5 17 41 5 1 55 31 87
## v4978 v4979 v4980 v4981 v4982 v4983 v4984 v4985 v4986 v4987 v4988 v4989
## 37 57 5 7 650 26 4 235 30 37 13 9
## v4990 v4991 v4993 v4994 v4995 v4996 v4997 v4998 v4999 v5000 v5001 v5002
## 31 6 13 36 4 42 15 149 4 22 5 8
## v5003 v5004 v5005 v5006 v5008 v5009 v5010 v5011 v5012 v5013 v5014 v5015
## 18 10 4 3 240 57 4 15 2 14 57 19
## v5016 v5017 v5018 v5019 v5020 v5021 v5022 v5023 v5024 v5025 v5026 v5027
## 234 39 8 7 64 3 101 1 41 13 25 24
## v5028 v5029 v5030 v5031 v5032 v5033 v5034 v5035 v5036 v5037 v5038 v5039
## 19 105 9 42 29 2 14 100 29 119 30 21
## v5040 v5042 v5043 v5044 v5045 v5046 v5047 v5048 v5049 v5050 v5051 v5052
## 8 1 2 3 4 23 10 14 7 5 29 127
## v5053 v5054 v5055 v5056 v5057 v5058 v5059 v5060 v5061 v5062 v5063 v5064
## 49 80 2 4 69 470 48 64 56 96 11 23
## v5065 v5066 v5067 v5068 v5069 v5070 v5071 v5072 v5073 v5074 v5075 v5076
## 278 16 57 563 245 51 22 40 95 1 13 51
## v5077 v5078 v5079 v5080 v5081 v5082 v5083 v5084 v5085 v5086 v5087 v5088
## 128 9 23 2 20 94 17 11 5 141 6 3
## v5089 v5090 v5091 v5092 v5093 v5094 v5095 v5096 v5097 v5098 v5099 v5100
## 30 16 2 17 32 3 156 3 4 4 38 2
## v5101 v5102 v5103 v5104 v5105 v5106 v5107 v5108 v5109 v5110 v5111 v5112
## 23 7 5 14 11 27 6 9 30 11 35 8
## v5113 v5114 v5115 v5116 v5117 v5118 v5119 v5120 v5121 v5122 v5123 v5124
## 4 3 7 9 7 7 4 9 20 8 7 7
## v5125 v5126 v5127 v5128 v5129 v5130 v5131 v5132 v5133 v5134 v5135 v5136
## 18 3 10 1 21 47 16 19 56 50 86 35
## v5137 v5138 v5139 v5140 v5141 v5142 v5143 v5144 v5145 v5146 v5147 v5148
## 426 160 13 14 20 5 56 102 4 9 45 3
## v5149 v5150 v5151 v5152 v5153 v5154 v5155 v5156 v5157 v5158 v5159 v5160
## 2 9 21 6 46 58 93 10 16 159 17 11
## v5161 v5162 v5163 v5164 v5165 v5166 v5167 v5168 v5169 v5170 v5171 v5172
## 17 1 31 12 13 31 24 25 11 12 15 15
## v5173 v5174 v5175 v5176 v5177 v5178 v5179 v5180 v5181 v5182 v5183 v5184
## 2 4 6 14 4 5 44 9 117 43 7 26
## v5185 v5186 v5187 v5188 v5189 v5190 v5191 v5192 v5193 v5194 v5195 v5196
## 74 34 39 82 31 6 3 8 31 2 7 7
## v5197 v5198 v5199 v5200 v5201 v5202 v5203 v5204 v5205 v5206 v5207 v5208
## 5 10 44 7 56 16 3 3 7 4 1 4
## v5209 v5210 v5211 v5212 v5213 v5214 v5215 v5216 v5217 v5218 v5219 v5220
## 3 1 1 1 4 5 1 1 8 2 4 2
## v5221 v5223 v5224 v5225 v5226 v5227 v5228 v5229 v5230 v5231 v5232 v5233
## 8 3 20 3 15 4 4 6 50 9 5 18
## v5234 v5235 v5236 v5237 v5238 v5239 v5240 v5241 v5242 v5243 v5244 v5245
## 15 15 7 17 18 8 3 72 29 26 102 14
## v5246 v5247 v5248 v5249 v5250 v5251 v5252 v5253 v5254 v5255 v5256 v5257
## 4 13 12 79 32 79 19 7 6 13 12 3
## v5258 v5259 v5260 v5261 v5262 v5263 v5264 v5265 v5266 v5267 v5268 v5269
## 5 8 7 53 4 1 87 75 11 29 24 46
## v5270 v5271 v5272 v5273 v5274 v5275 v5276 v5277 v5278 v5279 v5280 v5281
## 186 23 13 24 8 6 13 4 2 3 1 1
## v5282 v5283 v5284 v5285 v5286 v5287 v5288 v5289 v5290 v5291 v5292 v5294
## 30 2 13 9 10 3 5 8 13 4 1 6
## v5295 v5296 v5297 v5298 v5299 v5300 v5301 v5302 v5303 v5304 v5305 v5306
## 6 3 61 5 45 52 38 2 8 3 7 20
## v5307 v5308 v5309 v5310 v5311 v5312 v5313 v5314 v5315 v5316 v5317 v5318
## 9 6 16 131 50 9 8 23 38 5 18 30
## v5319 v5320 v5321 v5322 v5323 v5324 v5325 v5326 v5327 v5328 v5329 v5330
## 7 3 11 2 6 6 20 9 3 11 34 107
## v5331 v5332 v5333 v5334 v5335 v5336 v5337 v5338 v5339 v5340 v5341 v5342
## 12 18 4 34 11 12 12 2 2 1 11 4
## v5343 v5344 v5345 v5346 v5347 v5348 v5349 v5350 v5351 v5352 v5353 v5354
## 9 8 31 17 16 6 7 2 7 10 8 1
## v5355 v5356 v5357 v5358 v5359 v5360 v5361 v5362 v5363 v5364 v5365 v5366
## 7 16 4 6 6 4 11 11 5 8 46 12
## v5367 v5368 v5369 v5370 v5371 v5372 v5373 v5374 v5375 v5376 v5377 v5378
## 4 8 32 19 27 37 14 17 6 8 3 5
## v5379 v5380 v5381 v5382 v5383 v5384 v5385 v5386 v5387 v5388 v5389 v5390
## 38 7 9 4 20 11 10 4 72 1 6 5
## v5391 v5392 v5393 v5394 v5395 v5396 v5397 v5398 v5399 v5400 v5401 v5402
## 19 46 199 16 5 5 20 11 6 20 6 21
## v5403 v5404 v5405 v5406 v5407 v5408 v5409 v5410 v5411 v5412 v5413 v5414
## 3 9 4 26 17 13 16 2 1 12 4 25
## v5415 v5416 v5417 v5418 v5419 v5420 v5421 v5422 v5423 v5424 v5425 v5426
## 43 18 14 21 3 18 118 28 31 4 28 3
## v5427 v5428 v5429 v5430 v5431 v5432 v5433 v5434 v5435 v5436 v5437 v5438
## 18 12 5 11 11 39 11 14 12 1 7 2
## v5439 v5440 v5441 v5442 v5443 v5444 v5445 v5446 v5447 v5448 v5449 v5450
## 1 8 1 28 2 5 9 4 38 14 7 1
## v5451 v5452 v5453 v5454 v5455 v5456 v5457 v5458 v5459 v5460 v5461 v5462
## 5 1 63 15 72 9 5 12 3 5 5 4
## v5463 v5464 v5466 v5467 v5468 v5469 v5471 v5472 v5473 v5474 v5475 v5476
## 20 13 2 15 9 9 99 5 5 5 5 24
## v5477 v5478 v5479 v5480 v5481 v5482 v5483 v5484 v5485 v5486 v5487 v5488
## 25 29 15 6 41 6 2 7 3 1 14 17
## v5489 v5490 v5491 v5492 v5493 v5494 v5495 v5496 v5497 v5498 v5499 v5500
## 2 7 60 26 18 174 46 4 4 27 4 1
## v5501 v5502 v5503 v5504 v5505 v5506 v5507 v5508 v5509 v5510 v5511 v5512
## 1 8 1 6 94 4 8 28 2 7 18 8
## v5513 v5514 v5516 v5517 v5518 v5519 v5520 v5521 v5522 v5523 v5524 v5525
## 22 8 21 5 1 16 23 13 2 23 6 22
## v5526 v5527 v5528 v5529 v5530 v5531 v5532 v5533 v5534 v5535 v5536 v5537
## 3 10 5 48 200 78 16 57 4 5 5 2
## v5538 v5539 v5540 v5541 v5542 v5543 v5544 v5545 v5546 v5547 v5548 v5549
## 5 6 92 11 4 13 3 29 8 6 2 10
## v5550 v5551 v5552 v5553 v5554 v5555 v5556 v5557 v5559 v5560 v5561 v5562
## 9 2 15 10 4 3 1 1 9 16 9 1
## v5563 v5564 v5565 v5566 v5567 v5568 v5569 v5570 v5571 v5572 v5573 v5574
## 4 10 5 5 2 4 11 26 4 32 7 18
## v5575 v5576 v5577 v5578 v5579 v5580 v5581 v5582 v5583 v5585 v5586 v5587
## 11 5 4 22 12 3 3 14 6 8 27 4
## v5588 v5590 v5591 v5592 v5593 v5594 v5595 v5596 v5597 v5598 v5599 v5600
## 15 4 8 8 21 11 20 377 7 3 3 6
## v5601 v5602 v5603 v5604 v5605 v5606 v5607 v5608 v5609 v5610 v5611 v5612
## 69 4 1 5 12 29 1 1 330 3 1 94
## v5613 v5614 v5615 v5616 v5617 v5618 v5619 v5620 v5621 v5622 v5623 v5624
## 14 2 6 3 8 2 1 10 3 6 9 2
## v5625 v5626 v5627 v5628 v5629 v5630 v5631 v5632 v5633 v5634 v5635 v5636
## 2 26 2 13 3 1 3 8 3 4 14 16
## v5637 v5638 v5639 v5640 v5641 v5642 v5643 v5644 v5645 v5646 v5647 v5648
## 5 33 2 2 2 2 11 6 4 4 8 3
## v5649 v5650 v5651 v5652 v5653 v5654 v5655 v5656 v5657 v5658 v5659 v5660
## 10 1 1 2 1 15 4 4 5 16 7 1
## v5661 v5662 v5663 v5664 v5665 v5666 v5667 v5668 v5669 v5670 v5671 v5672
## 3 1 26 2 1 3 1 8 2 27 11 2
## v5673 v5674 v5675 v5676 v5677 v5678 v5679 v5680 v5681 v5683 v5684 v5685
## 1 3 4 24 41 1 1 4 10 4 23 3
## v5686 v5687 v5688 v5689 v5690 v5691 v5693 v5694 v5695 v5696 v5697 v5698
## 1 11 11 11 10 53 7 2 19 42 231 22
## v5699 v5700 v5701 v5702 v5703 v5704 v5705 v5706 v5707 v5709 v5710 v5711
## 74 177 54 5 36 31 51 53 1 7 3 314
## v5712 v5713 v5714 v5715 v5716 v5717 v5718 v5719 v5720 v5721 v5722 v5723
## 14 45 14 12 2 155 190 4 91 375 16 113
## v5724 v5725 v5726 v5727 v5728 v5729 v5730 v5731 v5732 v5733 v5734 v5735
## 35 8 3 35 68 41 114 26 7 238 72 28
## v5736 v5737 v5738 v5739 v5740 v5741 v5742 v5743 v5744 v5745 v5746 v5747
## 51 70 40 2 64 25 25 10 40 66 7 36
## v5748 v5749 v5750 v5751 v5752 v5753 v5754 v5755 v5756 v5757 v5758 v5759
## 18 12 24 20 20 25 16 39 86 34 32 34
## v5760 v5761 v5762 v5763 v5764 v5765 v5766 v5767 v5768 v5769 v5770 v5771
## 9 3 8 57 3 3 1 3 1 2 1 8
## v5772 v5773 v5774 v5775 v5776 v5777 v5778 v5779 v5780 v5781 v5782 v5783
## 41 5 16 16 2 45 13 18 3 45 32 11
## v5784 v5785 v5786 v5787 v5788 v5789 v5790 v5791 v5792 v5793 v5794 v5795
## 3 11 15 28 3 5 23 2 19 21 7 18
## v5796 v5797 v5798 v5799 v5800 v5801 v5802 v5803 v5804 v5805 v5806 v5807
## 13 7 3 1 112 36 7 2 1 6 17 14
## v5808 v5809 v5810 v5811 v5812 v5813 v5814 v5815 v5816 v5817 v5818 v5819
## 20 10 3 16 13 3 11 14 34 12 1 9
## v5820 v5822 v5823 v5824 v5825 v5826 v5827 v5828 v5829 v5830 v5831 v5832
## 7 1 15 6 11 6 3 10 2 42 12 15
## v5833 v5834 v5835 v5836 v5837 v5838 v5839 v5840 v5841 v5842 v5843 v5844
## 20 10 2 2 1 8 1 1 5 2 11 6
## v5845 v5846 v5847 v5848 v5849 v5850 v5851 v5852 v5853 v5854 v5855 v5856
## 20 7 19 1 21 10 2 21 2 5 20 1
## v5857 v5858 v5859 v5860 v5861 v5862 v5863 v5864 v5865 v5866 v5867 v5868
## 10 9 1 4 6 22 8 3 6 3 2 3
## v5869 v5870 v5871 v5872 v5873 v5874 v5875 v5876 v5877 v5878 v5879 v5880
## 8 1 3 5 5 1 2 1 1 3 15 7
## v5881 v5882 v5883 v5884 v5885 v5886 v5887 v5888 v5890 v5891 v5893 v5894
## 6 9 54 42 1 14 18 4 68 2 3 5
## v5895 v5896 v5897 v5898 v5899 v5900 v5901 v5902 v5903 v5904 v5905 v5906
## 38 83 8 12 17 3 10 41 5 37 19 2
## v5907 v5908 v5909 v5910 v5911 v5912 v5913 v5914 v5915 v5916 v5917 v5918
## 15 14 32 19 23 2 5 9 46 7 2 15
## v5919 v5920 v5921 v5922 v5923 v5924 v5925 v5926 v5927 v5928 v5929 v5930
## 14 6 11 10 19 3 2 5 2 8 2 19
## v5931 v5932 v5933 v5934 v5935 v5936 v5937 v5938 v5939 v5940 v5941 v5942
## 1 3 4 32 7 2 3 4 6 5 2 2
## v5943 v5944 v5945 v5946 v5947 v5948 v5949 v5950 v5951 v5952 v5953 v5954
## 2 13 2 10 2 2 3 32 6 2 7 1
## v5955 v5956 v5957 v5958 v5959 v5960 v5961 v5962 v5963 v5964 v5965 v5966
## 3 1 3 3 4 14 6 10 16 28 3 8
## v5967 v5968 v5969 v5971 v5972 v5973 v5974 v5975 v5976 v5977 v5978 v5979
## 112 12 2 1 3 4 6 18 1 2 9 674
## v5980 v5981 v5982 v5983 v5984 v5985 v5986 v5987 v5988 v5989 v5990 v5991
## 8 32 57 18 1 15 4 31 14 4 21 3
## v5992 v5993 v5994 v5995 v5996 v5997 v5998 v5999 v6000 v6001 v6002 v6003
## 3 46 2 3 5 1 1 3 6 80 1 50
## v6004 v6005 v6006 v6007 v6008 v6009 v6011 v6012 v6013 v6014 v6015 v6016
## 1 1 5 2 1 3 1 5 5 16 21 77
## v6017 v6018 v6019 v6020 v6021 v6022 v6023 v6024 v6025 v6026 v6027 v6028
## 1 2 2 1 4 4 1 1 4 1 1 1
## v6029 v6030 v6031 v6032 v6033 v6034 v6035 v6036 v6037 v6038 v6039 v6040
## 1 1 3 28 11 2 1 1 3 2 1 1
## v6041 v6042 v6043 v6044 v6045 v6046 v6048 v6050 v6051 v6052 v6053 v6054
## 4 1 2 1 4 5 1 1 1 3 4 1
## v6055 v6056 v6057 v6058 v6059 v6060 v6061 v6062 v6063 v6064 v6065 v6066
## 1 4 5 1 1 2 1 3 1 1 1 2
## v6067 v6068 v6069 v6070
## 4 1 1 7Then table of counts of transactions per product:
(totP <- table(sales$Prod))##
## p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12
## 210 81 35 114 161 63 52 11 38 38 266 64
## p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24
## 55 40 147 184 76 19 109 170 25 31 56 29
## p25 p26 p27 p28 p29 p30 p31 p32 p33 p34 p35 p36
## 36 128 64 45 42 69 24 47 213 46 198 137
## p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 p47 p48
## 62 13 14 15 77 96 28 121 78 156 13 21
## p49 p50 p51 p52 p53 p54 p55 p56 p57 p58 p59 p60
## 88 43 46 24 25 13 13 32 105 18 23 34
## p61 p62 p63 p64 p65 p66 p67 p68 p69 p70 p71 p72
## 96 30 86 20 15 40 47 36 22 33 48 35
## p73 p74 p75 p76 p77 p78 p79 p80 p81 p82 p83 p84
## 27 94 14 11 54 12 30 16 51 26 136 29
## p85 p86 p87 p88 p89 p90 p91 p92 p93 p94 p95 p96
## 16 46 26 63 17 77 57 45 20 27 82 90
## p97 p98 p99 p100 p101 p102 p103 p104 p105 p106 p107 p108
## 44 51 30 165 38 12 21 26 25 27 24 35
## p109 p110 p111 p112 p113 p114 p115 p116 p117 p118 p119 p120
## 52 39 26 113 75 55 130 68 23 30 29 12
## p121 p122 p123 p124 p125 p126 p127 p128 p129 p130 p131 p132
## 50 169 15 13 14 20 32 71 50 139 14 29
## p133 p134 p135 p136 p137 p138 p139 p140 p141 p142 p143 p144
## 15 14 23 157 36 19 44 19 11 28 63 33
## p145 p146 p147 p148 p149 p150 p151 p152 p153 p154 p155 p156
## 14 58 13 37 17 31 31 63 55 18 38 133
## p157 p158 p159 p160 p161 p162 p163 p164 p165 p166 p167 p168
## 39 112 23 52 16 12 13 14 29 124 17 45
## p169 p170 p171 p172 p173 p174 p175 p176 p177 p178 p179 p180
## 11 14 269 14 12 17 16 13 35 143 79 19
## p181 p182 p183 p184 p185 p186 p187 p188 p189 p190 p191 p192
## 139 42 17 12 23 29 19 27 11 19 90 26
## p193 p194 p195 p196 p197 p198 p199 p200 p201 p202 p203 p204
## 13 23 16 11 26 15 137 50 23 34 41 48
## p205 p206 p207 p208 p209 p210 p211 p212 p213 p214 p215 p216
## 91 42 46 14 11 27 58 12 84 138 163 80
## p217 p218 p219 p220 p221 p222 p223 p224 p225 p226 p227 p228
## 18 41 13 54 88 77 19 17 228 194 109 114
## p229 p230 p231 p232 p233 p234 p235 p236 p237 p238 p239 p240
## 21 95 27 57 18 26 80 12 130 44 12 36
## p241 p242 p243 p244 p245 p246 p247 p248 p249 p250 p251 p252
## 26 27 11 20 97 20 17 12 61 65 34 30
## p253 p254 p255 p256 p257 p258 p259 p260 p261 p262 p263 p264
## 28 39 33 82 50 34 20 15 33 12 16 13
## p265 p266 p267 p268 p269 p270 p271 p272 p273 p274 p275 p276
## 24 17 19 42 62 83 18 76 19 91 63 37
## p277 p278 p279 p280 p281 p282 p283 p284 p285 p286 p287 p288
## 20 61 29 19 16 82 44 86 175 41 123 93
## p289 p290 p291 p292 p293 p294 p295 p296 p297 p298 p299 p300
## 173 128 43 50 101 116 67 32 21 46 22 57
## p301 p302 p303 p304 p305 p306 p307 p308 p309 p310 p311 p312
## 31 16 18 88 43 21 17 32 21 17 11 36
## p313 p314 p315 p316 p317 p318 p319 p320 p321 p322 p323 p324
## 27 191 44 21 48 58 90 22 31 52 35 72
## p325 p326 p327 p328 p329 p330 p331 p332 p333 p334 p335 p336
## 20 16 22 50 19 41 23 20 19 22 44 11
## p337 p338 p339 p340 p341 p342 p343 p344 p345 p346 p347 p348
## 26 20 25 26 34 146 29 18 13 14 33 16
## p349 p350 p351 p352 p353 p354 p355 p356 p357 p358 p359 p360
## 12 12 12 22 14 29 36 19 11 24 12 12
## p361 p362 p363 p364 p365 p366 p367 p368 p369 p370 p371 p372
## 20 28 42 29 85 38 34 20 56 141 94 113
## p373 p374 p375 p376 p377 p378 p379 p380 p381 p382 p383 p384
## 20 29 11 28 21 53 73 31 20 26 71 31
## p385 p386 p387 p388 p389 p390 p391 p392 p393 p394 p395 p396
## 15 26 13 12 21 61 28 19 28 69 26 20
## p397 p398 p399 p400 p401 p402 p403 p404 p405 p406 p407 p408
## 23 14 41 26 44 14 17 17 25 22 20 23
## p409 p410 p411 p412 p413 p414 p415 p416 p417 p418 p419 p420
## 25 31 36 46 135 43 17 84 26 28 74 21
## p421 p422 p423 p424 p425 p426 p427 p428 p429 p430 p431 p432
## 49 77 21 118 85 47 119 20 28 32 119 389
## p433 p434 p435 p436 p437 p438 p439 p440 p441 p442 p443 p444
## 23 52 32 53 11 58 108 33 18 250 37 147
## p445 p446 p447 p448 p449 p450 p451 p452 p453 p454 p455 p456
## 201 44 45 83 46 166 61 78 318 21 50 29
## p457 p458 p459 p460 p461 p462 p463 p464 p465 p466 p467 p468
## 27 54 79 36 114 30 29 48 26 15 14 14
## p469 p470 p471 p472 p473 p474 p475 p476 p477 p478 p479 p480
## 22 41 75 107 55 42 17 19 40 44 40 38
## p481 p482 p483 p484 p485 p486 p487 p488 p489 p490 p491 p492
## 22 20 28 20 47 22 31 35 27 62 46 71
## p493 p494 p495 p496 p497 p498 p499 p500 p501 p502 p503 p504
## 95 64 60 35 77 55 25 32 76 172 13 14
## p505 p506 p507 p508 p509 p510 p511 p512 p513 p514 p515 p516
## 51 32 24 31 25 11 36 43 12 22 14 57
## p517 p518 p519 p520 p521 p522 p523 p524 p525 p526 p527 p528
## 14 15 16 61 12 18 15 24 12 26 62 29
## p529 p530 p531 p532 p533 p534 p535 p536 p537 p538 p539 p540
## 33 47 13 144 21 13 62 132 40 682 22 15
## p541 p542 p543 p544 p545 p546 p547 p548 p549 p550 p551 p552
## 43 72 26 64 214 17 82 40 53 15 69 67
## p553 p554 p555 p556 p557 p558 p559 p560 p561 p562 p563 p564
## 22 15 68 54 48 27 69 19 36 28 18 25
## p565 p566 p567 p568 p569 p570 p571 p572 p573 p574 p575 p576
## 18 104 91 26 38 66 15 36 88 27 30 14
## p577 p578 p579 p580 p581 p582 p583 p584 p585 p586 p587 p588
## 30 28 20 18 11 17 20 32 32 61 50 47
## p589 p590 p591 p592 p593 p594 p595 p596 p597 p598 p599 p600
## 98 30 18 51 33 21 374 27 19 26 31 90
## p601 p602 p603 p604 p605 p606 p607 p608 p609 p610 p611 p612
## 63 14 32 39 33 75 14 25 16 12 12 19
## p613 p614 p615 p616 p617 p618 p619 p620 p621 p622 p623 p624
## 14 35 23 21 12 167 26 21 42 64 36 64
## p625 p626 p627 p628 p629 p630 p631 p632 p633 p634 p635 p636
## 12 38 16 14 18 22 17 12 11 11 40 25
## p637 p638 p639 p640 p641 p642 p643 p644 p645 p646 p647 p648
## 104 11 15 28 13 20 15 35 24 21 21 15
## p649 p650 p651 p652 p653 p654 p655 p656 p657 p658 p659 p660
## 15 12 22 16 22 37 12 25 19 16 14 19
## p661 p662 p663 p664 p665 p666 p667 p668 p669 p670 p671 p672
## 34 11 11 42 26 45 14 26 11 15 18 24
## p673 p674 p675 p676 p677 p678 p679 p680 p681 p682 p683 p684
## 43 21 17 38 28 16 12 20 26 83 21 17
## p685 p686 p687 p688 p689 p690 p691 p692 p693 p694 p695 p696
## 15 20 35 11 13 14 31 26 16 35 14 17
## p697 p698 p699 p700 p701 p702 p703 p704 p705 p706 p707 p708
## 81 60 26 15 40 19 19 13 35 15 84 21
## p709 p710 p711 p712 p713 p714 p715 p716 p717 p718 p719 p720
## 59 74 31 29 19 24 50 29 63 17 37 100
## p721 p722 p723 p724 p725 p726 p727 p728 p729 p730 p731 p732
## 19 32 29 21 39 18 37 21 25 20 35 14
## p733 p734 p735 p736 p737 p738 p739 p740 p741 p742 p743 p744
## 12 23 14 26 52 36 30 26 65 96 23 42
## p745 p746 p747 p748 p749 p750 p751 p752 p753 p754 p755 p756
## 16 36 76 16 28 11 40 33 12 16 29 18
## p757 p758 p759 p760 p761 p762 p763 p764 p765 p766 p767 p768
## 44 20 17 13 46 14 79 22 67 25 38 73
## p769 p770 p771 p772 p773 p774 p775 p776 p777 p778 p779 p780
## 25 30 18 33 12 142 89 47 313 378 100 232
## p781 p782 p783 p784 p785 p786 p787 p788 p789 p790 p791 p792
## 132 446 543 267 162 37 52 186 117 149 56 16
## p793 p794 p795 p796 p797 p798 p799 p800 p801 p802 p803 p804
## 361 90 37 44 36 13 51 265 181 199 61 254
## p805 p806 p807 p808 p809 p810 p811 p812 p813 p814 p815 p816
## 404 154 192 318 57 200 40 103 149 64 204 42
## p817 p818 p819 p820 p821 p822 p823 p824 p825 p826 p827 p828
## 69 47 231 25 257 239 289 87 68 11 54 30
## p829 p830 p831 p832 p833 p834 p835 p836 p837 p838 p839 p840
## 162 116 67 208 105 91 88 49 254 181 69 80
## p841 p842 p843 p844 p845 p846 p847 p848 p849 p850 p851 p852
## 39 68 48 18 98 120 220 140 205 167 27 111
## p853 p854 p855 p856 p857 p858 p859 p860 p861 p862 p863 p864
## 17 187 70 352 132 36 325 400 310 90 45 126
## p865 p866 p867 p868 p869 p870 p871 p872 p873 p874 p875 p876
## 117 300 123 223 27 93 291 129 38 61 43 102
## p877 p878 p879 p880 p881 p882 p883 p884 p885 p886 p887 p888
## 96 128 134 50 38 35 25 167 19 28 258 426
## p889 p890 p891 p892 p893 p894 p895 p896 p897 p898 p899 p900
## 422 23 20 205 12 12 44 15 21 43 14 19
## p901 p902 p903 p904 p905 p906 p907 p908 p909 p910 p911 p912
## 26 18 18 38 26 85 148 33 27 17 31 18
## p913 p914 p915 p916 p917 p918 p919 p920 p921 p922 p923 p924
## 63 86 69 108 28 748 98 269 95 54 35 56
## p925 p926 p927 p928 p929 p930 p931 p932 p933 p934 p935 p936
## 61 66 68 193 20 27 455 116 73 25 58 311
## p937 p938 p939 p940 p941 p942 p943 p944 p945 p946 p947 p948
## 32 53 50 34 50 24 25 54 15 27 40 49
## p949 p950 p951 p952 p953 p954 p955 p956 p957 p958 p959 p960
## 135 16 115 839 66 204 131 81 111 225 30 150
## p961 p962 p963 p964 p965 p966 p967 p968 p969 p970 p971 p972
## 41 47 182 155 124 252 18 86 42 22 38 79
## p973 p974 p975 p976 p977 p978 p979 p980 p981 p982 p983 p984
## 45 83 23 95 58 81 24 770 54 85 87 61
## p985 p986 p987 p988 p989 p990 p991 p992 p993 p994 p995 p996
## 122 95 236 35 93 65 122 207 191 183 77 123
## p997 p998 p999 p1000 p1001 p1002 p1003 p1004 p1005 p1006 p1007 p1008
## 34 241 73 900 532 150 12 113 51 49 13 65
## p1009 p1010 p1011 p1012 p1013 p1014 p1015 p1016 p1017 p1018 p1019 p1020
## 56 36 22 24 17 212 78 31 19 47 103 19
## p1021 p1022 p1023 p1024 p1025 p1026 p1027 p1028 p1029 p1030 p1031 p1032
## 27 17 46 62 45 38 59 47 27 21 39 43
## p1033 p1034 p1035 p1036 p1037 p1038 p1039 p1040 p1041 p1042 p1043 p1044
## 27 72 65 82 27 59 40 28 105 416 53 63
## p1045 p1046 p1047 p1048 p1049 p1050 p1051 p1052 p1053 p1054 p1055 p1056
## 13 146 19 115 40 46 89 31 203 64 77 67
## p1057 p1058 p1059 p1060 p1061 p1062 p1063 p1064 p1065 p1066 p1067 p1068
## 78 44 244 118 152 184 75 31 11 35 44 130
## p1069 p1070 p1071 p1072 p1073 p1074 p1075 p1076 p1077 p1078 p1079 p1080
## 32 138 83 25 11 63 18 143 124 61 15 36
## p1081 p1082 p1083 p1084 p1085 p1086 p1087 p1088 p1089 p1090 p1091 p1092
## 55 48 71 117 110 241 138 205 15 65 15 60
## p1093 p1094 p1095 p1096 p1097 p1098 p1099 p1100 p1101 p1102 p1103 p1104
## 61 17 127 241 161 56 193 372 633 48 690 128
## p1105 p1106 p1107 p1108 p1109 p1110 p1111 p1112 p1113 p1114 p1115 p1116
## 125 14 154 76 256 12 326 361 104 285 68 49
## p1117 p1118 p1119 p1120 p1121 p1122 p1123 p1124 p1125 p1126 p1127 p1128
## 97 128 94 370 685 313 256 140 3923 31 26 63
## p1129 p1130 p1131 p1132 p1133 p1134 p1135 p1136 p1137 p1138 p1139 p1140
## 56 117 80 19 18 39 23 43 18 193 40 68
## p1141 p1142 p1143 p1144 p1145 p1146 p1147 p1148 p1149 p1150 p1151 p1152
## 32 73 97 63 84 125 44 238 107 17 470 102
## p1153 p1154 p1155 p1156 p1157 p1158 p1159 p1160 p1161 p1162 p1163 p1164
## 134 57 146 71 35 69 81 54 75 164 24 28
## p1165 p1166 p1167 p1168 p1169 p1170 p1171 p1172 p1173 p1174 p1175 p1176
## 338 218 64 67 15 11 30 30 12 77 72 13
## p1177 p1178 p1179 p1180 p1181 p1182 p1183 p1184 p1185 p1186 p1187 p1188
## 25 18 12 18 18 59 29 31 67 88 93 113
## p1189 p1190 p1191 p1192 p1193 p1194 p1195 p1196 p1197 p1198 p1199 p1200
## 93 48 674 25 59 274 22 584 13 11 177 17
## p1201 p1202 p1203 p1204 p1205 p1206 p1207 p1208 p1209 p1210 p1211 p1212
## 107 26 11 20 50 77 132 110 57 25 51 619
## p1213 p1214 p1215 p1216 p1217 p1218 p1219 p1220 p1221 p1222 p1223 p1224
## 513 991 1317 140 14 47 461 73 493 35 23 32
## p1225 p1226 p1227 p1228 p1229 p1230 p1231 p1232 p1233 p1234 p1235 p1236
## 380 277 14 288 119 70 72 85 32 127 61 106
## p1237 p1238 p1239 p1240 p1241 p1242 p1243 p1244 p1245 p1246 p1247 p1248
## 65 314 161 343 99 168 81 108 46 65 85 165
## p1249 p1250 p1251 p1252 p1253 p1254 p1255 p1256 p1257 p1258 p1259 p1260
## 15 98 372 179 100 504 44 101 37 570 117 13
## p1261 p1262 p1263 p1264 p1265 p1266 p1267 p1268 p1269 p1270 p1271 p1272
## 30 276 54 48 61 52 56 19 62 17 68 76
## p1273 p1274 p1275 p1276 p1277 p1278 p1279 p1280 p1281 p1282 p1283 p1284
## 15 210 104 11 29 46 27 32 65 27 73 12
## p1285 p1286 p1287 p1288 p1289 p1290 p1291 p1292 p1293 p1294 p1295 p1296
## 20 13 83 17 30 39 77 54 90 75 136 38
## p1297 p1298 p1299 p1300 p1301 p1302 p1303 p1304 p1305 p1306 p1307 p1308
## 98 16 176 53 112 101 29 14 20 15 14 37
## p1309 p1310 p1311 p1312 p1313 p1314 p1315 p1316 p1317 p1318 p1319 p1320
## 37 182 35 17 124 37 68 50 102 15 18 11
## p1321 p1322 p1323 p1324 p1325 p1326 p1327 p1328 p1329 p1330 p1331 p1332
## 62 15 114 156 99 59 525 244 60 107 92 20
## p1333 p1334 p1335 p1336 p1337 p1338 p1339 p1340 p1341 p1342 p1343 p1344
## 14 34 125 43 110 54 12 22 74 60 27 66
## p1345 p1346 p1347 p1348 p1349 p1350 p1351 p1352 p1353 p1354 p1355 p1356
## 24 35 48 13 17 31 44 12 15 53 18 22
## p1357 p1358 p1359 p1360 p1361 p1362 p1363 p1364 p1365 p1366 p1367 p1368
## 46 26 13 11 63 22 67 29 11 11 91 43
## p1369 p1370 p1371 p1372 p1373 p1374 p1375 p1376 p1377 p1378 p1379 p1380
## 92 46 11 41 62 89 88 30 97 25 25 53
## p1381 p1382 p1383 p1384 p1385 p1386 p1387 p1388 p1389 p1390 p1391 p1392
## 198 13 26 55 28 47 16 11 26 189 69 13
## p1393 p1394 p1395 p1396 p1397 p1398 p1399 p1400 p1401 p1402 p1403 p1404
## 247 48 166 211 38 467 44 64 83 12 18 31
## p1405 p1406 p1407 p1408 p1409 p1410 p1411 p1412 p1413 p1414 p1415 p1416
## 67 26 52 141 74 65 94 31 192 33 15 76
## p1417 p1418 p1419 p1420 p1421 p1422 p1423 p1424 p1425 p1426 p1427 p1428
## 20 37 58 317 319 82 30 353 156 129 15 93
## p1429 p1430 p1431 p1432 p1433 p1434 p1435 p1436 p1437 p1438 p1439 p1440
## 99 76 79 15 12 85 146 707 1720 212 149 61
## p1441 p1442 p1443 p1444 p1445 p1446 p1447 p1448 p1449 p1450 p1451 p1452
## 85 133 35 110 23 56 70 28 110 22 30 57
## p1453 p1454 p1455 p1456 p1457 p1458 p1459 p1460 p1461 p1462 p1463 p1464
## 51 172 537 39 574 35 81 48 12 180 95 295
## p1465 p1466 p1467 p1468 p1469 p1470 p1471 p1472 p1473 p1474 p1475 p1476
## 30 33 487 334 17 202 273 16 47 19 43 21
## p1477 p1478 p1479 p1480 p1481 p1482 p1483 p1484 p1485 p1486 p1487 p1488
## 105 60 78 165 94 63 39 38 39 29 198 27
## p1489 p1490 p1491 p1492 p1493 p1494 p1495 p1496 p1497 p1498 p1499 p1500
## 101 54 16 129 37 12 54 46 144 20 20 11
## p1501 p1502 p1503 p1504 p1505 p1506 p1507 p1508 p1509 p1510 p1511 p1512
## 75 17 25 262 216 36 23 13 517 97 624 146
## p1513 p1514 p1515 p1516 p1517 p1518 p1519 p1520 p1521 p1522 p1523 p1524
## 29 18 76 196 367 21 68 56 22 123 52 111
## p1525 p1526 p1527 p1528 p1529 p1530 p1531 p1532 p1533 p1534 p1535 p1536
## 23 174 102 510 303 27 15 19 283 394 157 66
## p1537 p1538 p1539 p1540 p1541 p1542 p1543 p1544 p1545 p1546 p1547 p1548
## 14 34 56 31 51 51 16 33 34 53 46 63
## p1549 p1550 p1551 p1552 p1553 p1554 p1555 p1556 p1557 p1558 p1559 p1560
## 327 40 28 44 45 83 79 30 46 17 54 52
## p1561 p1562 p1563 p1564 p1565 p1566 p1567 p1568 p1569 p1570 p1571 p1572
## 94 77 52 58 32 17 93 11 64 12 47 168
## p1573 p1574 p1575 p1576 p1577 p1578 p1579 p1580 p1581 p1582 p1583 p1584
## 134 38 45 212 32 304 40 241 455 176 971 326
## p1585 p1586 p1587 p1588 p1589 p1590 p1591 p1592 p1593 p1594 p1595 p1596
## 601 189 409 442 61 294 132 175 69 52 97 106
## p1597 p1598 p1599 p1600 p1601 p1602 p1603 p1604 p1605 p1606 p1607 p1608
## 438 664 540 256 738 163 68 274 279 192 265 59
## p1609 p1610 p1611 p1612 p1613 p1614 p1615 p1616 p1617 p1618 p1619 p1620
## 103 26 43 22 309 27 85 210 203 50 154 92
## p1621 p1622 p1623 p1624 p1625 p1626 p1627 p1628 p1629 p1630 p1631 p1632
## 39 72 38 44 15 20 143 158 312 99 33 13
## p1633 p1634 p1635 p1636 p1637 p1638 p1639 p1640 p1641 p1642 p1643 p1644
## 172 177 72 11 138 44 50 15 42 14 13 30
## p1645 p1646 p1647 p1648 p1649 p1650 p1651 p1652 p1653 p1654 p1655 p1656
## 32 23 18 51 19 28 27 24 11 31 15 30
## p1657 p1658 p1659 p1660 p1661 p1662 p1663 p1664 p1665 p1666 p1667 p1668
## 11 45 49 326 41 12 26 457 50 32 369 605
## p1669 p1670 p1671 p1672 p1673 p1674 p1675 p1676 p1677 p1678 p1679 p1680
## 523 109 58 198 161 395 107 724 18 35 433 44
## p1681 p1682 p1683 p1684 p1685 p1686 p1687 p1688 p1689 p1690 p1691 p1692
## 290 369 49 84 21 80 32 16 23 240 82 128
## p1693 p1694 p1695 p1696 p1697 p1698 p1699 p1700 p1701 p1702 p1703 p1704
## 29 70 13 110 86 23 199 26 42 454 33 47
## p1705 p1706 p1707 p1708 p1709 p1710 p1711 p1712 p1713 p1714 p1715 p1716
## 109 1105 94 452 32 791 31 274 179 53 485 106
## p1717 p1718 p1719 p1720 p1721 p1722 p1723 p1724 p1725 p1726 p1727 p1728
## 134 53 273 109 169 21 417 198 120 27 20 644
## p1729 p1730 p1731 p1732 p1733 p1734 p1735 p1736 p1737 p1738 p1739 p1740
## 335 20 94 283 35 547 131 316 20 189 19 110
## p1741 p1742 p1743 p1744 p1745 p1746 p1747 p1748 p1749 p1750 p1751 p1752
## 18 14 16 26 18 22 194 52 188 18 88 23
## p1753 p1754 p1755 p1756 p1757 p1758 p1759 p1760 p1761 p1762 p1763 p1764
## 25 21 18 48 62 16 66 258 140 152 77 41
## p1765 p1766 p1767 p1768 p1769 p1770 p1771 p1772 p1773 p1774 p1775 p1776
## 19 212 31 25 85 28 32 13 67 291 697 24
## p1777 p1778 p1779 p1780 p1781 p1782 p1783 p1784 p1785 p1786 p1787 p1788
## 57 155 194 119 36 27 60 160 32 13 26 38
## p1789 p1790 p1791 p1792 p1793 p1794 p1795 p1796 p1797 p1798 p1799 p1800
## 114 43 13 25 76 19 43 60 33 44 35 73
## p1801 p1802 p1803 p1804 p1805 p1806 p1807 p1808 p1809 p1810 p1811 p1812
## 96 80 103 55 23 23 76 38 28 84 99 16
## p1813 p1814 p1815 p1816 p1817 p1818 p1819 p1820 p1821 p1822 p1823 p1824
## 116 639 78 244 204 467 16 19 189 37 20 18
## p1825 p1826 p1827 p1828 p1829 p1830 p1831 p1832 p1833 p1834 p1835 p1836
## 52 75 135 76 466 230 221 80 855 22 267 216
## p1837 p1838 p1839 p1840 p1841 p1842 p1843 p1844 p1845 p1846 p1847 p1848
## 24 105 73 123 51 233 100 96 30 386 214 43
## p1849 p1850 p1851 p1852 p1853 p1854 p1855 p1856 p1857 p1858 p1859 p1860
## 49 27 113 20 44 196 27 144 123 19 233 134
## p1861 p1862 p1863 p1864 p1865 p1866 p1867 p1868 p1869 p1870 p1871 p1872
## 262 258 98 213 146 175 30 22 317 210 82 145
## p1873 p1874 p1875 p1876 p1877 p1878 p1879 p1880 p1881 p1882 p1883 p1884
## 87 178 88 20 52 145 255 67 22 24 102 26
## p1885 p1886 p1887 p1888 p1889 p1890 p1891 p1892 p1893 p1894 p1895 p1896
## 28 28 50 16 103 34 35 20 378 209 25 30
## p1897 p1898 p1899 p1900 p1901 p1902 p1903 p1904 p1905 p1906 p1907 p1908
## 205 26 36 11 46 19 28 83 27 173 65 22
## p1909 p1910 p1911 p1912 p1913 p1914 p1915 p1916 p1917 p1918 p1919 p1920
## 322 1123 241 335 656 1133 196 78 1702 1326 260 298
## p1921 p1922 p1923 p1924 p1925 p1926 p1927 p1928 p1929 p1930 p1931 p1932
## 489 459 207 57 173 98 221 164 33 75 93 35
## p1933 p1934 p1935 p1936 p1937 p1938 p1939 p1940 p1941 p1942 p1943 p1944
## 96 14 46 435 265 1325 201 601 601 32 22 125
## p1945 p1946 p1947 p1948 p1949 p1950 p1951 p1952 p1953 p1954 p1955 p1956
## 64 76 15 203 235 115 30 21 15 23 170 404
## p1957 p1958 p1959 p1960 p1961 p1962 p1963 p1964 p1965 p1966 p1967 p1968
## 56 159 16 45 36 33 292 38 48 21 175 32
## p1969 p1970 p1971 p1972 p1973 p1974 p1975 p1976 p1977 p1978 p1979 p1980
## 103 133 21 735 89 25 47 30 36 68 111 715
## p1981 p1982 p1983 p1984 p1985 p1986 p1987 p1988 p1989 p1990 p1991 p1992
## 60 112 22 74 34 51 37 69 132 370 74 202
## p1993 p1994 p1995 p1996 p1997 p1998 p1999 p2000 p2001 p2002 p2003 p2004
## 150 903 169 269 168 74 68 13 314 165 17 47
## p2005 p2006 p2007 p2008 p2009 p2010 p2011 p2012 p2013 p2014 p2015 p2016
## 19 195 261 56 75 53 56 215 27 37 37 117
## p2017 p2018 p2019 p2020 p2021 p2022 p2023 p2024 p2025 p2026 p2027 p2028
## 52 46 45 63 70 84 33 267 22 68 187 49
## p2029 p2030 p2031 p2032 p2033 p2034 p2035 p2036 p2037 p2038 p2039 p2040
## 171 118 109 256 166 68 70 289 348 95 292 123
## p2041 p2042 p2043 p2044 p2045 p2046 p2047 p2048 p2049 p2050 p2051 p2052
## 48 364 163 85 29 206 339 101 253 65 44 38
## p2053 p2054 p2055 p2056 p2057 p2058 p2059 p2060 p2061 p2062 p2063 p2064
## 49 11 44 27 240 12 155 13 154 29 78 220
## p2065 p2066 p2067 p2068 p2069 p2070 p2071 p2072 p2073 p2074 p2075 p2076
## 30 315 63 679 13 61 191 244 18 64 76 63
## p2077 p2078 p2079 p2080 p2081 p2082 p2083 p2084 p2085 p2086 p2087 p2088
## 185 66 189 114 93 99 54 262 109 68 406 256
## p2089 p2090 p2091 p2092 p2093 p2094 p2095 p2096 p2097 p2098 p2099 p2100
## 589 384 258 141 375 671 398 119 213 308 439 646
## p2101 p2102 p2103 p2104 p2105 p2106 p2107 p2108 p2109 p2110 p2111 p2112
## 408 240 330 32 23 226 42 618 87 170 767 101
## p2113 p2114 p2115 p2116 p2117 p2118 p2119 p2120 p2121 p2122 p2123 p2124
## 475 26 25 188 11 920 246 40 67 254 65 676
## p2125 p2126 p2127 p2128 p2129 p2130 p2131 p2132 p2133 p2134 p2135 p2136
## 869 139 104 44 25 36 19 42 20 73 53 37
## p2137 p2138 p2139 p2140 p2141 p2142 p2143 p2144 p2145 p2146 p2147 p2148
## 47 61 75 27 419 167 37 164 199 12 36 92
## p2149 p2150 p2151 p2152 p2153 p2154 p2155 p2156 p2157 p2158 p2159 p2160
## 33 93 34 37 56 102 13 289 81 174 29 24
## p2161 p2162 p2163 p2164 p2165 p2166 p2167 p2168 p2169 p2170 p2171 p2172
## 251 102 268 142 55 385 48 66 119 1074 295 15
## p2173 p2174 p2175 p2176 p2177 p2178 p2179 p2180 p2181 p2182 p2183 p2184
## 45 25 38 24 34 70 15 15 78 42 20 17
## p2185 p2186 p2187 p2188 p2189 p2190 p2191 p2192 p2193 p2194 p2195 p2196
## 115 14 49 147 84 21 45 47 186 36 35 33
## p2197 p2198 p2199 p2200 p2201 p2202 p2203 p2204 p2205 p2206 p2207 p2208
## 12 41 43 26 40 19 53 45 11 40 102 25
## p2209 p2210 p2211 p2212 p2213 p2214 p2215 p2216 p2217 p2218 p2219 p2220
## 44 16 610 68 68 49 29 34 51 125 25 30
## p2221 p2222 p2223 p2224 p2225 p2226 p2227 p2228 p2229 p2230 p2231 p2232
## 38 64 67 35 37 27 16 34 191 86 25 44
## p2233 p2234 p2235 p2236 p2237 p2238 p2239 p2240 p2241 p2242 p2243 p2244
## 76 19 38 63 15 11 17 67 16 19 97 184
## p2245 p2246 p2247 p2248 p2249 p2250 p2251 p2252 p2253 p2254 p2255 p2256
## 75 28 43 107 46 62 58 24 54 26 71 39
## p2257 p2258 p2259 p2260 p2261 p2262 p2263 p2264 p2265 p2266 p2267 p2268
## 117 204 253 166 223 135 137 16 133 116 68 34
## p2269 p2270 p2271 p2272 p2273 p2274 p2275 p2276 p2277 p2278 p2279 p2280
## 137 74 156 960 1402 20 21 362 158 639 26 55
## p2281 p2282 p2283 p2284 p2285 p2286 p2287 p2288 p2289 p2290 p2291 p2292
## 80 170 181 187 130 35 54 71 71 47 11 136
## p2293 p2294 p2295 p2296 p2297 p2298 p2299 p2300 p2301 p2302 p2303 p2304
## 28 69 23 22 17 128 189 19 64 43 49 52
## p2305 p2306 p2307 p2308 p2309 p2310 p2311 p2312 p2313 p2314 p2315 p2316
## 23 20 330 101 111 78 105 148 12 75 169 166
## p2317 p2318 p2319 p2320 p2321 p2322 p2323 p2324 p2325 p2326 p2327 p2328
## 49 32 160 81 60 32 19 31 72 41 105 12
## p2329 p2330 p2331 p2332 p2333 p2334 p2335 p2336 p2337 p2338 p2339 p2340
## 17 38 54 18 11 37 118 20 16 35 63 27
## p2341 p2342 p2343 p2344 p2345 p2346 p2347 p2348 p2349 p2350 p2351 p2352
## 64 14 29 41 23 76 70 105 23 17 87 13
## p2353 p2354 p2355 p2356 p2357 p2358 p2359 p2360 p2361 p2362 p2363 p2364
## 34 43 99 12 27 12 590 449 207 14 42 159
## p2365 p2366 p2367 p2368 p2369 p2370 p2371 p2372 p2373 p2374 p2375 p2376
## 239 306 74 57 52 157 163 238 61 91 220 52
## p2377 p2378 p2379 p2380 p2381 p2382 p2383 p2384 p2385 p2386 p2387 p2388
## 20 98 25 27 23 22 16 24 39 33 55 35
## p2389 p2390 p2391 p2392 p2393 p2394 p2395 p2396 p2397 p2398 p2399 p2400
## 134 18 37 170 30 36 45 272 143 365 14 15
## p2401 p2402 p2403 p2404 p2405 p2406 p2407 p2408 p2409 p2410 p2411 p2412
## 68 17 46 29 37 69 45 108 23 47 105 259
## p2413 p2414 p2415 p2416 p2417 p2418 p2419 p2420 p2421 p2422 p2423 p2424
## 89 168 106 150 77 262 64 131 381 56 28 52
## p2425 p2426 p2427 p2428 p2429 p2430 p2431 p2432 p2433 p2434 p2435 p2436
## 74 36 87 16 34 59 32 17 47 18 116 15
## p2437 p2438 p2439 p2440 p2441 p2442 p2443 p2444 p2445 p2446 p2447 p2448
## 41 34 61 120 29 38 16 54 46 22 47 17
## p2449 p2450 p2451 p2452 p2453 p2454 p2455 p2456 p2457 p2458 p2459 p2460
## 51 32 122 158 30 17 191 611 32 177 160 22
## p2461 p2462 p2463 p2464 p2465 p2466 p2467 p2468 p2469 p2470 p2471 p2472
## 54 113 141 284 53 16 59 24 14 12 11 27
## p2473 p2474 p2475 p2476 p2477 p2478 p2479 p2480 p2481 p2482 p2483 p2484
## 106 14 30 51 112 28 32 14 35 76 28 19
## p2485 p2486 p2487 p2488 p2489 p2490 p2491 p2492 p2493 p2494 p2495 p2496
## 20 33 13 15 58 26 44 44 39 35 70 85
## p2497 p2498 p2499 p2500 p2501 p2502 p2503 p2504 p2505 p2506 p2507 p2508
## 48 42 103 64 22 44 36 23 40 36 65 44
## p2509 p2510 p2511 p2512 p2513 p2514 p2515 p2516 p2517 p2518 p2519 p2520
## 15 24 37 135 41 203 19 109 42 19 13 14
## p2521 p2522 p2523 p2524 p2525 p2526 p2527 p2528 p2529 p2530 p2531 p2532
## 12 20 27 76 151 99 31 23 49 82 50 58
## p2533 p2534 p2535 p2536 p2537 p2538 p2539 p2540 p2541 p2542 p2543 p2544
## 42 54 19 13 27 12 11 23 50 103 11 47
## p2545 p2546 p2547 p2548 p2549 p2550 p2551 p2552 p2553 p2554 p2555 p2556
## 20 114 26 13 60 18 18 40 41 14 57 40
## p2557 p2558 p2559 p2560 p2561 p2562 p2563 p2564 p2565 p2566 p2567 p2568
## 16 30 105 220 74 191 29 13 16 13 26 31
## p2569 p2570 p2571 p2572 p2573 p2574 p2575 p2576 p2577 p2578 p2579 p2580
## 22 23 66 32 29 49 50 17 29 21 11 59
## p2581 p2582 p2583 p2584 p2585 p2586 p2587 p2588 p2589 p2590 p2591 p2592
## 36 12 13 28 12 14 19 11 50 23 17 28
## p2593 p2594 p2595 p2596 p2597 p2598 p2599 p2600 p2601 p2602 p2603 p2604
## 23 17 48 32 62 31 77 27 17 17 13 133
## p2605 p2606 p2607 p2608 p2609 p2610 p2611 p2612 p2613 p2614 p2615 p2616
## 12 30 26 88 122 107 21 40 19 28 19 24
## p2617 p2618 p2619 p2620 p2621 p2622 p2623 p2624 p2625 p2626 p2627 p2628
## 19 50 21 140 94 15 27 41 40 79 18 155
## p2629 p2630 p2631 p2632 p2633 p2634 p2635 p2636 p2637 p2638 p2639 p2640
## 144 75 38 169 55 13 81 17 57 71 27 12
## p2641 p2642 p2643 p2644 p2645 p2646 p2647 p2648 p2649 p2650 p2651 p2652
## 34 40 15 18 23 80 70 67 27 11 29 50
## p2653 p2654 p2655 p2656 p2657 p2658 p2659 p2660 p2661 p2662 p2663 p2664
## 55 34 38 67 11 64 69 94 99 115 84 44
## p2665 p2666 p2667 p2668 p2669 p2670 p2671 p2672 p2673 p2674 p2675 p2676
## 40 15 20 29 25 47 281 24 96 27 48 170
## p2677 p2678 p2679 p2680 p2681 p2682 p2683 p2684 p2685 p2686 p2687 p2688
## 19 45 28 15 43 145 37 86 41 30 75 409
## p2689 p2690 p2691 p2692 p2693 p2694 p2695 p2696 p2697 p2698 p2699 p2700
## 28 71 62 168 102 218 212 214 61 135 292 19
## p2701 p2702 p2703 p2704 p2705 p2706 p2707 p2708 p2709 p2710 p2711 p2712
## 49 54 138 44 77 91 28 166 289 67 90 63
## p2713 p2714 p2715 p2716 p2717 p2718 p2719 p2720 p2721 p2722 p2723 p2724
## 26 64 186 68 24 65 47 87 260 44 97 37
## p2725 p2726 p2727 p2728 p2729 p2730 p2731 p2732 p2733 p2734 p2735 p2736
## 43 166 155 27 28 15 191 67 41 77 33 105
## p2737 p2738 p2739 p2740 p2741 p2742 p2743 p2744 p2745 p2746 p2747 p2748
## 215 28 15 38 78 1519 11 46 12 15 14 51
## p2749 p2750 p2751 p2752 p2753 p2754 p2755 p2756 p2757 p2758 p2759 p2760
## 72 262 37 36 90 125 30 49 91 112 84 136
## p2761 p2762 p2763 p2764 p2765 p2766 p2767 p2768 p2769 p2770 p2771 p2772
## 104 23 26 24 11 141 137 74 20 11 75 366
## p2773 p2774 p2775 p2776 p2777 p2778 p2779 p2780 p2781 p2782 p2783 p2784
## 83 31 35 13 27 53 53 22 144 36 38 112
## p2785 p2786 p2787 p2788 p2789 p2790 p2791 p2792 p2793 p2794 p2795 p2796
## 30 23 116 62 46 212 18 246 60 28 48 93
## p2797 p2798 p2799 p2800 p2801 p2802 p2803 p2804 p2805 p2806 p2807 p2808
## 32 32 45 35 79 56 32 22 46 47 16 45
## p2809 p2810 p2811 p2812 p2813 p2814 p2815 p2816 p2817 p2818 p2819 p2820
## 41 16 54 29 133 112 92 42 20 51 156 20
## p2821 p2822 p2823 p2824 p2825 p2826 p2827 p2828 p2829 p2830 p2831 p2832
## 167 22 495 17 23 94 69 13 53 48 116 19
## p2833 p2834 p2835 p2836 p2837 p2838 p2839 p2840 p2841 p2842 p2843 p2844
## 35 34 40 61 22 146 72 78 60 77 211 66
## p2845 p2846 p2847 p2848 p2849 p2850 p2851 p2852 p2853 p2854 p2855 p2856
## 126 157 14 21 49 94 17 136 68 490 85 294
## p2857 p2858 p2859 p2860 p2861 p2862 p2863 p2864 p2865 p2866 p2867 p2868
## 504 219 207 93 250 186 316 263 383 633 15 235
## p2869 p2870 p2871 p2872 p2873 p2874 p2875 p2876 p2877 p2878 p2879 p2880
## 52 213 14 18 32 50 28 146 149 86 60 56
## p2881 p2882 p2883 p2884 p2885 p2886 p2887 p2888 p2889 p2890 p2891 p2892
## 55 117 23 209 114 19 23 63 44 120 120 51
## p2893 p2894 p2895 p2896 p2897 p2898 p2899 p2900 p2901 p2902 p2903 p2904
## 14 90 71 40 17 23 168 66 115 22 51 361
## p2905 p2906 p2907 p2908 p2909 p2910 p2911 p2912 p2913 p2914 p2915 p2916
## 78 13 89 39 30 30 90 66 93 33 241 47
## p2917 p2918 p2919 p2920 p2921 p2922 p2923 p2924 p2925 p2926 p2927 p2928
## 119 68 38 73 26 17 78 56 57 196 45 135
## p2929 p2930 p2931 p2932 p2933 p2934 p2935 p2936 p2937 p2938 p2939 p2940
## 110 180 152 147 243 55 169 303 507 234 59 98
## p2941 p2942 p2943 p2944 p2945 p2946 p2947 p2948 p2949 p2950 p2951 p2952
## 16 36 72 50 116 164 83 29 77 41 253 86
## p2953 p2954 p2955 p2956 p2957 p2958 p2959 p2960 p2961 p2962 p2963 p2964
## 84 74 676 197 1381 33 23 112 125 72 71 16
## p2965 p2966 p2967 p2968 p2969 p2970 p2971 p2972 p2973 p2974 p2975 p2976
## 68 29 12 49 96 116 13 20 11 36 19 13
## p2977 p2978 p2979 p2980 p2981 p2982 p2983 p2984 p2985 p2986 p2987 p2988
## 21 12 15 288 415 34 28 44 12 111 41 111
## p2989 p2990 p2991 p2992 p2993 p2994 p2995 p2996 p2997 p2998 p2999 p3000
## 61 246 32 11 13 96 60 11 39 42 51 25
## p3001 p3002 p3003 p3004 p3005 p3006 p3007 p3008 p3009 p3010 p3011 p3012
## 18 13 46 16 237 17 566 40 11 78 53 52
## p3013 p3014 p3015 p3016 p3017 p3018 p3019 p3020 p3021 p3022 p3023 p3024
## 105 35 37 77 97 81 291 177 38 19 33 32
## p3025 p3026 p3027 p3028 p3029 p3030 p3031 p3032 p3033 p3034 p3035 p3036
## 34 138 530 49 26 37 74 83 245 26 31 21
## p3037 p3038 p3039 p3040 p3041 p3042 p3043 p3044 p3045 p3046 p3047 p3048
## 116 20 18 70 48 69 38 45 82 61 12 46
## p3049 p3050 p3051 p3052 p3053 p3054 p3055 p3056 p3057 p3058 p3059 p3060
## 11 39 25 14 58 57 76 35 329 69 33 62
## p3061 p3062 p3063 p3064 p3065 p3066 p3067 p3068 p3069 p3070 p3071 p3072
## 22 17 83 59 106 20 138 11 233 24 11 253
## p3073 p3074 p3075 p3076 p3077 p3078 p3079 p3080 p3081 p3082 p3083 p3084
## 30 295 202 326 103 25 136 41 371 128 39 13
## p3085 p3086 p3087 p3088 p3089 p3090 p3091 p3092 p3093 p3094 p3095 p3096
## 104 178 67 742 67 19 14 46 95 17 49 194
## p3097 p3098 p3099 p3100 p3101 p3102 p3103 p3104 p3105 p3106 p3107 p3108
## 14 78 60 47 48 85 472 13 30 83 79 46
## p3109 p3110 p3111 p3112 p3113 p3114 p3115 p3116 p3117 p3118 p3119 p3120
## 11 45 17 28 45 43 63 57 38 28 29 35
## p3121 p3122 p3123 p3124 p3125 p3126 p3127 p3128 p3129 p3130 p3131 p3132
## 30 25 11 57 14 42 47 11 20 105 376 182
## p3133 p3134 p3135 p3136 p3137 p3138 p3139 p3140 p3141 p3142 p3143 p3144
## 220 17 148 234 14 12 36 35 107 25 131 118
## p3145 p3146 p3147 p3148 p3149 p3150 p3151 p3152 p3153 p3154 p3155 p3156
## 14 11 13 25 16 14 18 107 124 24 32 338
## p3157 p3158 p3159 p3160 p3161 p3162 p3163 p3164 p3165 p3166 p3167 p3168
## 24 19 12 90 16 14 31 201 26 22 55 90
## p3169 p3170 p3171 p3172 p3173 p3174 p3175 p3176 p3177 p3178 p3179 p3180
## 52 56 48 56 28 77 19 47 71 105 383 27
## p3181 p3182 p3183 p3184 p3185 p3186 p3187 p3188 p3189 p3190 p3191 p3192
## 38 54 19 42 21 27 35 38 47 54 121 66
## p3193 p3194 p3195 p3196 p3197 p3198 p3199 p3200 p3201 p3202 p3203 p3204
## 116 285 13 93 996 48 653 115 145 655 14 94
## p3205 p3206 p3207 p3208 p3209 p3210 p3211 p3212 p3213 p3214 p3215 p3216
## 27 20 28 21 133 39 41 61 43 69 45 134
## p3217 p3218 p3219 p3220 p3221 p3222 p3223 p3224 p3225 p3226 p3227 p3228
## 20 55 825 11 16 50 376 122 719 13 227 30
## p3229 p3230 p3231 p3232 p3233 p3234 p3235 p3236 p3237 p3238 p3239 p3240
## 501 15 11 39 13 16 22 16 27 23 14 22
## p3241 p3242 p3243 p3244 p3245 p3246 p3247 p3248 p3249 p3250 p3251 p3252
## 21 14 17 39 11 16 23 41 40 17 20 28
## p3253 p3254 p3255 p3256 p3257 p3258 p3259 p3260 p3261 p3262 p3263 p3264
## 98 14 68 89 28 41 15 13 100 108 18 158
## p3265 p3266 p3267 p3268 p3269 p3270 p3271 p3272 p3273 p3274 p3275 p3276
## 33 46 41 12 201 90 614 616 473 23 85 74
## p3277 p3278 p3279 p3280 p3281 p3282 p3283 p3284 p3285 p3286 p3287 p3288
## 37 36 60 144 54 31 13 62 47 74 12 19
## p3289 p3290 p3291 p3292 p3293 p3294 p3295 p3296 p3297 p3298 p3299 p3300
## 15 17 29 35 39 23 162 52 248 25 59 77
## p3301 p3302 p3303 p3304 p3305 p3306 p3307 p3308 p3309 p3310 p3311 p3312
## 29 18 87 101 69 39 21 302 471 13 35 24
## p3313 p3314 p3315 p3316 p3317 p3318 p3319 p3320 p3321 p3322 p3323 p3324
## 14 15 28 43 53 175 64 39 14 39 14 16
## p3325 p3326 p3327 p3328 p3329 p3330 p3331 p3332 p3333 p3334 p3335 p3336
## 73 101 458 43 19 24 21 26 32 157 27 363
## p3337 p3338 p3339 p3340 p3341 p3342 p3343 p3344 p3345 p3346 p3347 p3348
## 20 977 21 20 11 290 88 53 88 14 69 117
## p3349 p3350 p3351 p3352 p3353 p3354 p3355 p3356 p3357 p3358 p3359 p3360
## 84 39 120 134 111 169 265 173 31 76 190 67
## p3361 p3362 p3363 p3364 p3365 p3366 p3367 p3368 p3369 p3370 p3371 p3372
## 60 90 46 69 109 214 59 38 515 635 170 400
## p3373 p3374 p3375 p3376 p3377 p3378 p3379 p3380 p3381 p3382 p3383 p3384
## 140 26 192 150 143 84 112 15 23 43 114 109
## p3385 p3386 p3387 p3388 p3389 p3390 p3391 p3392 p3393 p3394 p3395 p3396
## 27 33 202 27 105 141 58 44 84 207 50 206
## p3397 p3398 p3399 p3400 p3401 p3402 p3403 p3404 p3405 p3406 p3407 p3408
## 16 277 193 286 24 14 13 29 120 297 56 41
## p3409 p3410 p3411 p3412 p3413 p3414 p3415 p3416 p3417 p3418 p3419 p3420
## 31 119 35 147 70 14 119 34 175 153 52 20
## p3421 p3422 p3423 p3424 p3425 p3426 p3427 p3428 p3429 p3430 p3431 p3432
## 17 21 21 32 17 16 41 255 71 37 66 128
## p3433 p3434 p3435 p3436 p3437 p3438 p3439 p3440 p3441 p3442 p3443 p3444
## 128 38 135 102 53 34 34 112 80 73 44 27
## p3445 p3446 p3447 p3448 p3449 p3450 p3451 p3452 p3453 p3454 p3455 p3456
## 19 26 32 46 15 98 66 95 28 26 13 29
## p3457 p3458 p3459 p3460 p3461 p3462 p3463 p3464 p3465 p3466 p3467 p3468
## 65 19 33 38 15 50 20 86 15 19 121 198
## p3469 p3470 p3471 p3472 p3473 p3474 p3475 p3476 p3477 p3478 p3479 p3480
## 68 103 119 26 53 31 53 28 143 21 104 86
## p3481 p3482 p3483 p3484 p3485 p3486 p3487 p3488 p3489 p3490 p3491 p3492
## 57 181 153 412 101 111 181 119 30 19 67 53
## p3493 p3494 p3495 p3496 p3497 p3498 p3499 p3500 p3501 p3502 p3503 p3504
## 34 18 52 30 28 11 254 44 11 12 31 86
## p3505 p3506 p3507 p3508 p3509 p3510 p3511 p3512 p3513 p3514 p3515 p3516
## 32 14 101 32 112 32 94 26 106 31 75 65
## p3517 p3518 p3519 p3520 p3521 p3522 p3523 p3524 p3525 p3526 p3527 p3528
## 29 149 30 155 290 23 20 17 35 57 64 33
## p3529 p3530 p3531 p3532 p3533 p3534 p3535 p3536 p3537 p3538 p3539 p3540
## 15 73 90 67 61 71 112 106 38 19 111 15
## p3541 p3542 p3543 p3544 p3545 p3546 p3547 p3548 p3549 p3550 p3551 p3552
## 42 15 139 63 42 38 75 71 18 38 30 93
## p3553 p3554 p3555 p3556 p3557 p3558 p3559 p3560 p3561 p3562 p3563 p3564
## 27 21 109 25 41 25 50 18 17 19 78 20
## p3565 p3566 p3567 p3568 p3569 p3570 p3571 p3572 p3573 p3574 p3575 p3576
## 13 29 46 27 24 66 23 30 33 18 62 29
## p3577 p3578 p3579 p3580 p3581 p3582 p3583 p3584 p3585 p3586 p3587 p3588
## 17 55 19 24 14 60 60 82 18 15 17 14
## p3589 p3590 p3591 p3592 p3593 p3594 p3595 p3596 p3597 p3598 p3599 p3600
## 25 140 51 51 25 49 46 118 20 23 150 236
## p3601 p3602 p3603 p3604 p3605 p3606 p3607 p3608 p3609 p3610 p3611 p3612
## 27 44 22 58 39 80 21 54 134 38 101 50
## p3613 p3614 p3615 p3616 p3617 p3618 p3619 p3620 p3621 p3622 p3623 p3624
## 57 115 96 36 120 123 83 36 39 108 143 66
## p3625 p3626 p3627 p3628 p3629 p3630 p3631 p3632 p3633 p3634 p3635 p3636
## 45 13 189 76 36 153 170 17 14 27 41 47
## p3637 p3638 p3639 p3640 p3641 p3642 p3643 p3644 p3645 p3646 p3647 p3648
## 160 115 73 95 63 45 26 70 157 211 144 133
## p3649 p3650 p3651 p3652 p3653 p3654 p3655 p3656 p3657 p3658 p3659 p3660
## 651 40 101 72 29 221 878 19 574 41 47 181
## p3661 p3662 p3663 p3664 p3665 p3666 p3667 p3668 p3669 p3670 p3671 p3672
## 106 36 28 112 504 23 43 102 137 142 14 81
## p3673 p3674 p3675 p3676 p3677 p3678 p3679 p3680 p3681 p3682 p3683 p3684
## 63 16 108 61 51 12 109 15 48 35 21 45
## p3685 p3686 p3687 p3688 p3689 p3690 p3691 p3692 p3693 p3694 p3695 p3696
## 29 48 16 22 13 34 16 66 80 30 24 16
## p3697 p3698 p3699 p3700 p3701 p3702 p3703 p3704 p3705 p3706 p3707 p3708
## 165 53 31 46 31 33 179 75 70 52 16 182
## p3709 p3710 p3711 p3712 p3713 p3714 p3715 p3716 p3717 p3718 p3719 p3720
## 196 57 230 47 254 276 59 277 75 18 135 50
## p3721 p3722 p3723 p3724 p3725 p3726 p3727 p3728 p3729 p3730 p3731 p3732
## 13 57 22 22 366 70 33 12 13 37 27 17
## p3733 p3734 p3735 p3736 p3737 p3738 p3739 p3740 p3741 p3742 p3743 p3744
## 35 39 122 40 45 135 53 16 20 25 39 17
## p3745 p3746 p3747 p3748 p3749 p3750 p3751 p3752 p3753 p3754 p3755 p3756
## 32 23 23 295 90 45 650 202 111 28 294 14
## p3757 p3758 p3759 p3760 p3761 p3762 p3763 p3764 p3765 p3766 p3767 p3768
## 96 98 36 174 179 32 299 11 217 25 198 11
## p3769 p3770 p3771 p3772 p3773 p3774 p3775 p3776 p3777 p3778 p3779 p3780
## 243 13 137 137 19 1824 16 32 23 20 21 92
## p3781 p3782 p3783 p3784 p3785 p3786 p3787 p3788 p3789 p3790 p3791 p3792
## 20 12 208 30 46 20 22 23 118 32 24 266
## p3793 p3794 p3795 p3796 p3797 p3798 p3799 p3800 p3801 p3802 p3803 p3804
## 12 15 31 49 24 25 55 81 137 116 94 92
## p3805 p3806 p3807 p3808 p3809 p3810 p3811 p3812 p3813 p3814 p3815 p3816
## 39 19 28 18 41 14 14 110 19 27 32 50
## p3817 p3818 p3819 p3820 p3821 p3822 p3823 p3824 p3825 p3826 p3827 p3828
## 105 12 47 16 33 88 12 25 93 11 16 54
## p3829 p3830 p3831 p3832 p3833 p3834 p3835 p3836 p3837 p3838 p3839 p3840
## 19 62 12 129 16 16 57 33 31 88 44 74
## p3841 p3842 p3843 p3844 p3845 p3846 p3847 p3848 p3849 p3850 p3851 p3852
## 83 302 159 126 38 50 50 86 23 183 30 89
## p3853 p3854 p3855 p3856 p3857 p3858 p3859 p3860 p3861 p3862 p3863 p3864
## 33 12 28 524 19 147 59 63 75 46 31 36
## p3865 p3866 p3867 p3868 p3869 p3870 p3871 p3872 p3873 p3874 p3875 p3876
## 35 28 43 38 71 50 103 17 155 76 141 105
## p3877 p3878 p3879 p3880 p3881 p3882 p3883 p3884 p3885 p3886 p3887 p3888
## 63 11 69 26 21 49 75 98 146 55 124 101
## p3889 p3890 p3891 p3892 p3893 p3894 p3895 p3896 p3897 p3898 p3899 p3900
## 70 30 50 56 93 23 93 124 73 29 42 77
## p3901 p3902 p3903 p3904 p3905 p3906 p3907 p3908 p3909 p3910 p3911 p3912
## 22 89 80 22 112 14 109 91 27 42 60 83
## p3913 p3914 p3915 p3916 p3917 p3918 p3919 p3920 p3921 p3922 p3923 p3924
## 49 14 23 30 44 205 39 154 33 331 81 122
## p3925 p3926 p3927 p3928 p3929 p3930 p3931 p3932 p3933 p3934 p3935 p3936
## 282 63 61 368 169 124 57 17 172 31 34 28
## p3937 p3938 p3939 p3940 p3941 p3942 p3943 p3944 p3945 p3946 p3947 p3948
## 41 11 21 34 48 49 103 55 35 76 33 11
## p3949 p3950 p3951 p3952 p3953 p3954 p3955 p3956 p3957 p3958 p3959 p3960
## 37 90 27 18 31 23 14 59 34 45 47 23
## p3961 p3962 p3963 p3964 p3965 p3966 p3967 p3968 p3969 p3970 p3971 p3972
## 24 18 16 58 28 35 151 17 94 91 40 36
## p3973 p3974 p3975 p3976 p3977 p3978 p3979 p3980 p3981 p3982 p3983 p3984
## 60 214 147 75 109 109 22 279 31 127 16 46
## p3985 p3986 p3987 p3988 p3989 p3990 p3991 p3992 p3993 p3994 p3995 p3996
## 51 39 373 51 41 56 29 190 99 221 445 42
## p3997 p3998 p3999 p4000 p4001 p4002 p4003 p4004 p4005 p4006 p4007 p4008
## 841 247 153 188 131 114 32 16 11 45 38 233
## p4009 p4010 p4011 p4012 p4013 p4014 p4015 p4016 p4017 p4018 p4019 p4020
## 35 55 71 157 230 16 80 37 57 143 16 138
## p4021 p4022 p4023 p4024 p4025 p4026 p4027 p4028 p4029 p4030 p4031 p4032
## 39 18 38 34 37 68 156 29 72 51 19 36
## p4033 p4034 p4035 p4036 p4037 p4038 p4039 p4040 p4041 p4042 p4043 p4044
## 62 127 41 77 43 49 80 40 50 27 141 55
## p4045 p4046 p4047 p4048 p4049 p4050 p4051 p4052 p4053 p4054 p4055 p4056
## 57 28 86 54 22 191 88 27 70 106 109 57
## p4057 p4058 p4059 p4060 p4061 p4062 p4063 p4064 p4065 p4066 p4067 p4068
## 79 171 30 82 12 120 128 11 48 87 48 99
## p4069 p4070 p4071 p4072 p4073 p4074 p4075 p4076 p4077 p4078 p4079 p4080
## 243 68 34 32 164 46 53 16 69 78 94 81
## p4081 p4082 p4083 p4084 p4085 p4086 p4087 p4088 p4089 p4090 p4091 p4092
## 50 145 37 205 141 170 65 533 1598 413 816 41
## p4093 p4094 p4095 p4096 p4097 p4098 p4099 p4100 p4101 p4102 p4103 p4104
## 528 1148 262 322 35 103 76 156 14 54 75 49
## p4105 p4106 p4107 p4108 p4109 p4110 p4111 p4112 p4113 p4114 p4115 p4116
## 14 45 102 32 60 108 113 22 92 38 16 27
## p4117 p4118 p4119 p4120 p4121 p4122 p4123 p4124 p4125 p4126 p4127 p4128
## 99 96 39 130 22 33 48 72 76 69 20 11
## p4129 p4130 p4131 p4132 p4133 p4134 p4135 p4136 p4137 p4138 p4139 p4140
## 20 15 12 13 23 13 27 14 11 19 11 14
## p4141 p4142 p4143 p4144 p4145 p4146 p4147 p4148 p4149 p4150 p4151 p4152
## 11 14 45 29 17 24 19 15 16 33 13 17
## p4153 p4154 p4155 p4156 p4157 p4158 p4159 p4160 p4161 p4162 p4163 p4164
## 15 11 13 12 16 14 11 11 14 16 13 11
## p4165 p4166 p4167 p4168 p4169 p4170 p4171 p4172 p4173 p4174 p4175 p4176
## 18 19 15 11 29 14 15 12 35 21 14 20
## p4177 p4178 p4179 p4180 p4181 p4182 p4183 p4184 p4185 p4186 p4187 p4188
## 12 25 18 18 13 15 13 14 12 19 12 13
## p4189 p4190 p4191 p4192 p4193 p4194 p4195 p4196 p4197 p4198 p4199 p4200
## 11 34 12 19 13 27 30 14 33 11 16 12
## p4201 p4202 p4203 p4204 p4205 p4206 p4207 p4208 p4209 p4210 p4211 p4212
## 11 28 21 20 24 13 11 20 33 12 27 22
## p4213 p4214 p4215 p4216 p4217 p4218 p4219 p4220 p4221 p4222 p4223 p4224
## 13 18 14 11 13 27 14 17 17 33 22 19
## p4225 p4226 p4227 p4228 p4229 p4230 p4231 p4232 p4233 p4234 p4235 p4236
## 14 14 16 14 20 11 13 23 11 37 14 21
## p4237 p4238 p4239 p4240 p4241 p4242 p4243 p4244 p4245 p4246 p4247 p4248
## 19 12 20 18 15 36 19 14 28 16 25 44
## p4249 p4250 p4251 p4252 p4253 p4254 p4255 p4256 p4257 p4258 p4259 p4260
## 19 16 17 14 18 12 44 22 13 22 17 12
## p4261 p4262 p4263 p4264 p4265 p4266 p4267 p4268 p4269 p4270 p4271 p4272
## 12 13 13 31 32 28 17 12 18 30 18 30
## p4273 p4274 p4275 p4276 p4277 p4278 p4279 p4280 p4281 p4282 p4283 p4284
## 17 12 25 15 32 13 32 106 13 14 29 14
## p4285 p4286 p4287 p4288 p4289 p4290 p4291 p4292 p4293 p4294 p4295 p4296
## 28 23 19 13 21 36 14 15 17 23 24 11
## p4297 p4298 p4299 p4300 p4301 p4302 p4303 p4304 p4305 p4306 p4307 p4308
## 19 15 24 17 11 13 25 27 39 18 17 29
## p4309 p4310 p4311 p4312 p4313 p4314 p4315 p4316 p4317 p4318 p4319 p4320
## 11 29 11 15 11 11 13 21 31 12 12 33
## p4321 p4322 p4323 p4324 p4325 p4326 p4327 p4328 p4329 p4330 p4331 p4332
## 16 15 15 15 47 12 11 36 20 13 20 12
## p4333 p4334 p4335 p4336 p4337 p4338 p4339 p4340 p4341 p4342 p4343 p4344
## 35 20 26 79 12 30 15 19 29 20 11 11
## p4345 p4346 p4347 p4348 p4349 p4350 p4351 p4352 p4353 p4354 p4355 p4356
## 11 16 18 11 13 24 11 11 12 15 25 15
## p4357 p4358 p4359 p4360 p4361 p4362 p4363 p4364 p4365 p4366 p4367 p4368
## 14 34 13 11 11 18 15 19 14 21 21 23
## p4369 p4370 p4371 p4372 p4373 p4374 p4375 p4376 p4377 p4378 p4379 p4380
## 60 12 18 17 16 13 12 38 31 13 18 13
## p4381 p4382 p4383 p4384 p4385 p4386 p4387 p4388 p4389 p4390 p4391 p4392
## 32 17 14 26 21 20 30 18 13 11 17 12
## p4393 p4394 p4395 p4396 p4397 p4398 p4399 p4400 p4401 p4402 p4403 p4404
## 20 14 23 23 12 26 55 28 12 27 57 16
## p4405 p4406 p4407 p4408 p4409 p4410 p4411 p4412 p4413 p4414 p4415 p4416
## 14 27 16 23 18 27 23 20 31 22 29 13
## p4417 p4418 p4419 p4420 p4421 p4422 p4423 p4424 p4425 p4426 p4427 p4428
## 14 14 20 23 14 17 20 12 25 16 11 13
## p4429 p4430 p4431 p4432 p4433 p4434 p4435 p4436 p4437 p4438 p4439 p4440
## 12 17 15 12 18 15 16 12 13 11 12 17
## p4441 p4442 p4443 p4444 p4445 p4446 p4447 p4448 p4449 p4450 p4451 p4452
## 19 13 11 13 11 11 12 16 23 18 35 13
## p4453 p4454 p4455 p4456 p4457 p4458 p4459 p4460 p4461 p4462 p4463 p4464
## 14 18 11 19 17 14 11 17 11 22 13 11
## p4465 p4466 p4467 p4468 p4469 p4470 p4471 p4472 p4473 p4474 p4475 p4476
## 13 19 12 14 19 41 17 11 23 11 13 33
## p4477 p4478 p4479 p4480 p4481 p4482 p4483 p4484 p4485 p4486 p4487 p4488
## 24 54 11 16 13 16 11 19 13 20 15 16
## p4489 p4490 p4491 p4492 p4493 p4494 p4495 p4496 p4497 p4498 p4499 p4500
## 12 13 11 22 15 13 11 18 13 11 46 13
## p4501 p4502 p4503 p4504 p4505 p4506 p4507 p4508 p4509 p4510 p4511 p4512
## 24 15 21 14 20 51 16 17 15 14 11 21
## p4513 p4514 p4515 p4516 p4517 p4518 p4519 p4520 p4521 p4522 p4523 p4524
## 22 17 12 14 16 16 29 34 12 19 11 14
## p4525 p4526 p4527 p4528 p4529 p4530 p4531 p4532 p4533 p4534 p4535 p4536
## 13 28 13 15 13 11 13 11 14 38 37 12
## p4537 p4538 p4539 p4540 p4541 p4542 p4543 p4544 p4545 p4546 p4547 p4548
## 42 44 22 12 30 14 20 29 20 17 11 19we can see the variability in plots as below;
Plot of transactions per salesperson:
barplot(totS,
main='Transactions per salesperson',
names.arg='',xlab='Salespeople',
ylab='Amount')
Plot of transactions per product:
barplot(totP,
main='Transactions per product',
names.arg='',xlab='Products',
ylab='Amount')
Variables Quant and Val show a lot of variability also, indicating differences in the products, thus, they might be better handled separately.
If prices of the products are very different it may only be possible to identify abnormal transactions in the context of the same product.
However, given the disparate quantity of products that are sold on each transaction,it might make more sense to carry out this analysis over the unit price instead.
We add this derived unit price per transaction as a new column to the dataframe
sales$Uprice <- sales$Val/sales$QuantUnit price should be relatively constant over the transactions of the same product.
When analyzing transactions over a short period of time, one does not expect strong variations of the unit price of the products.
We check the distribution of the unit price:
sales$Uprice <- sales$Val/sales$QuantUnit price should be relatively constant over the transactions of the same product.When analyzing transactions over a short period of time, one does not expect strong variations of the unit price of the products.
We check the distribution of the unit price:
summary(sales$Uprice)## Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
## 0.00 8.46 11.89 20.30 19.11 26460.00 14136We again observed a marked variability.
Given this observation, we should analyze the set of transactions on each product individually, looking for suspicious transactions on each of these sets.
One problem is that some products have very few transactions. Of the 4,548 products,982 have fewer than 20 transactions.
Declaring a report as unusual based on a sample of fewer than 20 reports may be too risky.
Attaching the varable sales as below
attach(sales)We obtain median unit price of each product.
upp <- aggregate(Uprice,list(Prod),median,na.rm=T)Let’s take a look at upp, just the first ten
upp[1:10,]## Group.1 x
## 1 p1 11.428571
## 2 p2 10.877863
## 3 p3 10.000000
## 4 p4 9.911243
## 5 p5 11.000000
## 6 p6 13.270677
## 7 p7 4.851453
## 8 p8 3.850211
## 9 p9 1.941457
## 10 p10 42.232846We generate five most (and least) expensive product lsiting
topP <- sapply(c(T,F),function(o)
upp[order(upp[,2],
decreasing=o)[1:5],1])
colnames(topP) <- c('Expensive','Cheap')
topP## Expensive Cheap
## [1,] "p3689" "p560"
## [2,] "p2453" "p559"
## [3,] "p2452" "p4195"
## [4,] "p2456" "p601"
## [5,] "p2459" "p563"We confirm the completely different price distribution of the top products using a boxplot of their unit price
tops <- sales[Prod %in% topP[1,],
c('Prod','Uprice')]
head(tops)## Prod Uprice
## 2382 p560 1.454210e-02
## 30006 p3689 2.069307e+01
## 30007 p3689 2.116790e+04
## 30008 p3689 2.267961e+02
## 36164 p560 1.454210e-02
## 69854 p3689 1.221728e+04tops$Prod <- factor(tops$Prod)The scales of the prices of the most expensive and least expensive products are rather different.So we use a log scale to keep the values of the cheap product from being indistinguishable.Y-axis is on log scale.
boxplot(Uprice ~ Prod,data=tops,
ylab='Uprice',log="y")
We carry out a similar analysis to discover which salespeople are ones who bring more (less) money into the company.
vs <- aggregate(Val,list(ID),sum,na.rm=T)
scoresSs <- sapply(c(T,F),function(o)
vs[order(vs$x,decreasing=o)[1:5],1])
colnames(scoresSs) <- c('Most','Least')
scoresSs## Most Least
## [1,] "v431" "v3355"
## [2,] "v54" "v6069"
## [3,] "v19" "v5876"
## [4,] "v4520" "v6058"
## [5,] "v955" "v4515"The top 100 salespeople account for almost 40% of the company income, while the bottom 2,000 (of 6,016 salespeople) generate less than 2% of the income:
Percent of company income top 100 salespeople:
sum(vs[order(vs$x,decreasing=T)[1:100],2])/sum(Val,na.rm=T)*100## [1] 38.33277Percent of company income bottom 2,000:
sum(vs[order(vs$x,decreasing=F)[1:2000],2])/sum(Val,na.rm=T)*100## [1] 1.988716If we carry out a similar analysis in terms of the quantity that is sold for each product, the results are even more unbalanced:
qs <- aggregate(Quant,list(Prod),sum,na.rm=T)
scoresPs <- sapply(c(T,F),function(o)
qs[order(qs$x,decreasing=o)[1:5],1])
colnames(scoresPs) <- c('Most','Least')
scoresPs## Most Least
## [1,] "p2516" "p2442"
## [2,] "p3599" "p2443"
## [3,] "p314" "p1653"
## [4,] "p569" "p4101"
## [5,] "p319" "p3678"Top 100 products represent nearly 75% of sales volume:
sum(as.double(qs[order(qs$x,decreasing=T)[1:100],2]))/sum(as.double(Quant),na.rm=T)*100## [1] 74.634784,000 of the 4,548 products account for less than 10% of the sales volume:
sum(as.double(qs[order(qs$x,decreasing=F)[1:4000],2]))/sum(as.double(Quant),na.rm=T)*100## [1] 8.944681Sales people can change the price of an item if they want, but we still assume that the unit price of any product should follow a near-normal distribution.
We can conduct some basic tests to find deviations from this normality assumption:
The Box-Plot Rule: Box-plots show outliers. The rule is that an observation should be tagged as an anomaly, a high (low) value if it is above (below) the high (low) whisker which is defined as Q3+(1.5 x IQR) for high and Q1-(1.5 x IQR) for the low values, where Q1 is the first quartile, Q3 is the third quartile, and IQR=(Q3-Q1) and is the inter-quartile range.
This ‘Box-Plot’ Rule works well for normally- distributed variables, and is robust to the presence of a few outliers since it is based in robust statistics using quartiles.
We determine the number of outliers (by above definition) of each product:
out <- tapply(Uprice,list(Prod=Prod),
function(x) length(boxplot.stats(x)$out))The products with more outliers are:
out[order(out,decreasing=T)[1:10]]## Prod
## p1125 p1437 p2273 p1917 p1918 p4089 p538 p3774 p2742 p3338
## 376 181 165 156 156 137 129 125 120 117We see that 29,446 transactions are outliers:
sum(out)## [1] 29446which is approximately 7% of total transactions:
sum(out)/nrow(sales)*100## [1] 7.34047Data problems
Unknown (Missing) Values
3 basic alternatives: 1) Remove them; 2) Fill them in with some strategy; or 3) Use tools that can handle them.
The salespersons and products involved in the problematic transactions with unknowns in both Val and Quant are :
nas <- sales[which(is.na(Quant) & is.na(Val)),
c('ID','Prod')]
nrow(nas)## [1] 888it appears that the option of removing all transactions with unknown values on both quantity and value is the best option.
detach(sales)
sales <- sales[-which(is.na(sales$Quant) & is.na(sales$Val)),]Calculate proportion of transactions of each product that have quantity unknown and delete them and update the levels
nnasQp <- tapply(sales$Quant,list(sales$Prod),
function(x) sum(is.na(x)))
propNAsQp <- nnasQp/table(sales$Prod)
propNAsQp[order(propNAsQp,decreasing=T)[1:10]]## p2442 p2443 p1653 p4101 p4243 p903 p3678
## 1.0000000 1.0000000 0.9090909 0.8571429 0.6842105 0.6666667 0.6666667
## p3955 p4464 p1261
## 0.6428571 0.6363636 0.6333333sales <- sales[!sales$Prod %in% c('p2442','p2443'),]
nlevels(sales$Prod)## [1] 4548sales$Prod <- factor(sales$Prod)
nlevels(sales$Prod)## [1] 4546Are there salespeople with all transactions that have an unknown quantity?:
nnasQs <- tapply(sales$Quant,list(sales$ID),function(x) sum(is.na(x)))
propNAsQs <- nnasQs/table(sales$ID)
propNAsQs[order(propNAsQs,decreasing=T)[1:10]]## v2925 v5537 v5836 v6058 v6065 v4368 v2923
## 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 0.8888889 0.8750000
## v2970 v4910 v4542
## 0.8571429 0.8333333 0.8095238We carry out a similar analysis for transactions with an unknown value in Val. First, proportion of transactions of each product with unknown value in this column:
nnasVp <- tapply(sales$Val,list(sales$Prod),
function(x) sum(is.na(x)))
propNAsVp <- nnasVp/table(sales$Prod)
propNAsVp[order(propNAsVp,decreasing=T)[1:10]]## p1110 p1022 p4491 p1462 p80 p4307
## 0.25000000 0.17647059 0.10000000 0.07500000 0.06250000 0.05882353
## p4471 p2821 p1017 p4287
## 0.05882353 0.05389222 0.05263158 0.05263158We begin by obtaining this typical unit price for each product. We skip prices of transactions already deemed to be fraudulent. For remaining (non-fraud) transactions we use median unit price of the transactions as the ‘typical’ unit price:
tPrice <- tapply(sales[sales$Insp != 'fraud','Uprice'],
list(sales[sales$Insp != 'fraud',
'Prod']),median,na.rm=T)
nrow(tPrice)## [1] 4546We fill in remaining missing values for Quantity and Value
noQuant <- which(is.na(sales$Quant))
sales[noQuant,'Quant'] <- ceiling(sales[noQuant,'Val'] /
tPrice[sales[noQuant,'Prod']])
noVal <- which(is.na(sales$Val))
sales[noVal,'Val'] <- sales[noVal,'Quant'] *
tPrice[sales[noVal,'Prod']]We have filled in Quant and Val values so now we recalculate the Uprice column to fill in the previously unknown unit prices:
sales$Uprice <- sales$Val/sales$QuantDefining the Data Mining Tasks
We want to decide which transaction reports should be considered for inspection as result of strong suspicion of being fraudulent.We want the guidance to take the form of a ranking of fraud probability.
‘Insp’ column has info about previous inspections.Have small number that have been judged, are ‘OK’ or ‘Fraud’
Have another large number that have not been inspected, they are marked ‘unkn’.
These represent two different groups for our purposes; there are different modeling approaches that can be applied to either group.
We might compare their unit price with the typical price of the reports of the same product. We would expect a higher difference to be an indication that something is wrong.
So we can use this difference, or ‘distance’, as a good indicator of the quality of the outlier ranking obtained by the model.
Here is a function the calculates the value of this statistic:
avgNDTP <- function(toInsp,train,stats) {
if (missing(train) && missing(stats))
stop('Provide either the training data or the product stats')
if (missing(stats)) {
notF <- which(train$Insp != 'fraud')
stats <- tapply(train$Uprice[notF],
list(Prod=train$Prod[notF]),
function(x) {
bp <- boxplot.stats(x)$stats
c(median=bp[3],
iqr=bp[4]-bp[2])
})
stats <- matrix(unlist(stats),
length(stats),2,byrow=T,
dimnames=list(names(stats),
c('median',
'iqr')))
stats[which(stats[,'iqr']==0),'iqr'] <-
stats[which(stats[,'iqr']==0),'median']
}
mdtp <- mean(abs(toInsp$Uprice-stats[toInsp$Prod,'median']) /
stats[toInsp$Prod,'iqr'])
return(mdtp)
}Must provide test set, ranking proposed by the model for this set, threshold of inspection limit effort and stats (median and IQR) of products. We use this function soon
evalOutlierRanking <- function(testSet,
rankOrder,
Threshold,
statsProds) {
ordTS <- testSet[rankOrder,]
N <- nrow(testSet)
nF <- if (Threshold < 1) as.integer(Threshold*N) else Threshold
cm <- table(c(rep('fraud',nF),
rep('ok',N-nF)),
ordTS$Insp)
prec <- cm['fraud','fraud']/sum(cm['fraud',])
rec <- cm['fraud','fraud']/sum(cm[,'fraud'])
AVGndtp <- avgNDTP(ordTS[nF,],
stats=statsProds)
return(c(Precision=prec,Recall=rec,
avgNDTP=AVGndtp))
}OBTAINING OUTLIER RANKINGS
We use different models to obtain outlier rankings.For each attempt we will estimate its results using a stratified 70%/30% hold-out strategy.
UNSUPERVISED APPROACHES
The modified box plot rule
We described the box plot rule, which can be used to detect outliers of a continuous variable if it follows a near-normal distribution like with the unit price of the products. In this context, we can think of this simple rule as the baseline method that we can apply to our data.
This function receives a set of transactions and obtains their ranking order and score. Parameters are the training and test data sets:
BPrule <- function(train,test) {
# leave out those already labeled 'fraud':
notF <- which(train$Insp != 'fraud')
# calculates median and IQR values per product:
ms <- tapply(train$Uprice[notF],list(Prod=train$Prod[notF]),
function(x) {
bp <- boxplot.stats(x)$stats
c(median=bp[3],iqr=bp[4]-bp[2])
})
ms <- matrix(unlist(ms),length(ms),2,byrow=T,
dimnames=list(names(ms),c('median','iqr')))
ms[which(ms[,'iqr']==0),'iqr'] <- ms[which(ms[,'iqr']==0),'median']
# then uses this stats to obtain outlier score:
ORscore <- abs(test$Uprice-ms[test$Prod,'median']) /
ms[test$Prod,'iqr']
# then returns a list with this score and the
# rank order of the test set observations:
return(list(rankOrder=order(ORscore,decreasing=T),
rankScore=ORscore))
}Now we evaluate this method using hold-out experimental methodology and calculate alues of median and IQR for each product required to calculate average NDTP score
notF <- which(sales$Insp != 'fraud')
globalStats <- tapply(sales$Uprice[notF],
list(Prod=sales$Prod[notF]),
function(x) {
bp <- boxplot.stats(x)$stats
c(median=bp[3],iqr=bp[4]-bp[2])
})
globalStats <- matrix(unlist(globalStats),
length(globalStats),2,byrow=T,
dimnames=list(names(globalStats),
c('median','iqr')))
globalStats[which(globalStats[,'iqr']==0),'iqr'] <-
globalStats[which(globalStats[,'iqr']==0),'median']The holdOut() function needs to call a routine to obtain and evaluate the BPrule method for each iteration of the experimental process as below
ho.BPrule <- function(form, train, test, ...) {
res <- BPrule(train,test)
# we attach other R objects to the attributes to the
# vector of scores with the BPrule method, it is a
# list that contains the predicted and true values
# that originated these scores:
structure(evalOutlierRanking(test,res$rankOrder,...),
# create object with attribute itInfo:
itInfo=list(preds=res$rankScore,
trues=ifelse(test$Insp=='fraud',1,0)
)
)
}
bp.res <- holdOut(learner('ho.BPrule',
pars=list(Threshold=0.1,
statsProds=globalStats)),
dataset(Insp ~ .,sales),
# args are number of reps,
# percentage of cases included in
# hold out sample, set.seed() value, and
# T means stratified sampling should
# be used:
hldSettings(3,0.3,1234,T),
# makes storage take place (see way above):
itsInfo=TRUE
)##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
## Repetition 1
## Repetition 2
## Repetition 3summary(bp.res)##
## == Summary of a Hold Out Experiment ==
##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
##
## * Data set :: sales
## * Learner :: ho.BPrule with parameters:
## Threshold = 0.1
## statsProds = 11.34 ...
##
## * Summary of Experiment Results:## Precision Recall avgNDTP
## avg 0.0166305736 0.52293272 1.87123901
## std 0.0008983669 0.01909992 0.05379945
## min 0.0159920040 0.51181102 1.80971393
## max 0.0176578377 0.54498715 1.90944329
## invalid 0.0000000000 0.00000000 0.00000000To obtain the PR and cumulative recall charts
par(mfrow=c(1,2))
info <- attr(bp.res,'itsInfo')
PTs.bp <- aperm(array(unlist(info),
dim=c(length(info[[1]]),2,3)),
c(1,3,2)
)
PRcurve(PTs.bp[,,1],PTs.bp[,,2],
main='PR curve',avg='vertical')
CRchart(PTs.bp[,,1],PTs.bp[,,2],
main='Cumulative Recall curve',
avg='vertical')
We see in cumulative recall chart that method obtains around 40% of recall with a very low inspective effort….to achieve values around 80%, we need to inspect 25%-30% of the reports.
Local outlier factors (LOF)
main idea of LOF system is to obtain an “outlyingness”score for each case by estimating its degree of isolation with respect to its local neighborhood.
Method is based on notion of local density of the observations. Cases in regions with very low density are considered outliers. The estimates of the density are obtained using the distances between cases. The authors defined a few concepts that drive algorithm to calculate “outlyingness”.score of each point. They are:
concept of core distance of a point p,which is defined as its distance to its nearest neighbor, (2) concept of reachability distance between case p1 and p2, which is given by the maximum of the core distance of p1 and the dis- tance between both cases, and (3) local reachability distance of a point which is inversely proportional to the average reachability distance of its k neighbors. The LOF of a case is calculated as a function of its local reachability distance.
DMwR package implementation of LOF algorithm: this function obtains evaluation statistics from applying LOF method to given training and test set. We merged train and test datasets and use LOF to rank this full set of reports. From the obtained ranking we then select the outlier scores of the cases belonging to the test set.
ho.LOF <- function(form, train, test, k, ...) {
require(dprep,quietly=T)
ntr <- nrow(train)
all <- rbind(train,test)
N <- nrow(all)
ups <- split(all$Uprice,all$Prod)
r <- list(length=ups)
for(u in seq(along=ups))
r[[u]] <- if (NROW(ups[[u]]) > 3)
lofactor(ups[[u]],min(k,NROW(ups[[u]]) %/% 2))
else if (NROW(ups[[u]])) rep(0,NROW(ups[[u]]))
else NULL
all$lof <- vector(length=N)
split(all$lof,all$Prod) <- r
all$lof[which(!(is.infinite(all$lof) | is.nan(all$lof)))] <-
SoftMax(all$lof[which(!(is.infinite(all$lof) | is.nan(all$lof)))])
structure(evalOutlierRanking(test,order(all[(ntr+1):N,'lof'],
decreasing=T),...),
itInfo=list(preds=all[(ntr+1):N,'lof'],
trues=ifelse(test$Insp=='fraud',1,0))
)
}
lof.res <- holdOut(learner('ho.LOF',
# set k = 7, might tune with more experiments
pars=list(k=7,Threshold=0.1,
statsProds=globalStats)),
dataset(Insp ~ .,sales),
hldSettings(3,0.3,1234,T),
itsInfo=TRUE
)##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
## Repetition 1## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'##
## Repetition 2## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'##
## Repetition 3## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'summary(lof.res)##
## == Summary of a Hold Out Experiment ==
##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
##
## * Data set :: sales
## * Learner :: ho.LOF with parameters:
## k = 7
## Threshold = 0.1
## statsProds = 11.34 ...
##
## * Summary of Experiment Results:## Precision Recall avgNDTP
## avg 0.0221278250 0.69595344 2.4631856
## std 0.0009136811 0.02019331 0.9750265
## min 0.0214059637 0.67454068 1.4420851
## max 0.0231550891 0.71465296 3.3844572
## invalid 0.0000000000 0.00000000 0.0000000precision and Recall plot for LOF method is below
par(mfrow=c(1,2))
info <- attr(lof.res,'itsInfo')
PTs.lof <- aperm(array(unlist(info),
dim=c(length(info[[1]]),2,3)),
c(1,3,2)
)
PRcurve(PTs.bp[,,1],PTs.bp[,,2],
main='PR curve',lty=1,
xlim=c(0,1),ylim=c(0,1),
avg='vertical')
PRcurve(PTs.lof[,,1],PTs.lof[,,2],
add=T,lty=2,
avg='vertical')
legend('topright',c('BPrule','LOF'),
lty=c(1,2))
CRchart(PTs.bp[,,1],PTs.bp[,,2],
main='Cumulative Recall curve',
lty=1,xlim=c(0,1),ylim=c(0,1),
avg='vertical')
CRchart(PTs.lof[,,1],PTs.lof[,,2],
add=T,lty=2,
avg='vertical')
legend('bottomright',c('BPrule','LOF'),
lty=c(1,2))
Clustering-based outlier rankings (OR_h)
The next outlier ranking method we consider is based on a clustering algorithm. The OR_h method uses a hierarchical agglomerative clustering algorithm to obtain a dendrogram of the given data.Dendrograms are visual representations of the merging process of these clustering methods.
ho.ORh <- function(form, train, test, ...) {
require(dprep,quietly=T)
ntr <- nrow(train)
all <- rbind(train,test)
N <- nrow(all)
ups <- split(all$Uprice,all$Prod)
r <- list(length=ups)
for(u in seq(along=ups))
r[[u]] <- if (NROW(ups[[u]]) > 3)
outliers.ranking(ups[[u]])$prob.outliers
else if (NROW(ups[[u]])) rep(0,NROW(ups[[u]]))
else NULL
all$lof <- vector(length=N)
split(all$lof,all$Prod) <- r
all$lof[which(!(is.infinite(all$lof) | is.nan(all$lof)))] <-
SoftMax(all$lof[which(!(is.infinite(all$lof) | is.nan(all$lof)))])
structure(evalOutlierRanking(test,order(all[(ntr+1):N,'lof'],
decreasing=T),...),
itInfo=list(preds=all[(ntr+1):N,'lof'],
trues=ifelse(test$Insp=='fraud',1,0))
)
}
orh.res <- holdOut(learner('ho.ORh',
pars=list(Threshold=0.1,
statsProds=globalStats)),
dataset(Insp ~ .,sales),
hldSettings(3,0.3,1234,T),
itsInfo=TRUE
)##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
## Repetition 1## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'##
## Repetition 2## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'##
## Repetition 3## Warning in library(package, lib.loc = lib.loc, character.only = TRUE,
## logical.return = TRUE, : there is no package called 'dprep'summary(orh.res)##
## == Summary of a Hold Out Experiment ==
##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
##
## * Data set :: sales
## * Learner :: ho.ORh with parameters:
## Threshold = 0.1
## statsProds = 11.34 ...
##
## * Summary of Experiment Results:## Precision Recall avgNDTP
## avg 0.0220445333 0.69345072 0.5444893
## std 0.0005545834 0.01187721 0.3712311
## min 0.0215725471 0.67979003 0.2893128
## max 0.0226553390 0.70133333 0.9703665
## invalid 0.0000000000 0.00000000 0.0000000The precision and Recall graph is below
par(mfrow=c(1,2))
info <- attr(orh.res,'itsInfo')
PTs.orh <- aperm(array(unlist(info),dim=c(length(info[[1]]),2,3)),
c(1,3,2)
)
PRcurve(PTs.bp[,,1],PTs.bp[,,2],
main='PR curve',lty=1,xlim=c(0,1),ylim=c(0,1),
avg='vertical')
PRcurve(PTs.lof[,,1],PTs.lof[,,2],
add=T,lty=2,
avg='vertical')
PRcurve(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('topright',c('BPrule','LOF','ORh'),
lty=c(1,2,1),col=c('black','black','grey'))
CRchart(PTs.bp[,,1],PTs.bp[,,2],
main='Cumulative Recall curve',lty=1,xlim=c(0,1),ylim=c(0,1),
avg='vertical')
CRchart(PTs.lof[,,1],PTs.lof[,,2],
add=T,lty=2,
avg='vertical')
CRchart(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('bottomright',c('BPrule','LOF','ORh'),
lty=c(1,2,1),col=c('black','black','grey'))
the results of the OR_h method are comparable to LOF in terms of the cumulative recall curve. However, regarding the PR curve, the OR_h system dominates the score of LOF, with a smaller advantage over BPrule.
SUPERVISED APPROACHES
Naive Bayes
Naive Bayes is a probabilistic classifier based on the Bayes theorem that uses strong assumptions on the independence between the predictors.
nb <- function(train,test) {
require(e1071,quietly=T)
sup <- which(train$Insp != 'unkn')
data <- train[sup,c('ID','Prod','Uprice','Insp')]
data$Insp <- factor(data$Insp,levels=c('ok','fraud'))
model <- naiveBayes(Insp ~ .,data)
preds <- predict(model,test[,c('ID','Prod','Uprice','Insp')],type='raw')
return(list(rankOrder=order(preds[,'fraud'],decreasing=T),
rankScore=preds[,'fraud'])
)
}
ho.nb <- function(form, train, test, ...) {
res <- nb(train,test)
structure(evalOutlierRanking(test,res$rankOrder,...),
itInfo=list(preds=res$rankScore,
trues=ifelse(test$Insp=='fraud',1,0)
)
)
}
nb.res <- holdOut(learner('ho.nb',
pars=list(Threshold=0.1,
statsProds=globalStats)),
dataset(Insp ~ .,sales),
hldSettings(3,0.3,1234,T),
itsInfo=TRUE
)##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
## Repetition 1## Warning: package 'e1071' was built under R version 3.0.3##
## Repetition 2
## Repetition 3summary(nb.res)##
## == Summary of a Hold Out Experiment ==
##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
##
## * Data set :: sales
## * Learner :: ho.nb with parameters:
## Threshold = 0.1
## statsProds = 11.34 ...
##
## * Summary of Experiment Results:## Precision Recall avgNDTP
## avg 0.013715365 0.43112103 0.8519657
## std 0.001083859 0.02613164 0.2406771
## min 0.012660336 0.40533333 0.5908980
## max 0.014825920 0.45758355 1.0650114
## invalid 0.000000000 0.00000000 0.0000000Precision and Recall plot
par(mfrow=c(1,2))
info <- attr(nb.res,'itsInfo')
PTs.nb <- aperm(array(unlist(info),dim=c(length(info[[1]]),2,3)),
c(1,3,2)
)
PRcurve(PTs.nb[,,1],PTs.nb[,,2],
main='PR curve',lty=1,xlim=c(0,1),
ylim=c(0,1), avg='vertical')
PRcurve(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('topright',c('NaiveBayes','ORh'),
lty=1,col=c('black','grey'))
CRchart(PTs.nb[,,1],PTs.nb[,,2],
main='Cumulative Recall curve',
lty=1,xlim=c(0,1),ylim=c(0,1),
avg='vertical')
CRchart(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('bottomright',c('NaiveBayes','ORh'),
lty=1,col=c('black','grey'))
A possible cause for the poor performance of the Naive Bayes may be the class imbalance. So now we will apply the Naive Bayes classifier using a training set obtained using SMOTE().
nb.s <- function(train,test) {
require(e1071,quietly=T)
sup <- which(train$Insp != 'unkn')
data <- train[sup,c('ID','Prod','Uprice','Insp')]
data$Insp <- factor(data$Insp,levels=c('ok','fraud'))
newData <- SMOTE(Insp ~ .,data,perc.over=700)
model <- naiveBayes(Insp ~ .,newData)
preds <- predict(model,test[,c('ID','Prod','Uprice','Insp')],type='raw')
return(list(rankOrder=order(preds[,'fraud'],decreasing=T),
rankScore=preds[,'fraud'])
)
}
ho.nbs <- function(form, train, test, ...) {
res <- nb.s(train,test)
structure(evalOutlierRanking(test,res$rankOrder,...),
itInfo=list(preds=res$rankScore,
trues=ifelse(test$Insp=='fraud',1,0)
)
)
}
nbs.res <- holdOut(learner('ho.nbs',
pars=list(Threshold=0.1,
statsProds=globalStats)),
dataset(Insp ~ .,sales),
hldSettings(3,0.3,1234,T),
itsInfo=TRUE
)##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
## Repetition 1
## Repetition 2
## Repetition 3summary(nbs.res)##
## == Summary of a Hold Out Experiment ==
##
## Stratified 3 x 70 %/ 30 % Holdout run with seed = 1234
##
## * Data set :: sales
## * Learner :: ho.nbs with parameters:
## Threshold = 0.1
## statsProds = 11.34 ...
##
## * Summary of Experiment Results:## Precision Recall avgNDTP
## avg 0.014215115 0.44686510 0.8913330
## std 0.001109167 0.02710388 0.8482740
## min 0.013493253 0.43044619 0.1934613
## max 0.015492254 0.47814910 1.8354999
## invalid 0.000000000 0.00000000 0.0000000par(mfrow=c(1,2))
info <- attr(nbs.res,'itsInfo')
PTs.nbs <- aperm(array(unlist(info),
dim=c(length(info[[1]]),
2,3)),c(1,3,2))
PRcurve(PTs.nb[,,1],PTs.nb[,,2],
main='PR curve',lty=1,xlim=c(0,1),
ylim=c(0,1),
avg='vertical')
PRcurve(PTs.nbs[,,1],PTs.nbs[,,2],
add=T,lty=2,
avg='vertical')
PRcurve(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('topright',c('NaiveBayes',
'smoteNaiveBayes','ORh'),
lty=c(1,2,1),col=c('black','black','grey'))
CRchart(PTs.nb[,,1],PTs.nb[,,2],
main='Cumulative Recall curve',
lty=1,xlim=c(0,1),ylim=c(0,1),
avg='vertical')
CRchart(PTs.nbs[,,1],PTs.nbs[,,2],
add=T,lty=2,
avg='vertical')
CRchart(PTs.orh[,,1],PTs.orh[,,2],
add=T,lty=1,col='grey',
avg='vertical')
legend('bottomright',c('NaiveBayes','smoteNaiveBayes','ORh'),
lty=c(1,2,1),col=c('black','black','grey'))
We see the disappointing results of the SMOTE’d version of Naive Bayes. In effect, it shows the same poor results as the standard Naive Bayes when compared to OR_h and, moreover, its performance is almost always surpassed by the standard version.