Problem 2 Final

2.1 Go to https://www.kaggle.com/c/digit-recognizer/overview, accept the rules of the competition, and download the data. You will not be required to submit work to Kaggle, but you do need the data. ’MNIST (“Modified National Institute of Standards and Technology”) is the de facto “hello world” dataset of computer vision. Since its release in 1999, this classic dataset of handwritten images has served as the basis for benchmarking classification algorithms. As new machine learning techniques emerge, MNIST remains a reliable resource for researchers and learners alike.”

The training data frame contains 1460 observations of 81 variables.

train <- read.csv('C:/Users/daria/Downloads/train.csv')

label	pixel121	pixel122	pixel123	pixel124	pixel125	pixel126	pixel127	pixel128	pixel129	pixel130	pixel132	pixel133	pixel134	pixel146	pixel147	pixel148	pixel149	pixel150	pixel151	pixel152	pixel153	pixel154	pixel155	pixel156	pixel157	pixel158	pixel159	pixel160	pixel161	pixel162	pixel174	pixel175	pixel176	pixel177	pixel178	pixel179	pixel180	pixel181	pixel182	pixel183	pixel184	pixel185	pixel186	pixel187	pixel188	pixel189	pixel190	pixel203	pixel204	pixel205	pixel206	pixel207	pixel208	pixel209	pixel210	pixel211	pixel212	pixel213	pixel214	pixel215	pixel216	pixel217	pixel230	pixel231	pixel232	pixel233	pixel234	pixel235	pixel236	pixel237	pixel238	pixel239	pixel240	pixel241	pixel242	pixel243	pixel244	pixel245	pixel258	pixel259	pixel260	pixel261	pixel262	pixel263	pixel264	pixel265	pixel266	pixel267	pixel268	pixel269	pixel270	pixel271	pixel272	pixel273	pixel274	pixel285	pixel286	pixel287	pixel288	pixel289	pixel290	pixel291	pixel292	pixel293	pixel294	pixel295	pixel296	pixel297	pixel298	pixel299	pixel300	pixel301	pixel302	pixel303	pixel313	pixel314	pixel315	pixel316	pixel317	pixel318	pixel319	pixel320	pixel321	pixel322	pixel323	pixel324	pixel325	pixel326	pixel327	pixel328	pixel329	pixel330	pixel331	pixel332	pixel341	pixel342	pixel343	pixel344	pixel345	pixel346	pixel348	pixel349	pixel350	pixel351	pixel352	pixel353	pixel354	pixel355	pixel356	pixel357	pixel358	pixel359	pixel360	pixel369	pixel370	pixel371	pixel372	pixel373	pixel374	pixel376	pixel377	pixel378	pixel379	pixel380	pixel381	pixel382	pixel383	pixel384	pixel385	pixel386	pixel387	pixel388	pixel396	pixel397	pixel398	pixel399	pixel400	pixel401	pixel402	pixel403	pixel404	pixel405	pixel406	pixel407	pixel408	pixel409	pixel410	pixel411	pixel412	pixel413	pixel414	pixel415	pixel416	pixel424	pixel425	pixel426	pixel427	pixel428	pixel429	pixel430	pixel431	pixel432	pixel433	pixel434	pixel435	pixel436	pixel437	pixel438	pixel439	pixel440	pixel441	pixel442	pixel443	pixel444	pixel452	pixel453	pixel454	pixel455	pixel456	pixel457	pixel458	pixel459	pixel460	pixel461	pixel462	pixel463	pixel464	pixel465	pixel466	pixel467	pixel468	pixel469	pixel470	pixel471	pixel472	pixel481	pixel482	pixel483	pixel484	pixel485	pixel486	pixel487	pixel488	pixel489	pixel490	pixel491	pixel492	pixel493	pixel494	pixel495	pixel496	pixel497	pixel498	pixel499	pixel500	pixel509	pixel510	pixel511	pixel512	pixel513	pixel514	pixel515	pixel516	pixel517	pixel518	pixel519	pixel520	pixel521	pixel522	pixel523	pixel524	pixel525	pixel526	pixel527	pixel528	pixel537	pixel538	pixel539	pixel540	pixel541	pixel542	pixel543	pixel544	pixel545	pixel546	pixel547	pixel548	pixel549	pixel550	pixel551	pixel552	pixel553	pixel554	pixel555	pixel565	pixel566	pixel567	pixel568	pixel569	pixel570	pixel571	pixel572	pixel573	pixel574	pixel575	pixel576	pixel577	pixel578	pixel579	pixel580	pixel581	pixel582	pixel594	pixel595	pixel596	pixel597	pixel598	pixel599	pixel600	pixel601	pixel602	pixel603	pixel604	pixel605	pixel606	pixel607	pixel608	pixel609	pixel622	pixel623	pixel624	pixel625	pixel626	pixel627	pixel628	pixel629	pixel630	pixel631	pixel632	pixel633	pixel634	pixel635	pixel636	pixel637	pixel651	pixel652	pixel653	pixel654	pixel655	pixel656	pixel657	pixel658	pixel659	pixel660	pixel661	pixel662	pixel664	pixel665	pixel691	pixel692	pixel693
1	0	0	0	0	0	0	0	0	0	0	188	255	94	0	0	0	0	0	0	0	0	0	0	0	0	0	191	250	253	93	0	0	0	0	0	0	0	0	0	0	0	0	123	248	253	167	10	0	0	0	0	0	0	0	0	0	0	80	247	253	208	13	0	0	0	0	0	0	0	0	0	0	29	207	253	235	77	0	0	0	0	0	0	0	0	0	0	54	209	253	253	88	0	0	0	0	0	0	0	0	0	0	0	0	93	254	253	238	170	17	0	0	0	0	0	0	0	0	0	0	0	0	23	210	254	253	159	0	0	0	0	0	0	0	0	0	0	0	0	0	16	209	253	254	240	81	0	0	0	0	0	0	0	0	0	0	0	0	0	27	253	253	254	13	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	20	206	254	254	198	7	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	168	253	253	196	7	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	20	203	253	248	76	0	0	0	0	0	0	0	0	0	0	0	0	0	0	22	188	253	245	93	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	103	253	253	191	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	89	240	253	195	25	0	0	0	0	0	0	0	0	0	0	0	0	0	15	220	253	253	80	0	0	0	0	0	0	0	0	0	0	0	0	94	253	253	253	94	0	0	0	0	0	0	0	0	0	0	0	89	251	253	250	131	0	0	0	0	0	0	0	0	0	0	0	214	218	95	0	0	0	0	0	0	0	0	0	0	0	0	0
0	0	18	30	137	137	192	86	72	1	0	0	0	0	0	0	13	86	250	254	254	254	254	217	246	151	32	0	0	0	0	0	16	179	254	254	254	254	254	254	254	254	254	231	54	15	0	0	72	254	254	254	254	254	254	254	254	254	254	254	254	104	0	61	191	254	254	254	254	254	109	83	199	254	254	254	254	243	85	172	254	254	254	202	147	147	45	0	11	29	200	254	254	254	171	0	1	174	254	254	89	67	0	0	0	0	0	0	128	252	254	254	212	76	0	47	254	254	254	29	0	0	0	0	0	0	0	0	83	254	254	254	153	0	0	80	254	254	240	24	0	0	0	0	0	0	0	25	240	254	254	153	0	0	64	254	254	186	7	0	0	0	0	0	0	0	0	166	254	254	224	12	0	14	232	254	254	254	29	0	0	0	0	0	0	0	0	0	75	254	254	254	17	0	18	254	254	254	254	29	0	0	0	0	0	0	0	0	0	48	254	254	254	17	0	2	163	254	254	254	29	0	0	0	0	0	0	0	0	0	48	254	254	254	17	0	94	254	254	254	200	12	0	0	0	0	0	0	0	16	209	254	254	150	1	0	15	206	254	254	254	202	66	0	0	0	0	0	21	161	254	254	245	31	0	0	0	60	212	254	254	254	194	48	48	34	41	48	209	254	254	254	171	0	0	0	0	86	243	254	254	254	254	254	233	243	254	254	254	254	254	86	0	0	0	114	254	254	254	254	254	254	254	254	254	254	239	86	11	0	0	13	182	254	254	254	254	254	254	254	254	243	70	0	0	0	0	8	76	146	254	255	254	255	146	19	15	0	0	0	0	0
1	0	0	0	3	141	139	3	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	8	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	8	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	106	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	6	185	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	89	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	4	146	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	9	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	156	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	185	255	255	184	0	0	0	0	0	0	0	0	0	0	0	0	0	185	254	254	184	0	0	0	0	0	0	0	0	0	0	0	0	185	254	254	184	0	0	0	0	0	0	0	0	0	0	0	63	254	254	62	0	0	0	0	0	0	0	0
4	0	0	0	0	0	0	0	0	0	0	0	0	0	220	179	6	0	0	0	0	0	0	0	0	9	77	0	0	0	0	28	247	17	0	0	0	0	0	0	0	0	27	202	0	0	0	0	242	155	0	0	0	0	0	0	0	0	27	254	63	0	0	0	160	207	6	0	0	0	0	0	0	0	27	254	65	0	0	0	127	254	21	0	0	0	0	0	0	0	20	239	65	0	0	0	0	0	77	254	21	0	0	0	0	0	0	0	0	195	65	0	0	0	0	0	0	70	254	21	0	0	0	0	0	0	0	0	195	142	0	0	0	0	0	0	0	56	251	21	0	0	0	0	0	0	0	195	227	0	0	0	0	0	0	0	0	222	153	5	0	0	0	0	0	0	120	240	13	0	0	0	0	0	0	0	0	67	251	40	0	0	0	0	0	0	0	94	255	69	0	0	0	0	0	0	0	0	0	234	184	0	0	0	0	0	0	0	19	245	69	0	0	0	0	0	0	0	0	0	234	169	0	0	0	0	0	0	0	3	199	182	10	0	0	0	0	0	0	0	154	205	4	0	0	26	72	128	203	208	254	254	131	0	0	0	0	0	0	0	61	254	129	113	186	245	251	189	75	56	136	254	73	0	0	0	0	0	0	0	15	216	233	233	159	104	52	0	0	0	38	254	73	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	18	254	73	0	0	0	0	0	0	0	0	0	0	0	0	0	0	18	254	73	0	0	0	0	0	0	0	0	0	0	0	0	0	5	206	106	0	0	0	0	0	0	0	0	0	0	0	0	186	159	6	209	101
0	1	25	130	155	254	254	254	157	30	2	0	0	0	0	0	8	103	253	253	253	253	253	253	253	253	114	2	0	0	0	0	11	208	253	253	253	253	253	253	253	253	253	253	107	0	0	0	31	253	253	253	253	253	253	253	253	253	253	253	215	101	3	23	210	253	253	253	248	161	222	222	246	253	253	253	253	253	39	136	253	253	253	229	77	0	0	0	70	218	253	253	253	253	215	91	5	214	253	253	253	195	0	0	0	0	0	104	224	253	253	253	253	215	29	116	253	253	253	247	75	0	0	0	0	0	0	26	200	253	253	253	253	216	4	254	253	253	253	195	0	0	0	0	0	0	0	26	200	253	253	253	253	5	254	253	253	253	99	0	0	0	0	0	0	0	0	25	231	253	253	253	36	0	254	253	253	253	99	0	0	0	0	0	0	0	0	0	0	223	253	253	253	129	0	254	253	253	253	99	0	0	0	0	0	0	0	0	0	0	127	253	253	253	129	0	254	253	253	253	99	0	0	0	0	0	0	0	0	0	0	139	253	253	253	90	254	253	253	253	99	0	0	0	0	0	0	0	0	0	78	248	253	253	253	5	254	253	253	253	216	34	0	0	0	0	0	0	0	33	152	253	253	253	107	1	206	253	253	253	253	140	0	0	0	0	0	30	139	234	253	253	253	154	2	16	205	253	253	253	250	208	106	106	106	200	237	253	253	253	253	209	22	82	253	253	253	253	253	253	253	253	253	253	253	253	253	209	22	1	91	253	253	253	253	253	253	253	253	253	253	213	90	7	0	1	18	129	208	253	253	253	253	159	129	90	4	0	0	0	0	0
0	0	0	0	3	141	202	254	193	44	0	0	0	0	0	0	0	0	0	5	165	254	179	163	249	244	72	0	0	0	0	0	0	0	0	0	135	254	150	0	0	189	254	243	31	0	0	0	0	0	0	82	248	209	5	0	0	164	236	254	115	0	0	0	0	0	8	211	254	58	0	0	0	0	33	230	212	6	0	0	0	0	119	254	156	3	0	0	0	0	18	230	254	33	0	0	0	0	0	10	212	254	35	0	0	0	0	0	33	254	254	33	0	0	0	0	0	0	116	254	154	3	0	0	0	0	0	33	254	254	33	0	0	0	0	0	0	0	124	254	115	0	0	0	0	0	160	254	239	23	0	0	0	0	0	0	0	203	254	35	0	0	0	0	0	197	254	178	0	0	0	0	0	0	0	0	23	239	221	11	0	0	0	0	0	0	198	255	123	0	0	0	0	0	0	0	0	23	238	178	0	0	0	0	0	0	10	219	254	96	0	0	0	0	0	0	0	0	30	249	204	0	0	0	0	0	0	25	235	254	62	0	0	0	0	0	0	0	26	243	204	0	0	0	0	0	0	91	254	248	36	0	0	0	0	0	0	0	33	254	204	0	0	0	0	0	67	241	254	133	0	0	0	0	0	0	0	0	33	254	214	7	0	0	0	50	242	254	194	24	0	0	0	0	0	0	0	5	193	254	78	0	0	19	128	254	195	36	0	0	0	0	0	0	0	103	254	222	74	143	235	254	228	83	0	0	0	0	0	0	0	30	242	254	254	254	254	252	84	0	0	0	0	0	0	0	0	23	64	158	200	174	61	0	0	0	0	0	0	0	0	0

## [1] 42000   785

2.2 Using the training.csv file, plot representations of the first 10 images to understand the data format. Go ahead and divide all pixels by 255 to produce values between 0 and 1. (This is equivalent to min-max scaling.) (5 points)

Plot first 10 images:

for (image_num in 1:10){
  p <- train[image_num,2:785]
  df1 <- as.data.frame(matrix(nrow=784, ncol=3))
  colnames(df1) <- c("x","y", "value")
  x <- 0
  y <- 1
  for (i in 1:784){
    x <- x+1
    if (x %% 29 == 0) {
      y <- y+1
      x <- 1
    }
    df1[i,] <- c(x,y,p[[i]])
  }
  as.cimg(df1) %>% plot    # image
}

We see that the pictures above match the first ten labels from the data frame.

train$label[1:10]

##  [1] 1 0 1 4 0 0 7 3 5 3

Now, we divide all pixels by 255 and plot again first ten numbers:

train_scaled <- train[,2:ncol(train)]/255
train_scaled$label <- train$label
head(train_scaled) %>% 
  kable %>%
  kable_styling("striped", full_width = F) %>% 
 scroll_box(width = "900px", height = "220px")

pixel121	pixel122	pixel123	pixel124	pixel125	pixel126	pixel127	pixel128	pixel129	pixel130	pixel132	pixel133	pixel134	pixel146	pixel147	pixel148	pixel149	pixel150	pixel151	pixel152	pixel153	pixel154	pixel155	pixel156	pixel157	pixel158	pixel159	pixel160	pixel161	pixel162	pixel174	pixel175	pixel176	pixel177	pixel178	pixel179	pixel180	pixel181	pixel182	pixel183	pixel184	pixel185	pixel186	pixel187	pixel188	pixel189	pixel190	pixel203	pixel204	pixel205	pixel206	pixel207	pixel208	pixel209	pixel210	pixel211	pixel212	pixel213	pixel214	pixel215	pixel216	pixel217	pixel230	pixel231	pixel232	pixel233	pixel234	pixel235	pixel236	pixel237	pixel238	pixel239	pixel240	pixel241	pixel242	pixel243	pixel244	pixel245	pixel258	pixel259	pixel260	pixel261	pixel262	pixel263	pixel264	pixel265	pixel266	pixel267	pixel268	pixel269	pixel270	pixel271	pixel272	pixel273	pixel274	pixel285	pixel286	pixel287	pixel288	pixel289	pixel290	pixel291	pixel292	pixel293	pixel294	pixel295	pixel296	pixel297	pixel298	pixel299	pixel300	pixel301	pixel302	pixel303	pixel313	pixel314	pixel315	pixel316	pixel317	pixel318	pixel319	pixel320	pixel321	pixel322	pixel323	pixel324	pixel325	pixel326	pixel327	pixel328	pixel329	pixel330	pixel331	pixel332	pixel341	pixel342	pixel343	pixel344	pixel345	pixel346	pixel348	pixel349	pixel350	pixel351	pixel352	pixel353	pixel354	pixel355	pixel356	pixel357	pixel358	pixel359	pixel360	pixel369	pixel370	pixel371	pixel372	pixel373	pixel374	pixel376	pixel377	pixel378	pixel379	pixel380	pixel381	pixel382	pixel383	pixel384	pixel385	pixel386	pixel387	pixel388	pixel396	pixel397	pixel398	pixel399	pixel400	pixel401	pixel402	pixel403	pixel404	pixel405	pixel406	pixel407	pixel408	pixel409	pixel410	pixel411	pixel412	pixel413	pixel414	pixel415	pixel416	pixel424	pixel425	pixel426	pixel427	pixel428	pixel429	pixel430	pixel431	pixel432	pixel433	pixel434	pixel435	pixel436	pixel437	pixel438	pixel439	pixel440	pixel441	pixel442	pixel443	pixel444	pixel452	pixel453	pixel454	pixel455	pixel456	pixel457	pixel458	pixel459	pixel460	pixel461	pixel462	pixel463	pixel464	pixel465	pixel466	pixel467	pixel468	pixel469	pixel470	pixel471	pixel472	pixel481	pixel482	pixel483	pixel484	pixel485	pixel486	pixel487	pixel488	pixel489	pixel490	pixel491	pixel492	pixel493	pixel494	pixel495	pixel496	pixel497	pixel498	pixel499	pixel500	pixel509	pixel510	pixel511	pixel512	pixel513	pixel514	pixel515	pixel516	pixel517	pixel518	pixel519	pixel520	pixel521	pixel522	pixel523	pixel524	pixel525	pixel526	pixel527	pixel528	pixel537	pixel538	pixel539	pixel540	pixel541	pixel542	pixel543	pixel544	pixel545	pixel546	pixel547	pixel548	pixel549	pixel550	pixel551	pixel552	pixel553	pixel554	pixel555	pixel565	pixel566	pixel567	pixel568	pixel569	pixel570	pixel571	pixel572	pixel573	pixel574	pixel575	pixel576	pixel577	pixel578	pixel579	pixel580	pixel581	pixel582	pixel594	pixel595	pixel596	pixel597	pixel598	pixel599	pixel600	pixel601	pixel602	pixel603	pixel604	pixel605	pixel606	pixel607	pixel608	pixel609	pixel622	pixel623	pixel624	pixel625	pixel626	pixel627	pixel628	pixel629	pixel630	pixel631	pixel632	pixel633	pixel634	pixel635	pixel636	pixel637	pixel651	pixel652	pixel653	pixel654	pixel655	pixel656	pixel657	pixel658	pixel659	pixel660	pixel661	pixel662	pixel664	pixel665	pixel691	pixel692	pixel693	label
0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7372549	1	0.3686275	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7490196	0.9803922	0.9921569	0.3647059	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4823529	0.9725490	0.9921569	0.654902	0.0392157	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3137255	0.9686275	0.9921569	0.8156863	0.0509804	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1137255	0.8117647	0.9921569	0.9215686	0.3019608	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2117647	0.8196078	0.9921569	0.9921569	0.3450980	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3647059	0.9960784	0.9921569	0.9333333	0.6666667	0.0666667	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0901961	0.8235294	0.9960784	0.9921569	0.6235294	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0627451	0.8196078	0.9921569	0.9960784	0.9411765	0.3176471	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1058824	0.9921569	0.9921569	0.9960784	0.0509804	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0784314	0.8078431	0.9960784	0.9960784	0.7764706	0.027451	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.6588235	0.9921569	0.9921569	0.7686275	0.0274510	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0784314	0.7960784	0.9921569	0.9725490	0.2980392	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0862745	0.7372549	0.9921569	0.9607843	0.3647059	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4039216	0.9921569	0.9921569	0.7490196	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3490196	0.9411765	0.9921569	0.7647059	0.0980392	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0588235	0.8627451	0.9921569	0.9921569	0.3137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3686275	0.9921569	0.9921569	0.9921569	0.3686275	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3490196	0.9843137	0.9921569	0.9803922	0.5137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.8392157	0.8549020	0.3725490	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	1
0.0000000	0.0705882	0.1176471	0.5372549	0.5372549	0.7529412	0.3372549	0.2823529	0.0039216	0.0000000	0.0000000	0	0.0000000	0.0000000	0.0000000	0.0509804	0.3372549	0.9803922	0.9960784	0.9960784	0.9960784	0.9960784	0.8509804	0.9647059	0.5921569	0.1254902	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0627451	0.7019608	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9058824	0.2117647	0.0588235	0.000000	0.0000000	0.2823529	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.4078431	0.0000000	0.2392157	0.7490196	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.4274510	0.3254902	0.7803922	0.9960784	0.9960784	0.9960784	0.9960784	0.9529412	0.3333333	0.6745098	0.9960784	0.9960784	0.9960784	0.7921569	0.5764706	0.5764706	0.1764706	0.0000000	0.0431373	0.1137255	0.7843137	0.9960784	0.9960784	0.9960784	0.6705882	0.0000000	0.0039216	0.6823529	0.9960784	0.9960784	0.3490196	0.2627451	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.5019608	0.9882353	0.9960784	0.9960784	0.8313725	0.2980392	0.0000000	0.1843137	0.9960784	0.9960784	0.9960784	0.1137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3254902	0.9960784	0.9960784	0.9960784	0.6000000	0.0000000	0.0000000	0.3137255	0.9960784	0.9960784	0.9411765	0.0941176	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0980392	0.9411765	0.9960784	0.9960784	0.6000000	0.0000000	0.0000000	0.2509804	0.9960784	0.9960784	0.7294118	0.0274510	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.6509804	0.9960784	0.9960784	0.8784314	0.0470588	0.0000000	0.054902	0.9098039	0.9960784	0.9960784	0.9960784	0.1137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.2941176	0.9960784	0.9960784	0.9960784	0.0666667	0.0000000	0.0705882	0.9960784	0.9960784	0.9960784	0.9960784	0.1137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1882353	0.9960784	0.9960784	0.9960784	0.0666667	0.0000000	0.0078431	0.6392157	0.9960784	0.9960784	0.9960784	0.1137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1882353	0.9960784	0.9960784	0.9960784	0.0666667	0.0000000	0.3686275	0.9960784	0.9960784	0.9960784	0.7843137	0.0470588	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0627451	0.8196078	0.9960784	0.9960784	0.5882353	0.0039216	0.0000000	0.0588235	0.8078431	0.9960784	0.9960784	0.9960784	0.7921569	0.2588235	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0823529	0.6313725	0.9960784	0.9960784	0.9607843	0.1215686	0.0000000	0.0000000	0.0000000	0.2352941	0.8313725	0.9960784	0.9960784	0.9960784	0.7607843	0.1882353	0.1882353	0.1333333	0.1607843	0.1882353	0.8196078	0.9960784	0.9960784	0.9960784	0.6705882	0.0000000	0.0000000	0.0000000	0.0000000	0.3372549	0.9529412	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9137255	0.9529412	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.3372549	0.0000000	0.0000000	0.0000000	0.4470588	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9372549	0.3372549	0.0431373	0.0000000	0.0000000	0.0509804	0.7137255	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9960784	0.9529412	0.2745098	0.0000000	0.0000000	0.0000000	0.0000000	0.0313725	0.2980392	0.5725490	0.9960784	1.0000000	0.9960784	1.0000000	0.5725490	0.0745098	0.0588235	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0
0.0000000	0.0000000	0.0000000	0.0117647	0.5529412	0.5450980	0.0117647	0.0000000	0.0000000	0.0000000	0.0000000	0	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.0313725	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.0313725	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.4156863	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0235294	0.7254902	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3490196	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0156863	0.5725490	0.9960784	0.7215686	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.6117647	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7254902	1.0000000	1.0000000	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7254902	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7254902	0.9960784	0.9960784	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2470588	0.9960784	0.9960784	0.2431373	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	1
0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0	0.0000000	0.8627451	0.7019608	0.0235294	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0352941	0.3019608	0.0000000	0.0000000	0.0000000	0.0000000	0.1098039	0.9686275	0.0666667	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1058824	0.7921569	0.0000000	0.0000000	0.000000	0.0000000	0.9490196	0.6078431	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1058824	0.9960784	0.2470588	0.0000000	0.0000000	0.0000000	0.6274510	0.8117647	0.0235294	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1058824	0.9960784	0.2549020	0.0000000	0.0000000	0.0000000	0.4980392	0.9960784	0.0823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0784314	0.9372549	0.2549020	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3019608	0.9960784	0.0823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7647059	0.2549020	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2745098	0.9960784	0.0823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7647059	0.5568627	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2196078	0.9843137	0.0823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7647059	0.8901961	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.8705882	0.6000000	0.0196078	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.4705882	0.9411765	0.0509804	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.2627451	0.9843137	0.1568627	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.3686275	1.0000000	0.2705882	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.9176471	0.7215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0745098	0.9607843	0.2705882	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.9176471	0.6627451	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0117647	0.7803922	0.7137255	0.0392157	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.6039216	0.8039216	0.0156863	0.0000000	0.0000000	0.1019608	0.2823529	0.5019608	0.7960784	0.8156863	0.9960784	0.9960784	0.5137255	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2392157	0.9960784	0.5058824	0.4431373	0.7294118	0.9607843	0.9843137	0.7411765	0.2941176	0.2196078	0.5333333	0.9960784	0.2862745	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0588235	0.8470588	0.9137255	0.9137255	0.6235294	0.4078431	0.2039216	0.0000000	0.0000000	0.0000000	0.1490196	0.9960784	0.2862745	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0705882	0.9960784	0.2862745	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0705882	0.9960784	0.2862745	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0196078	0.8078431	0.4156863	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7294118	0.6235294	0.0235294	0.8196078	0.3960784	4
0.0039216	0.0980392	0.5098039	0.6078431	0.9960784	0.9960784	0.9960784	0.6156863	0.1176471	0.0078431	0.0000000	0	0.0000000	0.0000000	0.0000000	0.0313725	0.4039216	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.4470588	0.0078431	0.0000000	0.0000000	0.0000000	0.0000000	0.0431373	0.8156863	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.4196078	0.0000000	0.000000	0.0000000	0.1215686	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.8431373	0.3960784	0.0117647	0.0901961	0.8235294	0.9921569	0.9921569	0.9921569	0.9725490	0.6313725	0.8705882	0.8705882	0.9647059	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.1529412	0.5333333	0.9921569	0.9921569	0.9921569	0.8980392	0.3019608	0.0000000	0.0000000	0.0000000	0.2745098	0.8549020	0.9921569	0.9921569	0.9921569	0.9921569	0.8431373	0.3568627	0.0196078	0.8392157	0.9921569	0.9921569	0.9921569	0.7647059	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4078431	0.8784314	0.9921569	0.9921569	0.9921569	0.9921569	0.8431373	0.1137255	0.4549020	0.9921569	0.9921569	0.9921569	0.9686275	0.2941176	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1019608	0.7843137	0.9921569	0.9921569	0.9921569	0.9921569	0.8470588	0.0156863	0.9960784	0.9921569	0.9921569	0.9921569	0.7647059	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1019608	0.7843137	0.9921569	0.9921569	0.9921569	0.9921569	0.0196078	0.9960784	0.9921569	0.9921569	0.9921569	0.3882353	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0980392	0.9058824	0.9921569	0.9921569	0.9921569	0.1411765	0.000000	0.9960784	0.9921569	0.9921569	0.9921569	0.3882353	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.8745098	0.9921569	0.9921569	0.9921569	0.5058824	0.0000000	0.9960784	0.9921569	0.9921569	0.9921569	0.3882353	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4980392	0.9921569	0.9921569	0.9921569	0.5058824	0.0000000	0.9960784	0.9921569	0.9921569	0.9921569	0.3882353	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.5450980	0.9921569	0.9921569	0.9921569	0.3529412	0.9960784	0.9921569	0.9921569	0.9921569	0.3882353	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3058824	0.9725490	0.9921569	0.9921569	0.9921569	0.0196078	0.9960784	0.9921569	0.9921569	0.9921569	0.8470588	0.1333333	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.5960784	0.9921569	0.9921569	0.9921569	0.4196078	0.0039216	0.8078431	0.9921569	0.9921569	0.9921569	0.9921569	0.5490196	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1176471	0.5450980	0.9176471	0.9921569	0.9921569	0.9921569	0.6039216	0.0078431	0.0627451	0.8039216	0.9921569	0.9921569	0.9921569	0.9803922	0.8156863	0.4156863	0.4156863	0.4156863	0.7843137	0.9294118	0.9921569	0.9921569	0.9921569	0.9921569	0.8196078	0.0862745	0.3215686	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.8196078	0.0862745	0.0039216	0.3568627	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.9921569	0.8352941	0.3529412	0.0274510	0.0000000	0.0039216	0.0705882	0.5058824	0.8156863	0.9921569	0.9921569	0.9921569	0.9921569	0.6235294	0.5058824	0.3529412	0.0156863	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0
0.0000000	0.0000000	0.0000000	0.0117647	0.5529412	0.7921569	0.9960784	0.7568627	0.1725490	0.0000000	0.0000000	0	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0196078	0.6470588	0.9960784	0.7019608	0.6392157	0.9764706	0.9568627	0.2823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.5294118	0.9960784	0.5882353	0.0000000	0.0000000	0.7411765	0.9960784	0.9529412	0.1215686	0.0000000	0.000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3215686	0.9725490	0.8196078	0.0196078	0.0000000	0.0000000	0.6431373	0.9254902	0.9960784	0.4509804	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0313725	0.8274510	0.9960784	0.2274510	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.9019608	0.8313725	0.0235294	0.0000000	0.0000000	0.0000000	0.0000000	0.4666667	0.9960784	0.6117647	0.0117647	0.0000000	0.0000000	0.0000000	0.0000000	0.0705882	0.9019608	0.9960784	0.1294118	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0392157	0.8313725	0.9960784	0.1372549	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.9960784	0.9960784	0.1294118	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4549020	0.9960784	0.6039216	0.0117647	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.9960784	0.9960784	0.1294118	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4862745	0.9960784	0.4509804	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.6274510	0.9960784	0.9372549	0.0901961	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.7960784	0.9960784	0.1372549	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.772549	0.9960784	0.6980392	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.0000000	0.0000000	0.0901961	0.9372549	0.8666667	0.0431373	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.000000	0.7764706	1.0000000	0.4823529	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0901961	0.9333333	0.6980392	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0392157	0.8588235	0.9960784	0.3764706	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1176471	0.9764706	0.8000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0980392	0.9215686	0.9960784	0.2431373	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1019608	0.9529412	0.8000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.3568627	0.9960784	0.9725490	0.1411765	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.9960784	0.8000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.2627451	0.9450980	0.9960784	0.5215686	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1294118	0.9960784	0.8392157	0.0274510	0.0000000	0.0000000	0.0000000	0.1960784	0.9490196	0.9960784	0.7607843	0.0941176	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0196078	0.7568627	0.9960784	0.3058824	0.0000000	0.0000000	0.0745098	0.5019608	0.9960784	0.7647059	0.1411765	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.4039216	0.9960784	0.8705882	0.2901961	0.5607843	0.9215686	0.9960784	0.8941176	0.3254902	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.1176471	0.9490196	0.9960784	0.9960784	0.9960784	0.9960784	0.9882353	0.3294118	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0901961	0.2509804	0.6196078	0.7843137	0.6823529	0.2392157	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0.0000000	0

for (image_num in 1:10){
  p <- train_scaled[image_num,2:785]
  df1 <- as.data.frame(matrix(nrow=784, ncol=3))
  colnames(df1) <- c("x","y", "value")
  x <- 0
  y <- 1
  for (i in 1:784){
    x <- x+1
    if (x %% 29 == 0) {
      y <- y+1
      x <- 1
    }
    df1[i,] <- c(x,y,p[[i]])
  }
  as.cimg(df1) %>% plot    
}

2.3 What is the frequency distribution of the numbers in the dataset?

Digit 1 is the most common in the data frame, followed by 7.

train_freq <- train_scaled  %>% 
  group_by(label) %>% 
  summarise(Frequency = n()) 
train_freq

## # A tibble: 10 x 2
##    label Frequency
##    <int>     <int>
##  1     0      4132
##  2     1      4684
##  3     2      4177
##  4     3      4351
##  5     4      4072
##  6     5      3795
##  7     6      4137
##  8     7      4401
##  9     8      4063
## 10     9      4188

ggplot(train_freq, aes(x = label,y=Frequency, fill = Frequency)) + 
  geom_histogram(stat='identity') + 
  labs(title = 'Frequency Distribution of Digits, Training Set',
       x = 'Digit') +
  scale_x_continuous(breaks = 0:9) +
  scale_fill_gradient(low = "green", high = "red")

2.4 For each number, provide the mean pixel intensity. What does this tell you? (5 points)

The intensity means the brightness. Number 5 has the highest intensity and the darkest picture as the result.

instensity <- train_scaled %>% 
  group_by(label) %>% 
  summarise(intensity = sum(train_scaled[2:785])/(n()*784), n= n())
instensity

## # A tibble: 10 x 3
##    label intensity     n
##    <int>     <dbl> <int>
##  1     0      1.39  4132
##  2     1      1.23  4684
##  3     2      1.37  4177
##  4     3      1.32  4351
##  5     4      1.41  4072
##  6     5      1.51  3795
##  7     6      1.39  4137
##  8     7      1.30  4401
##  9     8      1.41  4063
## 10     9      1.37  4188

2.5 Reduce the data by using principal components that account for 95% of the variance. How many components did you generate? Use PCA to generate all possible components (100% of the variance). How many components are possible? Why? (5 points)

prcomp() function will help with the Principal Components Analysis:

df <- train_scaled

train_prcomp <- prcomp(df)
train_cum <- (cumsum(train_prcomp$sdev^2) / sum(train_prcomp$sdev^2))

There are 139 components that account for 95% of the variance

plot(train_cum)

cum_95 <- which.max(train_cum  >= .95)
cum_95

## [1] 139

We hit 100% variance at 705 component. 784 components are possible as there are 784 columns in the training data frame.

which.max(train_cum >= 1)

## [1] 705

2.6 Plot the first 10 images generated by PCA. They will appear to be noise. Why? (5 points)

PCA is used for data reduction, non-useful pixels have been removed. PCA doesn’t serve for cleared images.

df <- train_prcomp$x[, 1:8] %*% t(train_prcomp$rotation[, 1:8])
df <- scale(df, scale = TRUE, center = TRUE)

for (image_num in 1:10){
  p <- df[image_num,]
  df1 <- as.data.frame(matrix(nrow=784, ncol=3))
  colnames(df1) <- c("x","y", "value")
  x <- 0
  y <- 1
  for (i in 1:784){
    x <- x+1
    if (x %% 29 == 0) {
      y <- y+1
      x <- 1
    }
    df1[i,] <- c(x,y,p[[i]])
  }
  as.cimg(df1) %>% plot    
}

2.7 Now, select only those images that have labels that are 8’s. Re-run PCA that accounts for all of the variance (100%). Plot the first 10 images. What do you see? (5 points)

The data frame with 8s only:

train_8 <- train %>% 
  filter(label == 8)
train_8 <- train_8 %>% dplyr::select(c(2:785))
dim(train_8)

## [1] 4063  784

We hit 100% variance at 537 component.

train_8_scaled <- train_8/255

train_8_prcomp <- prcomp(train_8_scaled)
train_8_cum <- (cumsum(train_8_prcomp$sdev^2) / sum(train_8_prcomp$sdev^2))

plot(train_8_cum)

cum_1 <- which.max(train_8_cum  >= 1)
cum_1

## [1] 537

The pictures are clearer for sure.

df <- train_8_prcomp$x[, 1:784] %*% t(train_8_prcomp$rotation[, 1:784])
df <- scale(df, scale = TRUE, center = TRUE)

for (image_num in 1:10){
  p <- df[image_num,]
  df1 <- as.data.frame(matrix(nrow=784, ncol=3))
  colnames(df1) <- c("x","y", "value")
  x <- 0
  y <- 1
  for (i in 1:784){
    x <- x+1
    if (x %% 29 == 0) {
      y <- y+1
      x <- 1
    }
    df1[i,] <- c(x,y,p[[i]])
  }
  as.cimg(df1) %>% plot    
}

2.8 An incorrect approach to predicting the images would be to build a linear regression model with y as the digit values and X as the pixel matrix. Instead, we can build a multinomial model that classifies the digits. Build a multinomial model on the entirety of the training set. Then provide its classification accuracy (percent correctly identified) as well as a matrix of observed versus forecast values (confusion matrix). This matrix will be a 10 x 10, and correct classifications will be on the diagonal. (10 points.

The multinomial model

model <- nnet::multinom(label~., data = train, MaxNWts = 42000)

## # weights:  7860 (7065 variable)
## initial  value 96708.573906 
## iter  10 value 27932.238987
## iter  20 value 22901.936560
## iter  30 value 21677.526948
## iter  40 value 21254.278400
## iter  50 value 21077.662091
## iter  60 value 21002.266892
## iter  70 value 20928.779026
## iter  80 value 20866.883697
## iter  90 value 20792.605325
## iter 100 value 20727.274628
## final  value 20727.274628 
## stopped after 100 iterations

Testing the model:

test_df <- train[,2:ncol(train)]
predicted <- predict(model, test_df, type = 'class')
summary(predicted)

##    0    1    2    3    4    5    6    7    8    9 
## 4081 5678 3943 4083 3620 3920 3885 4596 3770 4424

The 10x10 Confusion Matrix, the accuracy is 89%:

train_label <- as.factor(train$label)
confusionMatrix(predicted, train_label)

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction    0    1    2    3    4    5    6    7    8    9
##          0 3953    0   10    5   11   24   43    6   16   13
##          1   24 4605  146   96   83   77   54  113  397   83
##          2   17   12 3639   76   36   18   72   38   21   14
##          3    9   13   73 3762   11   88    3    7   66   51
##          4    8    4   26    3 3448   12   17   21   18   63
##          5   27   13   21  183   55 3283  101   13  178   46
##          6   10    2   13    6   24   44 3769    4   13    0
##          7   35   13  111   91   18   51   10 4053   35  179
##          8   38   22  111   91   47  141   41    7 3247   25
##          9   11    0   27   38  339   57   27  139   72 3714
## 
## Overall Statistics
##                                           
##                Accuracy : 0.8922          
##                  95% CI : (0.8892, 0.8952)
##     No Information Rate : 0.1115          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.8802          
##                                           
##  Mcnemar's Test P-Value : < 2.2e-16       
## 
## Statistics by Class:
## 
##                      Class: 0 Class: 1 Class: 2 Class: 3 Class: 4 Class: 5
## Sensitivity           0.95668   0.9831  0.87120  0.86463  0.84676  0.86509
## Specificity           0.99662   0.9712  0.99196  0.99147  0.99547  0.98333
## Pos Pred Value        0.96864   0.8110  0.92290  0.92138  0.95249  0.83750
## Neg Pred Value        0.99528   0.9978  0.98586  0.98447  0.98374  0.98655
## Prevalence            0.09838   0.1115  0.09945  0.10360  0.09695  0.09036
## Detection Rate        0.09412   0.1096  0.08664  0.08957  0.08210  0.07817
## Detection Prevalence  0.09717   0.1352  0.09388  0.09721  0.08619  0.09333
## Balanced Accuracy     0.97665   0.9772  0.93158  0.92805  0.92111  0.92421
##                      Class: 6 Class: 7 Class: 8 Class: 9
## Sensitivity           0.91105   0.9209  0.79916  0.88682
## Specificity           0.99694   0.9856  0.98621  0.98122
## Pos Pred Value        0.97014   0.8819  0.86127  0.83951
## Neg Pred Value        0.99035   0.9907  0.97866  0.98739
## Prevalence            0.09850   0.1048  0.09674  0.09971
## Detection Rate        0.08974   0.0965  0.07731  0.08843
## Detection Prevalence  0.09250   0.1094  0.08976  0.10533
## Balanced Accuracy     0.95399   0.9532  0.89269  0.93402