DATA 624 - Homework8 - Non-Linear Regression

Libraries

library(kableExtra)
library(tidyverse)
library(ggplot2)
library(dplyr)
library(TSstudio)
library(RColorBrewer)
library(GGally)
library(grid)
library(gridExtra)
library(mlbench)
library(psych)
library(cowplot)
library(corrplot)
library(caret)
library(geoR)
library(reshape)
library(naniar)
library(mice)
library(DMwR)
library(AppliedPredictiveModeling)
library(pls)
library(glmnet)
library(elasticnet)
library(earth)
library(kernlab)

Applied Predictive Modeling

Exercise 7.2

Friedman (1991) introduced several benchmark data sets create by simulation. One of these simulations used the following nonlinear equation to create data:

\(y=10\sin { (\pi { x }_{ 1 }{ x }_{ 2 }) } +20{ ({ x }_{ 3 }-0.5) }^{ 2 }+10{ x }_{ 4 }+5{ x }_{ 5 }+N(0,{ \sigma }^{ 2 })\)

where the x values are random variables uniformly distributed between [0, 1] (there are also 5 other non-informative variables also created in the simulation). The package mlbench contains a function called mlbench.friedman1 that simulates these data:

set.seed(200)

trainingData <- mlbench.friedman1(200, sd = 1)

## We convert the 'x' data from a matrix to a data frame. One reason is that this will give the columns names.
trainingData$x <- data.frame(trainingData$x)

## Look at the data using
featurePlot(trainingData$x, trainingData$y)

## or other methods.This creates a list with a vector 'y' and a matrix
## of predictors 'x'. Also simulate a large test set to
## estimate the true error rate with good precision:
testData <- mlbench.friedman1(5000, sd = 1)
testData$x <- data.frame(testData$x)

Tune several models on these data. For example:

KNN Model

knnModel <- train(x = trainingData$x,
                  y = trainingData$y,
                  method = "knn",
                  preProc = c("center", "scale"),
                  tuneLength = 10)

knnModel

## k-Nearest Neighbors 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Bootstrapped (25 reps) 
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
## Resampling results across tuning parameters:
## 
##   k   RMSE      Rsquared   MAE     
##    5  3.466085  0.5121775  2.816838
##    7  3.349428  0.5452823  2.727410
##    9  3.264276  0.5785990  2.660026
##   11  3.214216  0.6024244  2.603767
##   13  3.196510  0.6176570  2.591935
##   15  3.184173  0.6305506  2.577482
##   17  3.183130  0.6425367  2.567787
##   19  3.198752  0.6483184  2.592683
##   21  3.188993  0.6611428  2.588787
##   23  3.200458  0.6638353  2.604529
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 17.

knnPred <- predict(knnModel, newdata = testData$x)

## The function 'postResample' can be used to get the test set
## perforamnce values
postResample(pred = knnPred, obs = testData$y)

##      RMSE  Rsquared       MAE 
## 3.2040595 0.6819919 2.5683461

Which models appear to give the best performance? Does MARS select the informative predictors (those named X1–X5)?

MARS Model

MARS_grid <- expand.grid(.degree = 1:2, .nprune = 2:15)
MARS_model <- train(x = trainingData$x, 
                  y = trainingData$y,
                  method = "earth",
                  tuneGrid = MARS_grid,
                  preProcess = c("center", "scale"),
                  tuneLength = 10)
MARS_model

## Multivariate Adaptive Regression Spline 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Bootstrapped (25 reps) 
## Summary of sample sizes: 200, 200, 200, 200, 200, 200, ... 
## Resampling results across tuning parameters:
## 
##   degree  nprune  RMSE      Rsquared   MAE     
##   1        2      4.383438  0.2405683  3.597961
##   1        3      3.645469  0.4745962  2.930453
##   1        4      2.727602  0.7035031  2.184240
##   1        5      2.449243  0.7611230  1.939231
##   1        6      2.331605  0.7835496  1.833420
##   1        7      1.976830  0.8421599  1.562591
##   1        8      1.870959  0.8585503  1.464551
##   1        9      1.804342  0.8683110  1.410395
##   1       10      1.787676  0.8711960  1.386944
##   1       11      1.790700  0.8707740  1.393076
##   1       12      1.821005  0.8670619  1.419893
##   1       13      1.858688  0.8617344  1.445459
##   1       14      1.862343  0.8623072  1.446050
##   1       15      1.871033  0.8607099  1.457618
##   2        2      4.383438  0.2405683  3.597961
##   2        3      3.644919  0.4742570  2.929647
##   2        4      2.730222  0.7028372  2.183075
##   2        5      2.481291  0.7545789  1.965749
##   2        6      2.338369  0.7827873  1.825542
##   2        7      2.030065  0.8328250  1.602024
##   2        8      1.890997  0.8551326  1.477422
##   2        9      1.742626  0.8757904  1.371910
##   2       10      1.608221  0.8943432  1.255416
##   2       11      1.474325  0.9111463  1.157848
##   2       12      1.437483  0.9157967  1.120977
##   2       13      1.439395  0.9164721  1.128309
##   2       14      1.428565  0.9184503  1.118634
##   2       15      1.434093  0.9182413  1.121622
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 14 and degree = 2.

The optimal MARS model minimized the RMSE when the nprune = 14 and the degree = 2.

MARS_predictions <- predict(MARS_model, newdata = testData$x)
postResample(pred = MARS_predictions, obs = testData$y)

##      RMSE  Rsquared       MAE 
## 1.2779993 0.9338365 1.0147070

The RMSE of the MARS model is a lot lower than the KNN model.

Variable Importance Check

Let’s see what variables are important for MARS Model -

varImp(MARS_model)

## earth variable importance
## 
##    Overall
## X1  100.00
## X4   75.40
## X2   49.00
## X5   15.72
## X3    0.00

The MARS model picks up the X1-X5 variables.

SVM Model

SVM_model <- train(x = trainingData$x,
                   y = trainingData$y,
                   method = "svmRadial",
                   preProcess = c("center", "scale"),
                   tuneLength = 10,
                   trControl = trainControl(method = "cv"))


SVM_model

## Support Vector Machines with Radial Basis Function Kernel 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ... 
## Resampling results across tuning parameters:
## 
##   C       RMSE      Rsquared   MAE     
##     0.25  2.548095  0.7967225  2.001583
##     0.50  2.290826  0.8118225  1.779407
##     1.00  2.095918  0.8337552  1.634192
##     2.00  1.989469  0.8462100  1.539721
##     4.00  1.901332  0.8565135  1.510431
##     8.00  1.879815  0.8588126  1.514502
##    16.00  1.878866  0.8591568  1.518094
##    32.00  1.878866  0.8591568  1.518094
##    64.00  1.878866  0.8591568  1.518094
##   128.00  1.878866  0.8591568  1.518094
## 
## Tuning parameter 'sigma' was held constant at a value of 0.06670077
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.06670077 and C = 16.

The optimal SVM mdoel has a σ of 0.07 and an C of 16.

SVM_predictions <- predict(SVM_model, newdata = testData$x)
postResample(pred = SVM_predictions, obs = testData$y)

##      RMSE  Rsquared       MAE 
## 2.0829707 0.8242096 1.5826017

Variable Importance Check

varImp(SVM_model)

## loess r-squared variable importance
## 
##      Overall
## X4  100.0000
## X1   95.5047
## X2   89.6186
## X5   45.2170
## X3   29.9330
## X9    6.3299
## X10   5.5182
## X8    3.2527
## X6    0.8884
## X7    0.0000

The SVM picked up the important variables.

Neural Network Model

nnet_grid <- expand.grid(.decay = c(0, 0.01, .1), .size = c(1:10), .bag = FALSE)
nnet_maxnwts <- 5 * (ncol(trainingData$x) + 1) + 5 + 1
nnet_model <- train(x = trainingData$x,
                    y = trainingData$y,
                    method = "avNNet",
                    preProcess = c("center", "scale"),
                    tuneGrid = nnet_grid,
                    trControl = trainControl(method = "cv"),
                    linout = TRUE,
                    trace = FALSE,
                    MaxNWts = nnet_maxnwts,
                    maxit = 500)
nnet_model

## Model Averaged Neural Network 
## 
## 200 samples
##  10 predictor
## 
## Pre-processing: centered (10), scaled (10) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 180, 180, 180, 180, 180, 180, ... 
## Resampling results across tuning parameters:
## 
##   decay  size  RMSE      Rsquared   MAE     
##   0.00    1    2.459523  0.7691775  1.920417
##   0.00    2    2.517390  0.7556275  1.973484
##   0.00    3    2.170133  0.8136101  1.698007
##   0.00    4    2.206065  0.8194369  1.649463
##   0.00    5    2.319063  0.7986176  1.783178
##   0.00    6         NaN        NaN       NaN
##   0.00    7         NaN        NaN       NaN
##   0.00    8         NaN        NaN       NaN
##   0.00    9         NaN        NaN       NaN
##   0.00   10         NaN        NaN       NaN
##   0.01    1    2.443513  0.7700880  1.902169
##   0.01    2    2.498223  0.7610107  1.965276
##   0.01    3    2.243864  0.8018360  1.802783
##   0.01    4    2.060693  0.8303930  1.626430
##   0.01    5    2.158157  0.8242275  1.716552
##   0.01    6         NaN        NaN       NaN
##   0.01    7         NaN        NaN       NaN
##   0.01    8         NaN        NaN       NaN
##   0.01    9         NaN        NaN       NaN
##   0.01   10         NaN        NaN       NaN
##   0.10    1    2.450808  0.7677685  1.905063
##   0.10    2    2.566992  0.7445497  2.007383
##   0.10    3    2.203412  0.8081681  1.717865
##   0.10    4    2.001697  0.8435571  1.594042
##   0.10    5    2.019386  0.8412724  1.606449
##   0.10    6         NaN        NaN       NaN
##   0.10    7         NaN        NaN       NaN
##   0.10    8         NaN        NaN       NaN
##   0.10    9         NaN        NaN       NaN
##   0.10   10         NaN        NaN       NaN
## 
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 4, decay = 0.1 and bag = FALSE.

The best neural network has a size = 3 and a decay of 0.

nnet_predictions <- predict(nnet_model, newdata = testData$x)
postResample(pred = nnet_predictions, obs = testData$y)

##      RMSE  Rsquared       MAE 
## 2.0532719 0.8346367 1.5444560

Variable Importance Check

varImp(nnet_model)

## loess r-squared variable importance
## 
##      Overall
## X4  100.0000
## X1   95.5047
## X2   89.6186
## X5   45.2170
## X3   29.9330
## X9    6.3299
## X10   5.5182
## X8    3.2527
## X6    0.8884
## X7    0.0000

The top 5 variables include the intended list of X1-X5 variables.

Model Summary

results <- data.frame(t(postResample(pred = knnPred, obs = testData$y))) %>% 
  mutate("Model" = "KNN")

results <- data.frame(t(postResample(pred = MARS_predictions, obs = testData$y))) %>%
  mutate("Model"= "MARS") %>%
  bind_rows(results)

results <- data.frame(t(postResample(pred = SVM_predictions, obs = testData$y))) %>%
  mutate("Model"= "SVM") %>%
  bind_rows(results)

results <- data.frame(t(postResample(pred = nnet_predictions, obs = testData$y))) %>%
  mutate("Model"= "Neural Network") %>%
  bind_rows(results)

results %>%
  select(Model, RMSE, Rsquared, MAE) %>%
  arrange(RMSE) %>%
  kable() %>%
  kable_styling()

## Warning: namespace 'highr' is not available and has been replaced
## by .GlobalEnv when processing object '<unknown>'

Model	RMSE	Rsquared	MAE
MARS	1.277999	0.9338365	1.014707
Neural Network	2.053272	0.8346367	1.544456
SVM	2.082971	0.8242096	1.582602
KNN	3.204060	0.6819919	2.568346

The MARS model preformed the best and identified the right variables as the important ones. The \({ R }^{ 2 }\) on it is extremely high. This model’s performance with test data is pretty impressive.

Exercise 7.5

Exercise 6.3 describes data for a chemical manufacturing process. Use the same data imputation, data splitting, and pre-processing steps as before and train several nonlinear regression models.

Data set Preview

data(ChemicalManufacturingProcess)

ChemicalManufacturingProcess %>% kable() %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive")) %>% 
  scroll_box(width="100%",height="300px")

Yield	BiologicalMaterial01	BiologicalMaterial02	BiologicalMaterial03	BiologicalMaterial04	BiologicalMaterial05	BiologicalMaterial06	BiologicalMaterial07	BiologicalMaterial08	BiologicalMaterial09	BiologicalMaterial10	BiologicalMaterial11	BiologicalMaterial12	ManufacturingProcess01	ManufacturingProcess02	ManufacturingProcess03	ManufacturingProcess04	ManufacturingProcess05	ManufacturingProcess06	ManufacturingProcess07	ManufacturingProcess08	ManufacturingProcess09	ManufacturingProcess10	ManufacturingProcess11	ManufacturingProcess12	ManufacturingProcess13	ManufacturingProcess14	ManufacturingProcess15	ManufacturingProcess16	ManufacturingProcess17	ManufacturingProcess18	ManufacturingProcess19	ManufacturingProcess20	ManufacturingProcess21	ManufacturingProcess22	ManufacturingProcess23	ManufacturingProcess24	ManufacturingProcess25	ManufacturingProcess26	ManufacturingProcess27	ManufacturingProcess28	ManufacturingProcess29	ManufacturingProcess30	ManufacturingProcess31	ManufacturingProcess32	ManufacturingProcess33	ManufacturingProcess34	ManufacturingProcess35	ManufacturingProcess36	ManufacturingProcess37	ManufacturingProcess38	ManufacturingProcess39	ManufacturingProcess40	ManufacturingProcess41	ManufacturingProcess42	ManufacturingProcess43	ManufacturingProcess44	ManufacturingProcess45
38.00	6.25	49.58	56.97	12.74	19.51	43.73	100.00	16.66	11.44	3.46	138.09	18.83	NA	NA	NA	NA	NA	NA	NA	NA	43.00	NA	NA	NA	35.5	4898	6108	4682	35.5	4865	6049	4665	0.0	NA	NA	NA	4873	6074	4685	10.7	21.0	9.9	69.1	156	66	2.4	486	0.019	0.5	3	7.2	NA	NA	11.6	3.0	1.8	2.4
42.44	8.01	60.97	67.48	14.65	19.36	53.14	100.00	19.04	12.55	3.46	153.67	21.05	0.0	0.0	NA	917	1032.2	210.0	177	178	46.57	NA	NA	0	34.0	4869	6095	4617	34.0	4867	6097	4621	0.0	3	0	3	4869	6107	4630	11.2	21.4	9.9	68.7	169	66	2.6	508	0.019	2.0	2	7.2	0.1	0.15	11.1	0.9	1.9	2.2
42.03	8.01	60.97	67.48	14.65	19.36	53.14	100.00	19.04	12.55	3.46	153.67	21.05	0.0	0.0	NA	912	1003.6	207.1	178	178	45.07	NA	NA	0	34.8	4878	6087	4617	34.8	4877	6078	4621	0.0	4	1	4	4897	6116	4637	11.1	21.3	9.4	69.3	173	66	2.6	509	0.018	0.7	2	7.2	0.0	0.00	12.0	1.0	1.8	2.3
41.42	8.01	60.97	67.48	14.65	19.36	53.14	100.00	19.04	12.55	3.46	153.67	21.05	0.0	0.0	NA	911	1014.6	213.3	177	177	44.92	NA	NA	0	34.8	4897	6102	4635	34.8	4872	6073	4611	0.0	5	2	5	4892	6111	4630	11.1	21.3	9.4	69.3	171	68	2.5	496	0.018	1.2	2	7.2	0.0	0.00	10.6	1.1	1.8	2.1
42.49	7.47	63.33	72.25	14.02	17.91	54.66	100.00	18.22	12.80	3.05	147.61	21.05	10.7	0.0	NA	918	1027.5	205.7	178	178	44.96	NA	NA	0	34.6	4992	6233	4733	33.9	4886	6102	4659	-0.7	8	4	18	4930	6151	4684	11.3	21.6	9.0	69.4	171	70	2.5	468	0.017	0.2	2	7.3	0.0	0.00	11.0	1.1	1.7	2.1
43.57	6.12	58.36	65.31	15.17	21.79	51.23	100.00	18.30	12.13	3.78	151.88	20.76	12.0	0.0	NA	924	1016.8	208.9	178	178	45.32	NA	NA	0	34.0	4985	6222	4786	33.4	4862	6115	4696	-0.6	9	1	1	4871	6128	4687	11.4	21.7	10.1	68.2	173	70	2.5	490	0.018	0.4	2	7.2	0.0	0.00	11.5	2.2	1.8	2.0
43.12	7.48	64.47	72.41	13.82	17.71	54.45	100.00	18.72	12.95	3.04	147.11	20.75	11.5	0.0	1.56	933	988.9	210.0	177	178	49.36	11.6	11.5	0	32.4	4745	5999	4486	33.8	4758	6013	4522	1.4	1	1	1	4795	6057	4572	11.2	21.2	11.2	67.6	159	65	2.5	475	0.019	0.8	2	7.3	0.0	0.00	11.7	0.7	2.0	2.2
43.06	6.94	63.60	72.06	15.70	19.42	54.72	100.00	18.85	13.13	3.85	154.20	21.45	12.0	0.0	1.55	929	1010.9	211.7	178	178	48.68	10.2	11.3	0	33.6	4854	6105	4626	33.6	4766	6022	4552	0.0	2	2	2	4806	6059	4586	11.1	21.2	10.9	67.9	161	65	2.5	478	0.019	1.0	2	7.3	0.0	0.00	11.4	0.8	2.0	2.2
41.49	6.94	63.60	72.06	15.70	19.42	54.72	100.00	18.85	13.13	3.85	154.20	21.45	12.0	0.0	1.56	928	1003.5	208.7	177	177	47.20	9.7	11.1	0	33.9	4893	6144	4658	33.9	4769	6033	4556	0.0	3	3	3	4842	6103	4609	11.3	21.5	10.5	68.0	160	65	2.5	491	0.019	1.2	3	7.4	0.0	0.00	11.4	0.9	1.9	2.1
42.45	6.94	63.60	72.06	15.70	19.42	54.72	100.00	18.85	13.13	3.85	154.20	21.45	12.0	0.0	1.55	938	1003.8	209.8	177	177	47.11	10.1	10.2	0	34.3	4846	6077	4614	35.3	4840	6091	4614	1.0	4	1	4	4893	6135	4650	11.4	21.7	9.8	68.5	164	66	2.5	488	0.019	1.8	3	7.1	0.0	0.00	11.3	0.8	1.9	2.4
42.04	7.17	61.23	70.01	13.36	18.67	52.83	100.00	17.88	12.62	2.90	143.28	20.21	10.3	0.0	1.55	932	983.1	209.4	177	177	46.24	9.0	9.5	0	35.8	4944	6156	4690	35.8	4900	6126	4665	0.0	6	3	6	4925	6161	4687	11.5	21.9	9.4	68.7	166	67	2.5	493	0.019	1.5	2	7.0	0.0	0.00	11.0	1.0	1.9	1.8
42.68	7.17	61.23	70.01	13.36	18.67	52.83	100.00	17.88	12.62	2.90	143.28	20.21	10.3	0.0	1.55	930	992.0	209.4	178	178	46.10	8.8	9.7	0	35.6	4959	6178	4708	35.2	4878	6134	4673	-0.4	7	4	7	4924	6161	4692	11.5	22.0	9.4	68.6	169	67	2.5	498	0.018	0.3	3	7.0	0.0	0.00	11.2	0.8	2.0	1.8
43.44	7.17	61.23	70.01	13.36	18.67	52.83	100.00	17.88	12.62	2.90	143.28	20.21	10.3	0.0	1.55	934	1004.1	207.8	177	177	47.53	9.3	10.4	0	35.1	4917	6158	4704	35.1	4835	6090	4651	0.0	8	1	8	4888	6129	4653	11.4	21.7	9.9	68.4	166	68	2.4	490	0.018	1.1	3	7.1	0.0	0.00	11.1	0.8	1.9	2.4
40.28	7.63	60.51	69.24	17.59	20.67	52.83	100.00	18.74	13.21	4.94	158.42	21.77	11.1	0.0	1.59	934	1036.8	209.1	178	178	45.28	9.6	9.8	0	34.9	4882	6108	4655	35.8	4858	6070	4628	0.9	10	2	2	4911	6124	4684	11.1	21.5	9.4	69.1	161	66	2.4	490	0.019	0.6	3	7.4	0.0	0.00	11.7	0.6	1.7	1.9
41.50	6.23	62.93	69.74	11.80	20.54	54.57	100.00	18.89	12.82	2.30	152.83	22.18	11.3	0.0	NA	930	1120.7	207.8	177	177	46.44	9.4	10.2	0	35.3	4918	6156	4690	35.3	4849	6093	4646	0.0	11	3	15	4874	6125	4659	11.3	21.6	9.9	68.5	160	64	2.5	490	0.019	1.6	3	7.2	0.0	0.00	11.6	0.8	1.9	2.0
41.21	7.13	60.30	68.18	13.80	20.72	52.49	100.00	18.68	12.75	3.25	152.82	21.35	11.1	0.0	NA	928	1073.6	207.1	177	177	45.40	9.6	10.2	0	35.4	4805	6124	4666	35.5	4842	6091	4641	0.1	12	4	16	4848	6095	4630	11.1	21.2	10.3	68.5	166	66	2.5	493	0.019	1.2	3	7.3	0.1	0.20	11.6	1.0	1.9	2.5
40.89	7.85	58.22	66.95	15.38	20.86	50.84	100.00	18.51	12.70	4.01	152.82	20.70	11.1	0.0	NA	928	1027.5	205.9	177	177	45.54	9.0	10.0	0	35.8	4942	6177	4725	35.8	4858	6100	4656	0.0	1	1	1	4860	6100	4659	11.1	21.3	10.0	68.7	167	68	2.5	495	0.019	1.3	3	7.4	0.0	0.00	11.5	1.1	1.9	2.3
40.14	7.64	59.44	67.22	15.67	21.50	52.02	100.00	18.72	12.86	4.16	156.51	21.47	12.4	0.0	NA	930	1021.4	208.4	177	177	45.73	9.5	10.6	0	35.5	4899	6123	4692	35.5	4811	6041	4636	0.0	2	1	2	4815	6055	4631	10.8	20.9	10.6	69.0	157	64	2.5	497	0.020	1.1	3	7.4	0.0	0.00	11.7	1.7	1.8	2.2
39.30	7.51	59.74	67.28	15.72	21.80	52.30	100.00	18.81	12.98	4.24	158.36	21.82	12.7	0.0	NA	929	1092.2	207.5	178	178	44.46	8.9	9.8	0	35.5	4957	6181	4751	35.5	4887	6124	4710	0.0	3	2	3	4876	6109	4696	11.1	21.4	9.8	68.8	156	64	2.4	514	0.021	0.7	3	7.3	0.0	0.00	11.6	1.8	1.7	2.3
39.53	7.51	59.74	67.28	15.72	21.80	52.30	100.00	18.81	12.98	4.24	158.36	21.82	12.7	0.0	NA	929	1175.3	204.1	177	177	45.02	8.9	9.9	0	35.6	4961	6180	4775	35.6	4869	6094	4690	0.0	4	3	4	4850	6094	4665	11.0	21.2	10.1	68.7	155	62	2.5	490	0.020	1.2	2	7.3	0.0	0.00	11.4	1.5	1.5	2.0
40.22	7.51	59.74	67.28	15.72	21.80	52.30	100.00	18.81	12.98	4.24	158.36	21.82	12.7	0.0	1.56	925	1102.8	206.6	178	178	45.22	9.0	10.0	0	35.2	4949	6161	4766	35.2	4871	6108	4705	0.0	5	4	5	4879	6117	4696	11.1	21.5	9.9	68.7	157	63	2.5	495	0.020	0.9	3	7.3	0.1	0.20	11.4	2.0	1.6	2.0
41.18	7.08	61.83	70.69	13.43	17.72	53.27	100.00	19.14	13.38	3.02	153.10	21.90	10.9	0.0	NA	936	1024.4	218.7	178	178	47.04	NA	NA	0	33.2	4791	6026	4482	35.7	0	6111	0	2.5	6	2	16	4918	6152	4618	11.4	21.1	8.8	70.1	162	68	2.4	479	0.018	0.9	2	7.4	0.0	0.00	11.9	0.8	1.9	2.1
40.70	6.58	58.38	67.17	12.22	18.46	51.45	100.00	18.22	12.83	2.69	148.49	21.23	11.1	0.0	NA	937	997.7	209.6	177	177	46.43	NA	NA	0	32.9	NA	6002	0	36.5	4902	6120	4621	3.6	7	3	17	4906	6134	4626	11.2	21.0	8.9	70.1	160	66	2.4	492	0.019	1.5	3	7.4	0.0	0.00	11.6	0.8	1.9	2.4
41.89	6.27	56.23	64.98	11.47	18.93	50.31	100.00	17.64	12.48	2.49	145.61	20.81	11.3	0.0	NA	940	1031.3	215.1	177	177	47.09	NA	NA	0	33.6	4864	6085	4615	35.1	4847	6072	4607	1.5	1	1	1	4875	6095	4606	11.2	20.9	9.5	69.7	161	65	2.5	468	0.018	1.3	2	7.2	0.0	0.00	11.4	1.1	2.0	1.8
43.38	8.17	63.66	73.44	18.37	23.76	56.64	100.00	17.94	12.18	4.15	151.54	20.27	9.0	0.0	1.54	934	1007.3	209.4	178	178	47.28	11.1	9.0	0	33.3	4750	5994	4517	36.3	4887	6146	4653	3.0	5	5	12	4907	6150	4631	11.5	21.3	9.2	69.5	167	66	2.5	505	0.019	1.0	2	7.2	0.0	0.00	11.5	0.7	1.7	2.4
36.83	6.60	55.74	66.25	11.83	22.52	48.95	100.00	16.67	12.11	2.75	140.84	19.07	11.6	0.0	1.52	930	950.7	203.6	178	178	38.89	9.2	7.6	0	37.3	4840	5988	4577	40.0	4971	6114	4657	2.7	1	1	6	4990	6160	4693	11.0	21.0	7.5	71.5	161	65	2.5	500	0.019	0.9	3	6.9	0.0	0.00	11.8	0.8	1.7	2.2
35.25	6.90	54.26	60.99	12.22	20.16	47.23	100.00	16.57	11.73	3.06	139.52	18.62	9.2	0.0	1.53	926	955.8	203.0	177	178	39.02	8.4	7.5	0	38.0	4894	6022	4590	40.0	4971	6107	4675	2.0	2	2	7	4966	6112	4660	10.7	20.6	7.6	71.9	159	65	2.4	492	0.019	0.9	3	7.1	0.0	0.00	11.7	0.7	1.7	2.0
36.12	6.86	55.66	63.43	12.53	20.39	48.94	100.00	16.71	11.94	3.07	142.45	19.12	9.2	0.0	1.54	922	975.8	203.6	178	178	40.46	7.8	7.5	0	38.1	4942	6081	4645	39.3	4967	6117	4664	1.2	3	3	8	4961	6112	4649	10.7	20.5	7.6	71.8	157	64	2.5	522	0.021	0.9	3	7.4	0.0	0.00	11.6	1.3	1.7	2.2
38.52	6.67	63.44	76.94	14.28	21.67	58.42	100.00	17.47	13.12	3.10	158.66	21.88	9.2	0.0	1.52	924	986.9	206.4	177	177	42.67	7.5	7.7	0	38.6	5055	6213	4714	37.7	4960	6123	4639	-0.9	4	4	9	4966	6137	4641	10.8	20.7	7.6	71.7	158	63	2.5	499	0.020	1.9	3	7.3	0.1	0.20	11.7	0.8	1.7	2.2
38.35	6.53	61.68	77.15	11.81	21.12	59.38	100.00	17.31	12.83	2.14	154.56	22.18	10.4	0.0	1.52	921	1001.1	205.5	178	178	44.95	9.4	9.0	0	35.3	4831	6017	4556	35.3	4866	6028	4573	0.0	6	4	4	4863	6039	4564	10.6	20.1	9.0	70.9	157	64	2.5	504	0.020	0.7	3	6.8	0.0	0.00	11.6	0.5	1.9	2.2
39.98	6.89	61.54	76.07	12.41	21.14	57.89	100.00	17.42	12.79	2.35	153.00	21.58	10.3	0.0	1.54	928	1006.7	206.2	177	177	46.27	9.0	9.1	0	35.4	4872	6050	4585	35.0	4857	6036	4579	-0.4	7	1	5	4872	6060	4583	10.7	20.4	9.1	70.6	160	63	2.5	486	0.019	1.6	2	7.1	0.0	0.00	11.7	0.6	2.0	2.4
41.87	7.13	61.39	74.10	13.27	20.96	55.95	100.00	17.58	12.63	2.61	151.00	20.80	11.2	0.0	1.53	926	1024.0	208.2	178	178	47.08	10.7	9.5	0	33.5	4756	5972	4515	34.0	4854	6069	4600	0.5	8	2	6	4842	6057	4574	10.9	20.4	9.7	69.9	163	66	2.5	486	0.019	0.7	2	7.1	0.0	0.00	11.6	0.6	2.0	2.1
43.62	6.84	61.46	73.91	13.18	20.77	56.39	100.00	17.56	12.55	2.53	151.48	20.99	11.6	0.0	1.52	923	1041.2	207.3	177	177	48.77	10.1	10.5	0	33.4	4815	6033	4569	33.4	4773	5991	4521	0.0	9	3	7	4820	6042	4563	10.9	20.4	10.0	69.6	161	64	2.5	476	0.018	1.6	2	7.2	0.1	0.10	11.6	0.7	1.9	2.4
38.60	5.17	61.17	76.39	12.51	20.32	58.17	100.00	17.79	13.05	2.50	157.45	22.21	10.6	0.0	1.52	928	999.5	209.4	178	178	45.78	9.8	9.7	0	34.5	4811	5992	4578	34.5	4823	6010	4579	0.0	5	5	12	4829	6020	4590	10.6	20.1	9.7	70.3	156	61	2.5	496	0.020	1.1	2	7.5	0.1	0.10	11.6	0.5	1.8	2.3
39.65	7.01	61.30	72.71	13.23	24.85	54.89	100.00	17.38	12.47	2.76	149.46	20.42	11.6	0.0	1.53	925	1010.3	209.1	177	177	44.77	9.4	9.3	0	34.6	4852	6047	4852	34.4	4861	6073	4759	-0.2	3	3	16	4854	6061	4654	10.7	20.4	9.5	70.1	155	61	2.5	507	0.020	1.7	3	7.3	0.0	0.00	11.7	0.6	1.9	2.6
40.87	6.30	57.04	70.03	11.70	18.04	50.50	100.00	17.27	12.79	2.41	145.62	20.18	12.5	0.0	1.53	930	1002.4	207.5	177	177	46.65	9.7	10.0	4549	33.9	4844	6057	4562	33.1	4821	6033	4547	-0.8	1	1	13	4818	6041	4537	10.9	20.4	10.1	69.5	153	63	2.4	481	0.020	1.0	2	7.4	0.1	0.20	11.0	0.9	2.0	2.1
42.46	5.32	59.14	71.05	13.02	21.17	52.58	100.00	16.94	12.32	2.46	144.80	20.03	11.9	19.7	1.53	926	1019.6	206.4	178	178	46.17	9.4	10.2	4549	33.4	4854	6051	4595	32.2	4794	5992	4552	-1.2	3	2	2	4798	5982	4534	10.4	19.8	10.1	70.1	166	65	2.6	495	0.019	0.9	3	7.1	0.0	0.00	11.2	1.1	1.9	2.4
42.66	5.32	59.14	71.05	13.02	21.17	52.58	100.00	16.94	12.32	2.46	144.80	20.03	11.9	19.9	1.52	924	1008.6	208.7	177	177	46.38	9.7	10.0	4549	33.1	4825	6028	4571	32.6	4798	5986	4552	-0.5	4	3	3	4820	6006	4552	10.5	20.0	9.8	70.2	166	65	2.5	484	0.018	1.7	2	7.1	0.0	0.00	11.1	1.2	1.8	2.4
42.23	5.32	59.14	71.05	13.02	21.17	52.58	100.00	16.94	12.32	2.46	144.80	20.03	11.9	19.3	1.54	926	1014.3	208.7	178	178	45.92	9.5	10.0	4549	33.0	4838	6029	4593	32.6	4800	5991	4562	-0.4	5	4	4	4820	6013	4554	10.6	20.1	9.8	70.1	166	68	2.4	506	0.019	0.0	2	7.1	0.1	0.20	11.0	1.5	1.8	2.4
41.43	5.71	57.68	69.37	12.26	21.32	50.79	100.00	17.01	12.44	2.46	143.94	19.78	11.0	19.5	1.52	924	1027.0	206.6	177	177	46.77	10.2	9.8	4549	32.8	4806	6002	4591	32.8	4831	6027	4603	0.0	7	2	6	4847	6053	4607	10.9	20.5	9.7	69.9	160	65	2.5	488	0.019	0.0	2	7.2	0.0	0.00	11.4	1.0	1.9	2.3
41.47	6.60	58.80	71.17	12.40	22.14	52.24	100.00	17.21	12.77	2.58	148.28	20.33	11.3	19.3	1.52	925	1015.0	205.9	178	178	46.69	9.8	10.0	4549	33.0	4828	6020	4587	32.6	4815	6005	4582	-0.4	8	3	7	4837	6050	4598	10.9	20.4	9.8	59.8	159	64	2.5	493	0.019	0.0	2	7.2	0.0	0.00	10.9	1.0	1.9	2.2
42.07	6.76	55.42	69.80	11.25	18.15	49.89	100.00	17.61	13.40	2.57	151.50	20.88	10.8	22.5	1.48	936	954.7	211.2	177	177	45.95	11.1	10.5	4549	32.8	4713	5904	4467	32.8	4750	5927	4497	0.0	1	1	1	4784	5974	4543	10.3	19.6	10.1	70.3	157	61	2.6	509	0.020	1.1	3	7.4	0.0	0.00	11.4	0.7	1.8	2.4
44.35	6.76	55.42	69.80	11.25	18.15	49.89	100.00	17.61	13.40	2.57	151.50	20.88	10.8	20.5	1.53	923	954.0	210.0	178	177	46.66	8.1	8.9	4549	33.3	4934	6110	4635	32.5	4873	6058	4598	-0.8	2	2	2	4900	6089	4617	10.8	20.5	8.6	70.8	163	64	2.5	498	0.019	0.9	3	7.3	0.1	0.10	11.4	0.7	2.0	2.4
44.16	6.76	55.42	69.80	11.25	18.15	49.89	100.00	17.61	13.40	2.57	151.50	20.88	10.8	21.5	1.55	930	969.3	208.7	177	178	47.33	9.3	9.3	4549	32.5	4836	6014	4564	32.3	4844	6032	4577	-0.2	3	3	3	4869	6061	4603	10.7	20.3	9.0	70.7	160	63	2.5	501	0.020	1.0	3	7.1	0.0	0.00	11.0	0.7	1.8	2.4
43.33	6.77	58.76	72.74	12.12	20.65	53.17	100.00	17.59	13.31	2.61	154.06	21.31	11.4	20.5	1.56	928	980.3	208.2	178	178	46.78	9.0	8.8	4549	32.8	4869	6061	4583	32.3	4905	6117	4635	-0.5	4	4	4	4885	6098	4611	11.0	20.6	8.9	70.4	162	64	2.5	490	0.019	0.3	2	7.1	0.0	0.00	10.5	0.9	1.8	2.1
42.61	6.95	60.31	73.97	12.42	19.05	52.31	100.00	17.64	13.28	2.75	151.21	20.75	12.2	20.5	1.50	926	986.4	208.7	177	177	46.48	8.5	9.2	4549	33.2	4914	6108	4594	32.4	4850	6025	4535	-0.8	5	5	5	4854	6034	4547	10.6	20.0	9.1	70.9	165	65	2.5	490	0.019	1.2	3	7.0	0.0	0.00	10.7	0.7	1.8	2.3
42.96	6.95	60.31	73.97	12.42	19.05	52.31	100.00	17.64	13.28	2.75	151.21	20.75	12.2	20.5	1.50	925	977.8	227.4	178	178	46.03	9.4	9.1	4549	32.6	4833	6009	4521	32.6	4853	6027	4538	0.0	6	6	6	4864	6056	4556	10.7	20.2	9.0	70.7	168	65	2.6	492	0.018	0.7	3	7.4	0.0	0.00	11.1	1.1	1.8	2.1
43.84	6.95	60.31	73.97	12.42	19.05	52.31	100.00	17.64	13.28	2.75	151.21	20.75	12.2	20.0	1.56	927	1006.4	210.7	177	177	48.11	8.9	9.9	4549	32.6	4883	6146	4533	31.3	4812	6082	4484	-1.3	7	1	7	4816	6086	4481	11.0	20.2	9.9	69.9	164	69	2.4	493	0.019	1.8	3	6.7	0.1	0.10	11.6	1.4	1.8	2.4
46.34	7.97	64.75	74.10	15.11	22.66	56.22	100.00	18.82	12.76	3.18	157.34	21.33	11.7	18.0	1.52	921	1002.4	209.8	177	178	47.45	9.5	9.6	4549	32.1	4855	6077	4563	31.5	4836	6055	4551	-0.6	9	3	9	4850	6099	4548	11.2	20.7	9.6	69.7	167	68	2.4	490	0.018	0.6	3	7.3	0.0	0.00	11.6	1.3	1.8	2.1
39.74	6.94	57.02	69.51	13.45	18.44	50.16	100.00	17.55	12.83	3.09	149.60	20.25	10.4	19.0	1.52	923	971.2	207.8	178	178	44.65	8.9	9.1	4549	33.9	4852	6013	4518	33.6	4849	6015	4525	-0.3	10	4	10	4866	6045	4531	10.6	20.0	8.8	71.2	170	68	2.5	490	0.018	0.8	3	7.3	0.0	0.00	11.6	0.9	1.8	2.0
41.12	6.94	57.02	69.51	13.45	18.44	50.16	100.00	17.55	12.83	3.09	149.60	20.25	10.4	18.0	1.53	918	977.6	204.8	177	177	46.47	8.9	9.4	4549	33.4	4869	6043	4547	32.8	4827	6014	4526	-0.6	11	5	11	4846	6032	4521	10.6	20.0	9.2	70.8	169	67	2.5	504	0.019	1.3	3	7.1	0.0	0.00	11.4	2.5	1.8	2.1
40.14	6.94	57.02	69.51	13.45	18.44	50.16	100.00	17.55	12.83	3.09	149.60	20.25	10.4	19.5	1.54	926	989.2	206.4	178	178	47.33	8.7	11.0	4549	33.8	4874	6049	4551	33.1	4701	5890	4392	-0.7	12	6	12	4721	5901	4416	10.0	18.9	10.8	70.2	162	65	2.5	496	0.019	0.4	3	7.1	0.1	0.20	11.4	1.0	1.8	2.2
42.69	7.56	61.62	72.17	14.46	21.02	53.78	100.00	18.33	12.82	3.18	154.71	20.95	11.1	19.5	1.54	929	989.6	205.5	177	177	46.34	9.0	9.5	4549	33.1	4867	6061	4567	32.5	4836	6033	4541	-0.6	1	1	13	4861	6071	4550	10.9	20.5	9.3	70.3	165	66	2.5	495	0.019	1.9	3	7.4	0.0	0.00	11.4	1.1	2.0	2.3
40.15	6.87	57.33	71.52	13.22	15.62	50.85	100.00	17.74	13.16	2.91	148.45	20.44	9.7	19.5	1.54	923	1003.8	206.8	178	177	48.84	8.7	10.8	4549	33.6	4867	6039	4539	33.1	4718	5898	4404	-0.5	2	2	14	4747	5929	4423	10.1	19.2	10.5	70.3	157	63	2.5	501	0.020	1.4	3	7.3	0.0	0.00	11.6	1.2	1.9	2.2
39.77	6.87	57.33	71.52	13.22	15.62	50.85	100.00	17.74	13.16	2.91	148.45	20.44	9.7	19.5	1.54	925	984.2	205.2	177	178	46.32	8.4	8.6	4549	33.7	4891	6059	4563	33.1	4880	6048	4555	-0.6	3	3	15	4890	6069	4552	10.7	20.3	8.6	71.2	164	67	2.5	483	0.018	0.9	3	7.2	0.0	0.00	11.2	0.9	1.9	2.1
39.40	6.87	57.33	71.52	13.22	15.62	50.85	100.00	17.74	13.16	2.91	148.45	20.44	9.7	19.5	1.52	924	992.5	207.3	178	178	47.03	8.2	9.4	4549	33.9	4901	6055	4571	34.8	4816	5992	4493	0.9	4	4	16	4847	6021	4518	10.5	19.9	9.1	71.0	160	66	2.4	509	0.020	0.6	3	7.3	0.0	0.00	11.0	0.9	2.1	2.2
39.14	6.65	55.61	68.93	12.72	15.91	48.64	100.00	17.87	13.31	2.98	146.08	20.33	10.7	19.5	1.50	924	987.0	206.2	177	177	45.66	10.3	9.4	4549	33.8	4747	5918	4447	34.5	4829	6006	4531	0.7	5	5	17	4853	6033	4545	10.6	20.1	9.2	70.7	156	63	2.5	463	0.019	1.1	2	7.3	0.1	0.20	11.0	1.0	1.9	2.2
40.36	6.65	55.61	68.93	12.72	15.91	48.64	100.00	17.87	13.31	2.98	146.08	20.33	10.7	19.5	1.52	926	991.5	211.4	178	178	47.09	10.3	9.8	4549	33.5	4765	5946	4471	33.9	4808	5991	4509	0.4	6	6	18	4820	6016	4524	10.6	20.0	9.7	70.3	156	64	2.4	488	0.020	0.5	3	7.3	0.0	0.00	11.3	0.0	2.0	2.2
42.31	6.65	55.61	68.93	12.72	15.91	48.64	100.00	17.87	13.31	2.98	146.08	20.33	10.7	18.0	1.58	920	1003.2	207.8	177	177	49.04	11.1	10.2	4549	32.5	4724	5926	4441	33.5	4792	6002	4506	1.0	7	1	1	4811	6016	4519	10.7	20.1	10.0	69.9	157	63	2.5	512	0.020	1.4	3	7.1	0.0	0.00	11.8	11.0	1.9	2.3
40.49	6.72	57.24	71.27	12.25	16.58	50.88	100.00	18.18	13.60	2.75	151.23	21.30	9.3	20.0	1.57	928	983.2	208.0	177	178	46.30	9.4	9.1	4549	34.6	4813	5957	4491	35.2	4834	5983	4514	0.6	9	3	3	4863	6019	4541	10.4	19.9	8.8	71.3	160	63	2.5	499	0.019	0.8	3	7.0	0.0	0.00	11.5	0.7	1.8	2.3
40.57	6.57	55.93	69.48	11.95	18.00	50.50	100.00	17.92	13.23	2.63	151.67	21.29	8.7	19.0	1.60	921	980.8	206.2	178	178	46.37	9.6	9.1	4549	34.6	4803	5958	4506	35.0	4846	6009	4535	0.4	10	4	4	4867	6041	4554	10.6	20.1	8.9	71.0	161	64	2.5	513	0.020	0.7	3	7.1	0.1	0.10	11.6	0.9	1.8	2.2
38.20	6.16	54.67	66.95	11.15	17.25	48.12	100.83	17.97	13.12	2.48	149.03	20.88	9.1	20.0	1.51	925	982.3	205.9	177	177	46.32	10.4	9.0	4549	34.0	4756	5917	4484	35.3	4858	6039	4557	1.3	11	5	5	4877	6051	4587	10.7	20.3	8.9	70.8	153	61	2.5	484	0.020	1.3	2	7.1	0.0	0.00	11.7	0.8	1.8	2.1
38.70	6.16	54.67	66.95	11.15	17.25	48.12	100.83	17.97	13.12	2.48	149.03	20.88	9.1	19.5	1.51	923	983.4	206.2	178	178	46.02	10.2	8.7	4549	34.1	4772	5957	4494	35.4	4885	6065	4571	1.3	12	6	6	4889	6069	4590	10.8	20.4	8.8	70.8	156	62	2.5	502	0.020	0.5	3	7.1	0.0	0.00	11.6	0.8	1.8	2.0
38.94	6.16	54.67	66.95	11.15	17.25	48.12	100.83	17.97	13.12	2.48	149.03	20.88	9.1	19.5	1.49	928	1004.5	206.8	177	177	48.17	9.6	10.2	4549	33.9	4828	6010	4550	33.9	4763	5949	4510	0.0	1	1	7	4814	6010	4545	10.4	19.8	9.7	70.5	150	60	2.5	492	0.021	1.1	2	7.1	0.0	0.00	11.5	0.7	1.9	2.3
41.90	6.37	52.67	64.34	12.02	17.40	46.52	100.00	17.38	12.48	2.75	144.50	19.82	11.0	20.0	1.50	927	1016.1	210.3	178	177	47.44	11.2	9.9	4549	32.7	4716	5937	4476	34.1	4818	6043	4586	1.4	2	2	8	4835	6074	4592	10.9	20.4	9.7	69.8	162	63	2.6	492	0.019	1.0	2	7.0	0.0	0.00	11.6	0.9	2.0	2.3
42.03	6.37	52.67	64.34	12.02	17.40	46.52	100.00	17.38	12.48	2.75	144.50	19.82	11.0	19.5	1.51	922	999.7	213.3	177	178	47.30	10.9	10.1	4549	33.3	4742	5963	4505	33.9	4800	6038	4578	0.6	3	3	9	4841	6084	4593	11.0	20.5	9.7	69.8	163	63	2.6	498	0.019	1.0	3	6.9	0.0	0.00	11.7	1.4	1.9	1.8
41.96	6.37	52.67	64.34	12.02	17.40	46.52	100.00	17.38	12.48	2.75	144.50	19.82	11.0	19.5	1.50	923	1013.9	209.6	178	178	48.11	11.0	10.4	4549	32.9	4737	5967	4499	33.5	4785	6025	4558	0.6	4	4	10	4819	6065	4575	10.9	20.4	10.0	69.6	160	62	2.6	504	0.020	0.5	3	6.8	0.0	0.00	11.7	1.4	1.9	1.8
41.85	6.31	54.42	66.13	11.97	18.40	48.01	100.00	17.81	12.75	2.76	147.17	20.41	12.0	19.5	1.48	923	994.0	206.8	178	178	46.00	9.7	9.6	4549	34.2	4818	6019	4583	34.2	4830	6031	4603	0.0	6	6	12	4857	6065	4613	10.7	20.3	9.3	70.4	162	64	2.5	494	0.019	0.4	3	7.0	0.0	0.00	11.2	1.2	1.9	1.8
39.71	6.27	53.51	66.52	11.83	16.95	46.62	100.00	17.76	13.31	2.87	143.65	20.35	10.0	20.0	1.49	929	960.0	210.7	177	177	45.05	9.4	9.0	0	34.8	4824	5974	4534	35.4	4862	6021	4567	0.6	1	1	13	4877	6036	4569	10.4	20.0	8.7	71.3	156	62	2.5	500	0.020	1.1	3	7.3	0.0	0.00	11.9	1.2	1.8	2.5
39.38	6.27	53.51	66.52	11.83	16.95	46.62	100.00	17.76	13.31	2.87	143.65	20.35	10.2	19.0	1.48	921	961.3	205.0	178	177	43.83	8.9	8.4	0	35.5	4862	6023	4568	35.5	4905	6066	4597	0.0	2	2	14	4901	6063	4588	10.6	20.2	8.4	71.4	159	63	2.5	487	0.019	0.9	3	7.1	0.1	0.10	11.9	1.7	1.8	2.3
39.16	6.27	53.51	66.52	11.83	16.95	46.62	100.00	17.76	13.31	2.87	143.65	20.35	10.2	19.0	1.48	921	969.7	207.8	177	178	43.86	8.4	8.3	0	35.5	4897	6043	4584	35.5	4907	6064	4596	0.0	3	3	15	4916	6075	4595	10.6	20.3	8.2	70.5	158	62	2.6	484	0.019	0.7	3	7.0	0.0	0.00	11.5	1.7	1.9	2.2
39.38	6.58	52.50	63.29	12.24	18.28	46.04	100.00	17.67	12.47	2.85	144.40	19.85	9.5	19.0	1.47	923	989.7	205.7	177	177	47.40	9.7	9.5	0	34.2	4836	6047	4588	33.9	4854	6072	4603	-0.3	5	5	17	4876	6106	4609	11.1	20.8	9.3	69.9	162	65	2.5	494	0.019	1.3	3	7.1	0.0	0.00	11.7	1.2	1.8	1.9
40.08	6.45	53.18	64.98	12.11	18.77	46.80	100.00	17.58	12.69	2.79	145.90	19.96	10.9	19.5	1.51	923	996.4	206.2	178	178	47.52	10.2	9.9	0	33.5	4810	6039	4568	33.5	4827	6057	4580	0.0	6	6	18	4825	6063	4582	11.0	20.5	9.9	69.6	160	63	2.6	500	0.020	0.6	2	7.0	0.0	0.00	11.5	1.2	1.8	1.8
39.17	6.39	58.85	73.45	12.69	16.92	51.25	100.00	17.92	13.50	2.62	151.25	20.89	11.3	21.5	1.55	935	976.4	204.8	177	177	45.88	8.5	9.1	0	34.8	4879	6044	4574	34.4	4841	6018	4565	-0.4	7	1	1	4849	6041	4557	10.3	19.6	8.9	71.6	158	64	2.5	495	0.020	1.1	3	7.4	0.1	0.10	11.7	0.8	2.0	2.3
38.37	6.39	58.85	73.45	12.69	16.92	51.25	100.00	17.92	13.50	2.62	151.25	20.89	11.3	22.2	1.54	933	971.6	208.0	178	177	45.52	8.2	8.7	0	35.2	4899	6059	4609	34.5	4867	6037	4583	-0.7	8	2	2	4883	6045	4583	10.2	19.6	8.3	72.1	156	62	2.5	497	0.020	1.9	2	7.1	0.0	0.00	11.8	0.8	2.0	2.1
38.76	6.39	58.85	73.45	12.69	16.92	51.25	100.00	17.92	13.50	2.62	151.25	20.89	11.3	22.0	1.54	933	974.0	205.9	177	178	45.11	8.7	8.6	0	35.0	4857	6028	4569	34.6	4870	6051	4582	-0.4	9	3	3	4890	6058	4587	10.3	19.7	8.3	72.0	159	64	2.5	497	0.020	0.7	2	7.0	0.0	0.00	11.6	1.3	1.9	2.4
38.73	6.35	56.93	70.87	12.27	18.06	49.92	100.00	17.76	13.29	2.61	150.06	20.65	11.5	22.5	1.55	932	978.0	209.4	178	178	45.57	9.8	9.3	0	34.2	4795	5991	4520	34.2	4844	6031	4579	0.0	10	4	4	4859	6058	4581	10.5	19.8	8.9	71.2	157	61	2.6	507	0.020	0.8	3	7.1	0.0	0.00	11.6	0.6	2.0	2.3
38.95	6.17	53.80	65.53	12.70	18.65	47.67	100.00	16.82	12.11	2.58	142.66	19.64	11.4	21.5	1.55	932	980.2	205.7	177	177	44.92	8.7	9.1	0	35.1	4889	6089	4619	34.5	4857	6063	4594	-0.6	11	5	5	4876	6087	4601	10.6	20.0	8.8	71.2	163	65	2.5	498	0.019	1.7	3	7.0	0.0	0.00	11.4	1.3	1.9	2.2
40.41	6.97	58.18	71.85	14.25	17.08	50.06	100.00	18.02	13.51	3.37	148.92	20.33	11.0	21.5	1.55	937	993.4	205.5	177	177	46.11	9.3	9.4	0	34.5	4835	6014	4527	34.5	4827	5994	4525	0.0	1	1	7	4835	6015	4537	10.3	19.5	9.1	71.3	156	63	2.5	494	0.020	1.0	3	7.1	0.0	0.00	11.4	0.8	2.0	2.5
39.90	6.97	58.18	71.85	14.25	17.08	50.06	100.00	18.02	13.51	3.37	148.92	20.33	11.0	22.0	1.52	931	990.5	207.1	178	177	46.12	8.4	9.3	0	35.2	4899	6061	4572	33.8	4813	5977	4515	-1.4	2	2	8	4834	6015	4526	10.3	19.5	9.1	71.4	154	61	2.5	517	0.021	1.2	3	7.1	0.0	0.00	11.3	0.9	2.1	2.2
39.79	6.97	58.18	71.85	14.25	17.08	50.06	100.00	18.02	13.51	3.37	148.92	20.33	11.0	22.0	1.53	935	982.9	206.6	177	178	44.55	8.2	8.4	0	35.2	4897	6048	4572	34.8	4881	6059	4584	-0.4	3	3	9	4898	6073	4573	10.5	19.9	8.3	71.8	159	65	2.4	495	0.019	0.3	2	7.1	0.0	0.00	11.3	1.0	1.9	2.1
41.25	8.81	63.99	78.25	23.09	19.96	53.87	100.00	18.84	14.08	6.87	158.73	20.57	12.7	22.0	1.55	934	1000.7	205.5	178	178	45.10	8.6	8.8	0	34.9	4844	5990	4524	34.9	4825	5981	4517	0.0	4	4	10	4878	6049	4546	10.3	19.6	8.4	72.1	167	65	2.6	507	0.019	0.2	3	7.1	0.0	0.00	11.6	1.2	1.9	2.2
41.00	6.80	57.21	70.34	13.98	16.81	48.46	100.00	17.91	13.34	3.37	146.67	19.95	11.2	20.5	1.55	934	1000.8	205.2	178	178	44.99	9.1	8.5	0	34.7	4829	5998	4550	34.7	4879	6040	4573	0.0	6	6	12	4881	6060	4576	10.5	19.9	8.6	71.5	164	65	2.5	492	0.019	0.3	3	7.1	0.0	0.00	11.5	1.6	1.9	2.4
41.59	6.80	57.21	70.34	13.98	16.81	48.46	100.00	17.91	13.34	3.37	146.67	19.95	11.2	21.0	1.55	939	1171.2	206.4	177	178	45.64	8.9	9.1	0	35.1	4853	6022	4582	34.8	4826	5996	4564	-0.3	1	1	13	4866	6051	4567	10.4	19.7	8.7	71.6	164	66	2.5	506	0.019	0.9	2	7.2	0.0	0.00	11.5	1.5	1.9	2.4
40.91	6.80	57.21	70.34	13.98	16.81	48.46	100.00	17.91	13.34	3.37	146.67	19.95	11.2	22.0	1.54	936	979.2	208.7	178	178	44.62	8.6	8.4	0	35.0	4864	6030	4577	34.8	4887	6068	4599	-0.2	2	2	14	4907	6087	4602	10.5	20.0	8.2	71.8	165	68	2.4	493	0.019	0.9	2	7.1	0.0	0.00	11.6	1.6	2.0	2.4
38.99	6.30	51.96	64.07	12.65	19.23	44.66	100.00	17.16	12.71	3.15	142.11	19.13	11.6	21.0	1.55	932	994.5	207.8	177	177	45.45	9.4	9.4	0	34.8	4841	6048	4631	34.1	4839	6062	4631	-0.7	3	3	15	4867	6082	4642	10.7	20.2	9.1	70.7	160	64	2.5	499	0.019	1.4	3	7.2	0.0	0.00	11.5	1.0	1.9	2.4
38.81	6.30	51.96	64.07	12.65	19.23	44.66	100.00	17.16	12.71	3.15	142.11	19.13	11.6	21.5	1.55	937	989.2	205.2	178	178	45.52	9.3	9.5	0	34.5	4841	6052	4635	33.9	4830	6038	4618	-0.6	4	4	16	4844	6067	4632	10.7	20.1	9.4	70.5	159	64	2.5	513	0.020	0.9	2	7.1	0.1	0.10	11.7	1.1	1.8	2.1
39.30	6.30	51.96	64.07	12.65	19.23	44.66	100.00	17.16	12.71	3.15	142.11	19.13	11.6	21.5	1.55	939	983.7	206.4	177	177	46.10	9.4	9.6	0	34.6	4832	6035	4624	34.3	4822	6039	4617	-0.3	5	5	17	4838	6053	4622	10.6	20.0	9.4	70.6	159	62	2.5	498	0.020	1.2	2	7.2	0.0	0.00	11.7	1.1	1.9	2.3
40.77	6.45	55.09	69.18	11.98	17.22	47.12	100.00	17.74	13.47	2.86	148.00	20.04	10.8	21.5	1.54	937	960.4	204.8	177	178	43.52	8.1	8.3	0	35.8	4893	6028	4564	35.5	4879	6026	4556	-0.3	7	2	2	4873	6023	4546	10.2	19.5	8.4	72.1	163	65	2.5	486	0.019	0.8	2	7.2	0.0	0.00	11.5	1.1	1.9	2.3
39.27	6.45	55.09	69.18	11.98	17.22	47.12	100.00	17.74	13.47	2.86	148.00	20.04	10.8	21.5	1.55	935	954.8	NA	178	178	43.52	8.4	8.3	0	35.6	4866	6005	4550	35.6	4893	6046	4569	0.0	8	3	3	4897	6047	4572	10.3	19.7	8.1	72.2	157	64	2.5	493	0.020	1.2	2	7.2	0.0	0.00	11.3	0.8	1.9	2.1
40.06	6.45	55.09	69.18	11.98	17.22	47.12	100.00	17.74	13.47	2.86	148.00	20.04	10.8	21.7	1.55	938	975.8	205.2	177	177	44.11	8.4	8.3	0	35.4	4874	6012	4561	35.3	4885	6046	4571	-0.1	9	4	4	4885	6042	4546	10.3	19.7	8.3	72.0	158	63	2.5	478	0.019	1.0	2	7.2	0.1	0.10	11.6	0.9	1.9	2.5
39.17	6.28	54.33	68.30	12.37	15.46	46.72	100.00	17.41	13.28	2.96	143.47	19.85	12.4	22.0	1.54	941	1009.1	206.4	178	178	46.33	8.7	9.2	0	34.2	4867	6034	4588	33.8	4835	6015	4569	-0.4	10	5	5	4846	6019	4572	10.3	19.6	9.1	71.3	156	62	2.5	522	0.021	0.8	3	7.2	0.0	0.00	11.6	1.1	1.8	2.2
39.98	7.33	56.87	70.17	14.20	20.16	49.06	100.00	18.01	13.34	3.62	147.63	19.77	9.9	21.5	1.54	933	992.6	205.9	177	177	45.69	8.4	9.1	0	35.3	4896	6062	4614	34.5	4844	6021	4583	-0.8	11	6	6	4870	6054	4595	10.5	19.9	8.8	71.3	154	60	2.5	492	0.020	1.1	2	7.2	0.0	0.00	11.5	0.9	1.8	2.0
39.91	7.22	57.32	71.02	14.05	19.16	49.21	100.00	18.02	13.43	3.52	148.07	19.87	10.0	21.5	1.55	937	1003.7	205.0	178	178	44.22	8.6	8.9	0	35.6	4848	6005	4559	35.6	4866	6028	4579	0.0	1	1	7	4887	6056	4575	10.4	19.8	8.4	71.8	158	63	2.5	491	0.019	0.9	2	7.2	0.0	0.00	11.9	0.9	1.9	2.5
40.77	6.19	53.59	66.40	11.99	18.73	47.41	100.00	17.04	12.58	2.45	146.20	19.83	9.1	21.5	1.53	931	1006.1	206.8	178	177	45.49	9.1	8.9	0	35.2	4839	6020	4559	35.2	4849	6024	4576	0.0	3	3	9	4872	6054	4566	10.5	19.9	8.8	71.4	164	66	2.5	490	0.019	1.0	2	7.2	0.1	0.10	11.8	0.8	1.9	2.3
39.86	5.98	54.10	66.50	11.73	18.80	47.98	100.00	17.35	12.75	2.51	147.95	20.25	10.6	22.0	1.53	934	1001.0	204.8	177	177	44.09	8.8	8.7	0	35.3	4853	6021	4566	35.3	4864	6043	4582	0.0	4	4	10	4889	6070	4590	10.5	20.0	8.5	71.6	160	64	2.5	482	0.019	1.0	2	7.2	0.0	0.00	11.6	0.8	1.9	2.2
40.03	5.98	54.10	66.50	11.73	18.80	47.98	100.00	17.35	12.75	2.51	147.95	20.25	10.6	22.0	1.54	937	994.9	205.0	178	178	44.56	9.3	9.0	0	35.2	4815	5991	4541	35.2	4834	6008	4555	0.0	5	5	11	4862	6040	4562	10.4	19.7	8.8	71.5	159	64	2.5	499	0.020	0.7	2	7.2	0.0	0.00	11.6	0.9	2.0	2.1
40.81	5.98	54.10	66.50	11.73	18.80	47.98	100.00	17.35	12.75	2.51	147.95	20.25	10.6	20.9	1.55	933	1001.3	205.2	177	177	45.14	9.1	NA	0	35.1	4834	6022	4560	34.8	4859	6030	4568	-0.3	6	6	12	4869	6050	4574	10.5	19.9	8.8	71.3	160	64	2.5	493	0.019	0.9	2	7.3	0.0	0.00	11.5	0.9	1.9	2.1
37.94	5.85	51.75	64.02	10.41	20.40	44.30	100.00	16.96	12.68	2.34	143.33	19.63	11.5	22.0	1.54	934	1004.3	205.0	177	178	43.73	11.4	8.7	0	34.3	4701	5914	4579	35.7	4911	6124	4705	1.4	2	2	14	4923	6137	4710	11.0	20.7	8.6	70.7	154	63	2.4	490	0.020	0.8	3	7.2	0.1	0.10	11.7	1.1	1.9	2.2
37.73	5.85	51.75	64.02	10.41	20.40	44.30	100.00	16.96	12.68	2.34	143.33	19.63	11.5	21.0	1.55	934	1008.1	204.6	178	177	44.35	8.7	9.2	0	35.3	4895	6079	4670	34.3	4850	6022	4634	-1.0	3	3	15	4855	6046	4644	10.5	19.9	9.2	70.9	151	60	2.5	491	0.020	1.3	2	7.2	0.0	0.00	11.5	0.9	1.9	2.3
37.30	5.85	51.75	64.02	10.41	20.40	44.30	100.00	16.96	12.68	2.34	143.33	19.63	11.5	21.5	1.53	937	1004.8	206.6	177	177	44.13	8.9	9.3	0	35.3	4865	6037	4651	34.8	4839	6003	4619	-0.5	4	4	16	4856	6041	4634	10.4	19.9	9.1	71.1	150	60	2.5	486	0.020	1.6	2	7.2	0.0	0.00	11.5	0.8	1.9	2.3
37.86	6.01	51.83	63.80	11.22	19.69	44.57	100.00	17.09	12.57	2.55	143.07	19.47	11.8	21.9	1.54	937	992.8	204.6	178	178	43.49	8.8	9.0	0	35.4	4864	6025	4607	35.0	4847	6024	4614	-0.4	5	5	17	4865	6041	4606	10.4	19.7	8.8	71.4	154	62	2.5	507	0.021	0.8	3	7.2	0.0	0.00	11.6	1.1	1.8	2.3
38.05	5.89	51.28	64.04	9.99	16.90	44.74	100.00	16.79	13.04	2.33	142.66	19.67	12.3	21.7	1.55	936	974.0	206.4	177	177	42.66	8.9	8.7	0	35.5	4838	5997	4571	35.5	4855	6021	4584	0.0	6	6	18	4886	6051	4594	10.3	19.7	8.3	72.0	157	63	2.5	498	0.020	1.3	3	7.2	0.0	0.00	11.4	1.0	1.8	2.4
37.87	5.90	51.44	63.61	10.49	18.04	44.73	100.00	17.18	12.95	2.46	143.84	19.85	11.8	21.6	1.55	937	987.7	206.6	178	178	43.58	8.7	9.1	0	35.3	4861	6020	4611	34.9	4837	6007	4593	-0.4	7	1	1	4845	6019	4590	10.2	19.6	9.0	71.4	156	62	2.5	496	0.020	0.7	3	7.3	0.1	0.10	11.8	0.6	1.9	2.1
38.60	5.90	51.44	63.61	10.49	18.04	44.73	100.00	17.18	12.95	2.46	143.84	19.85	11.8	21.8	1.55	936	987.9	205.5	177	177	44.70	9.3	9.1	0	35.2	4816	5983	4579	35.2	4834	5997	4583	0.0	8	2	2	4841	6010	4583	10.2	19.5	9.1	71.4	155	62	2.5	481	0.019	1.0	2	6.9	0.0	0.00	11.7	0.8	1.9	2.0
38.44	5.90	51.44	63.61	10.49	18.04	44.73	100.00	17.18	12.95	2.46	143.84	19.85	11.8	20.8	1.55	933	986.6	205.5	178	178	44.23	8.7	9.0	0	35.2	4873	6031	4606	34.8	4846	6014	4595	-0.4	9	3	3	4853	6027	4605	0.0	19.7	9.0	71.4	156	62	2.5	493	0.020	0.5	3	7.1	0.0	0.00	11.9	0.8	1.9	2.3
39.42	5.79	53.96	66.53	10.40	18.26	47.57	100.00	17.24	12.99	2.16	146.64	20.57	9.8	22.0	1.54	932	992.3	208.4	177	177	45.61	9.0	8.9	0	35.1	4843	6016	4564	35.1	4850	6016	4569	0.0	10	4	4	4857	6036	4577	0.0	19.7	8.9	71.5	156	63	2.5	495	0.020	1.4	3	7.0	0.0	0.00	11.7	0.8	1.9	1.9
39.75	5.79	53.96	66.53	10.40	18.26	47.57	100.00	17.24	12.99	2.16	146.64	20.57	9.8	21.9	1.50	934	987.9	208.9	178	178	45.99	9.4	9.3	0	34.7	4807	5967	4536	34.7	4819	5997	4544	0.0	11	5	5	0	0	0	0.0	0.0	0.0	0.0	156	62	2.5	485	0.019	0.7	3	7.0	0.0	0.00	11.6	0.8	1.9	2.4
39.51	5.79	53.96	66.53	10.40	18.26	47.57	100.00	17.24	12.99	2.16	146.64	20.57	9.8	22.4	1.48	937	994.3	211.4	177	177	46.01	9.0	8.9	0	35.0	4839	6013	4561	35.0	4856	6032	4571	0.0	12	6	6	4859	6037	4590	0.0	19.7	8.8	71.5	155	62	2.5	491	0.020	1.5	2	7.1	0.0	0.10	11.7	0.8	1.9	1.9
38.35	5.94	51.27	63.54	10.48	17.72	44.43	100.00	17.13	13.00	2.52	143.19	19.69	11.7	22.0	1.52	940	980.3	205.2	178	178	43.75	9.6	9.4	0	35.1	4793	5953	4550	35.4	4808	5972	4551	0.3	1	1	7	4854	6013	4594	0.0	19.5	8.9	71.6	155	63	2.5	510	0.021	0.6	3	7.3	0.0	0.00	11.5	0.9	1.8	2.4
40.38	6.40	58.73	71.51	12.29	17.09	50.62	100.00	17.44	13.10	2.58	147.66	20.25	11.4	20.5	1.57	929	976.0	204.8	178	177	43.75	9.5	7.9	0	35.2	4783	5932	4485	35.7	4935	6105	4602	0.5	3	3	9	4912	6080	4572	0.0	19.9	8.0	72.1	162	65	2.5	494	0.019	0.7	2	7.0	0.0	0.00	11.6	1.7	1.8	2.2
40.19	6.40	58.73	71.51	12.29	17.09	50.62	100.00	17.44	13.10	2.58	147.66	20.25	11.4	22.2	1.54	937	994.1	208.9	177	177	44.64	7.8	8.7	0	35.9	4937	6098	4602	34.7	4845	6014	4533	-1.2	4	4	10	4859	6011	4535	0.0	19.3	8.6	72.1	158	63	2.5	488	0.019	0.7	3	7.0	0.0	0.00	11.3	1.2	1.9	2.3
39.96	6.40	58.73	71.51	12.29	17.09	50.62	100.00	17.44	13.10	2.58	147.66	20.25	11.4	22.3	1.53	936	983.2	213.7	178	178	43.84	8.1	8.7	0	35.7	4896	6050	4559	35.0	4849	6010	4536	-0.7	5	5	11	4871	6027	4544	0.0	19.5	8.4	72.1	160	64	2.5	497	0.019	0.9	2	7.0	0.1	0.10	11.3	0.9	1.8	2.3
39.79	6.10	56.36	69.52	11.71	16.35	49.10	100.00	17.15	12.99	2.44	146.29	20.16	11.7	22.0	1.51	935	981.6	204.3	177	177	43.37	7.8	8.1	0	36.2	4936	6076	4602	35.3	4887	6033	4567	-0.9	6	6	12	4884	6037	4563	0.0	19.5	8.2	72.3	161	65	2.5	500	0.019	0.7	3	7.1	0.0	0.00	11.4	1.1	1.8	2.3
41.86	5.78	53.70	67.22	11.17	15.44	47.33	100.00	16.76	12.81	2.30	144.34	19.95	12.0	21.2	1.55	942	992.0	204.1	177	177	44.94	9.1	9.0	0	34.7	4808	5941	4504	35.1	4822	5959	4526	0.4	7	1	13	4848	5976	4524	0.0	19.2	8.7	72.1	163	65	2.5	492	0.019	1.3	3	7.1	0.0	0.00	11.8	0.5	1.9	2.3
42.15	6.15	53.06	70.29	11.87	16.59	49.97	100.00	16.78	12.63	2.31	144.75	19.94	12.3	21.1	1.54	934	980.0	206.4	178	178	43.12	8.3	8.3	0	35.2	4880	6031	4567	35.2	4877	6035	4570	0.0	8	2	14	4902	6060	4566	0.0	19.9	8.1	72.0	167	68	2.5	501	0.019	0.4	3	7.2	0.0	0.00	11.8	0.6	1.9	2.2
43.88	6.40	52.62	72.39	12.34	17.38	51.77	100.00	16.80	12.51	2.32	145.03	19.93	12.5	21.0	1.55	938	997.8	205.7	177	177	45.23	8.5	9.1	0	34.9	4879	6047	4564	34.4	4827	6000	4535	-0.5	1	1	15	4858	6029	4546	0.0	19.6	8.8	71.6	163	65	2.5	495	0.019	1.0	3	6.9	0.0	0.00	11.8	0.7	1.9	2.2
39.58	5.60	51.45	63.44	10.50	19.17	45.06	100.00	16.83	12.53	2.27	142.90	19.64	11.4	21.0	1.55	934	991.9	205.0	178	177	44.10	8.8	9.2	0	35.1	4869	6037	4622	34.7	4835	5999	4585	-0.4	2	2	16	4857	6032	4598	0.0	19.7	9.0	71.3	153	61	2.5	500	0.020	0.9	3	7.0	0.1	0.10	11.7	0.8	1.9	2.0
40.19	5.60	51.45	63.44	10.50	19.17	45.06	100.00	16.83	12.53	2.27	142.90	19.64	11.4	20.9	1.54	934	986.1	204.1	177	178	43.92	9.0	9.1	0	35.1	4849	6011	4584	35.0	4844	6013	4589	-0.1	3	3	17	4863	6036	4608	0.0	19.7	8.9	71.4	156	61	2.6	501	0.020	0.7	2	7.1	0.0	0.00	11.6	0.8	1.7	2.4
39.84	5.60	51.45	63.44	10.50	19.17	45.06	100.00	16.83	12.53	2.27	142.90	19.64	11.4	21.1	1.55	935	923.0	204.8	178	178	43.26	8.7	8.6	0	35.6	4876	6040	4598	35.2	4880	6037	4611	-0.4	4	4	18	4873	6043	4601	0.0	19.8	8.7	71.5	157	61	2.6	497	0.020	0.5	3	6.9	0.0	0.00	11.9	0.5	1.9	2.2
40.59	5.43	49.10	62.08	10.11	19.09	44.45	100.00	16.54	12.21	2.11	141.49	19.46	12.0	21.2	1.55	935	1003.2	206.8	177	177	45.63	9.0	9.6	0	34.7	4872	6060	4633	34.1	4828	6015	4592	-0.6	5	5	19	4844	6055	4617	0.0	20.0	9.4	70.6	149	59	2.6	486	0.020	1.1	3	6.9	0.0	0.00	11.6	0.5	1.7	1.9
40.66	6.02	50.01	60.82	11.79	19.43	43.24	100.00	17.54	12.61	3.08	142.33	19.17	12.3	21.5	1.55	939	995.2	205.7	178	178	46.51	9.8	10.1	0	33.9	4839	6053	4654	33.1	4806	6022	4623	-0.8	6	6	20	4825	6053	4657	0.0	20.2	10.0	69.9	146	56	2.6	499	0.021	0.5	2	7.0	0.0	0.00	11.8	0.5	1.8	2.1
42.58	6.03	52.58	65.05	11.45	19.11	46.06	100.00	17.23	12.84	2.67	145.08	19.74	11.3	21.2	1.55	936	991.8	208.2	177	178	45.85	8.8	9.7	0	34.9	4882	6056	4600	33.9	4814	6004	4580	-1.0	7	1	1	4820	6022	4585	0.0	19.8	9.6	70.6	159	64	2.5	489	0.019	0.9	2	7.1	0.1	0.20	11.5	0.5	1.9	2.2
43.42	6.08	52.89	66.72	11.23	19.06	47.29	100.00	17.03	12.96	2.49	147.33	20.04	11.2	20.8	1.55	935	994.7	207.3	178	178	46.95	8.9	9.8	0	34.3	4878	6072	4614	33.2	4811	6018	4567	-1.1	8	2	2	4824	6045	4578	0.0	20.0	9.7	70.3	160	64	2.6	481	0.019	0.7	2	7.1	0.0	0.00	11.6	0.7	1.8	2.2
41.45	5.76	52.73	63.88	10.59	20.49	47.29	100.00	17.70	12.48	2.23	150.39	20.62	12.1	20.9	1.54	934	996.9	207.3	177	178	47.21	10.0	10.2	0	33.7	4812	6032	4626	33.2	4797	6019	4611	-0.5	9	3	3	4809	6057	4623	0.0	20.2	10.1	69.7	151	58	2.5	498	0.021	1.1	3	7.1	0.0	0.00	11.5	0.6	1.9	2.2
41.31	5.79	52.70	64.33	10.70	20.09	47.21	100.00	17.54	12.55	2.27	149.52	20.48	11.9	21.2	1.55	935	996.5	209.1	178	178	46.23	9.3	10.0	0	34.4	4859	6063	4635	33.3	4804	6024	4601	-1.1	10	4	4	4820	6050	4606	0.0	20.1	9.9	70.1	154	60	2.5	505	0.020	0.7	3	7.0	0.0	0.00	11.5	0.7	1.9	2.1
42.28	6.23	52.95	66.71	12.75	16.31	45.84	100.00	16.70	12.75	2.98	140.93	19.10	11.4	21.3	1.54	939	987.3	211.4	177	178	45.71	8.1	9.1	0	35.0	4905	6061	4595	33.8	4820	5982	4529	-1.2	11	5	5	4829	6009	4534	0.0	19.5	9.1	71.4	160	63	2.5	507	0.020	1.0	3	7.2	0.0	0.00	11.4	0.6	1.9	2.2
41.62	6.23	52.95	66.71	12.75	16.31	45.84	100.00	16.70	12.75	2.98	140.93	19.10	11.4	21.3	1.55	939	981.9	205.7	178	178	44.99	8.3	8.8	0	34.8	4883	6030	4577	34.2	4855	6024	4558	-0.6	12	6	6	4862	6035	4564	0.0	19.7	8.8	71.6	160	66	2.4	502	0.020	0.7	2	7.2	0.1	0.10	11.4	0.7	1.9	2.0
42.73	6.23	52.95	66.71	12.75	16.31	45.84	100.00	16.70	12.75	2.98	140.93	19.10	11.4	21.4	1.54	941	990.0	207.1	177	178	45.62	8.2	9.4	0	35.2	4903	6056	4550	34.1	4805	5955	4496	-1.1	1	1	7	4832	5990	4533	0.0	19.3	9.1	71.6	162	65	2.5	500	0.019	0.4	2	7.0	0.0	0.00	11.7	0.5	2.0	2.2
41.66	6.26	55.94	69.22	12.07	18.08	48.92	100.00	17.56	13.14	2.60	148.53	20.45	12.2	21.5	1.55	939	1004.4	206.4	178	178	46.67	9.1	10.1	0	33.8	4853	6027	4544	33.4	4777	5962	4509	-0.4	2	2	8	4856	6058	4580	0.0	20.0	9.1	70.9	158	64	2.5	516	0.020	0.8	3	7.0	0.0	0.00	11.4	0.5	1.9	2.3
40.89	6.26	55.94	69.22	12.07	18.08	48.92	100.00	17.56	13.14	2.60	148.53	20.45	12.2	21.4	1.55	936	990.8	208.2	177	177	45.23	8.9	8.2	0	34.4	4865	6034	4562	34.4	4926	6122	4643	0.0	3	3	9	4935	6145	4651	0.0	20.7	8.1	71.2	160	65	2.5	490	0.019	1.0	2	7.2	0.0	0.00	11.4	0.4	1.9	2.2
40.82	6.26	55.94	69.22	12.07	18.08	48.92	100.00	17.56	13.14	2.60	148.53	20.45	12.2	21.5	1.54	939	999.2	206.4	178	178	45.73	7.9	8.8	0	34.6	4950	6143	4665	33.6	4884	6084	4615	-1.0	4	4	10	4895	6120	4632	0.0	20.5	8.6	70.9	158	65	2.4	501	0.020	0.4	3	7.2	0.0	0.00	11.6	0.6	1.8	2.1
39.77	6.36	53.18	66.57	12.07	16.22	46.51	100.00	17.19	12.97	2.80	144.38	19.51	10.8	21.2	1.55	940	998.9	208.9	178	178	46.36	8.2	9.2	0	34.9	4912	6091	4625	33.5	4842	6027	4575	-1.4	1	1	12	4846	6040	4574	0.0	19.8	9.1	71.1	158	65	2.5	499	0.020	0.6	2	7.2	0.0	0.00	11.9	0.4	2.0	2.3
38.05	6.72	53.85	67.10	12.50	16.15	46.87	100.00	17.61	13.15	3.02	145.80	19.56	10.9	NA	1.55	934	1000.1	208.7	177	178	47.06	8.7	9.4	0	34.5	4875	6056	4596	33.4	4821	6003	4550	-1.1	2	2	13	4816	6037	4568	0.0	19.8	9.6	70.6	152	62	2.5	495	0.020	1.3	2	7.3	0.0	0.00	11.8	0.4	1.9	2.2
37.86	5.18	48.60	61.04	9.38	16.30	43.78	100.00	15.88	12.16	1.87	138.14	19.20	11.3	21.4	1.54	936	1001.2	206.2	178	177	46.95	9.3	9.6	0	34.3	4830	6001	4581	33.6	4809	5977	4569	-0.7	3	3	14	4817	6012	4584	0.0	19.6	9.6	70.8	150	62	2.4	502	0.021	1.1	3	7.2	0.0	0.00	11.8	0.4	1.9	2.4
38.03	5.18	48.60	61.04	9.38	16.30	43.78	100.00	15.88	12.16	1.87	138.14	19.20	11.3	21.3	1.55	936	1011.9	207.5	177	177	47.47	8.5	10.0	0	34.4	4893	6061	4635	32.6	4780	5952	4559	-1.8	4	4	15	4785	5973	4557	0.0	19.3	10.0	70.6	149	60	2.5	500	0.021	1.1	2	7.2	0.0	0.00	11.7	0.5	1.8	2.4
37.39	5.18	48.60	61.04	9.38	16.30	43.78	100.00	15.88	12.16	1.87	138.14	19.20	11.3	21.3	1.55	936	998.6	206.2	178	178	46.06	9.3	9.3	0	34.4	4833	6004	4589	34.0	4831	6005	4601	-0.4	5	5	16	4843	6014	4599	0.0	19.6	9.2	71.2	151	61	2.5	495	0.020	1.1	3	7.3	0.1	0.10	11.7	0.5	1.9	2.4
39.16	6.29	50.64	63.92	11.46	13.24	43.50	100.00	16.58	12.88	2.90	136.35	18.35	12.2	21.6	1.55	940	1002.1	207.5	178	178	46.58	9.7	9.3	0	33.4	4792	5964	4540	33.8	4822	5985	4562	0.4	7	1	1	4841	5996	4569	0.0	19.4	9.1	71.5	153	64	2.4	502	0.021	1.1	3	7.3	0.0	0.00	11.8	0.5	1.8	2.1
37.64	6.10	50.60	63.37	10.90	19.05	44.18	100.00	17.37	13.10	2.67	146.66	19.73	10.0	NA	1.55	935	986.1	205.2	177	178	46.06	9.5	9.5	0	34.7	4823	6003	4617	34.4	4831	6015	4609	-0.3	8	2	2	4822	6018	4606	0.0	19.8	9.6	70.6	152	60	2.6	497	0.020	1.1	3	7.2	0.0	0.00	11.7	0.7	1.9	2.3
39.77	5.30	46.87	57.56	9.93	18.07	40.60	100.00	16.34	12.12	2.43	135.81	18.57	13.1	21.3	1.55	936	983.6	207.5	178	178	45.30	10.4	9.8	0	33.7	4772	5962	4606	34.4	4812	5995	4621	0.7	9	3	3	4806	6000	4612	0.0	19.7	9.9	70.4	151	59	2.6	509	0.021	1.3	3	7.3	0.0	0.00	11.8	0.4	1.8	2.4
38.66	5.83	50.17	63.11	10.32	17.24	44.31	100.00	16.86	12.88	2.38	143.75	19.59	10.2	21.2	1.54	934	989.8	206.8	177	178	46.53	8.6	9.3	0	34.9	4883	6042	4623	33.9	4826	5997	4590	-1.0	10	4	4	4840	6001	4588	0.0	19.6	9.2	71.3	152	60	2.5	494	0.020	1.0	3	7.2	0.1	0.10	11.7	0.5	1.8	2.3
40.31	6.25	54.57	67.56	12.10	17.66	47.80	100.00	17.32	12.89	2.73	145.57	19.76	12.4	21.2	1.55	935	1008.1	206.6	178	178	46.12	8.7	9.5	0	34.4	4877	6059	4604	33.4	4821	6008	4568	-1.0	11	5	5	4829	6026	4572	0.0	19.8	9.4	70.7	156	61	2.5	497	0.020	1.0	3	7.2	0.0	0.00	11.7	0.6	1.8	2.3
40.54	6.25	54.57	67.56	12.10	17.66	47.80	100.00	17.32	12.89	2.73	145.57	19.76	12.4	21.4	1.54	934	1000.6	207.3	177	178	46.39	8.9	9.6	0	34.3	4864	6044	4589	33.4	4820	6008	4550	-0.9	12	6	6	4823	6022	4558	0.0	19.7	9.5	70.8	156	61	2.5	509	0.020	1.0	3	7.2	0.0	0.00	11.7	0.5	1.8	2.3
40.64	6.25	54.57	67.56	12.10	17.66	47.80	100.00	17.32	12.89	2.73	145.57	19.76	12.4	21.4	1.55	939	1011.6	206.8	178	178	46.50	9.0	10.0	0	33.9	4858	6035	4577	33.1	4790	5980	4531	-0.8	1	1	7	4802	5993	4533	0.0	19.5	9.8	70.7	156	64	2.4	493	0.020	1.1	2	7.3	0.0	0.00	11.8	0.5	1.9	2.3
38.60	6.00	53.29	65.50	11.71	18.80	46.34	100.00	17.32	12.75	2.63	145.37	19.75	12.7	21.4	1.55	937	1005.1	208.9	177	178	44.78	9.3	9.7	0	34.2	4846	6027	4608	33.7	4612	5999	4581	-0.5	2	2	8	4851	6049	4596	0.0	20.0	9.3	70.7	153	62	2.5	485	0.020	1.8	2	7.3	0.1	0.10	11.5	0.5	1.9	2.2
38.13	6.00	53.29	65.50	11.71	18.80	46.34	100.00	17.32	12.75	2.63	145.37	19.75	12.7	21.6	1.54	938	1014.4	206.4	178	178	44.52	9.3	9.4	0	34.2	4847	6029	4610	33.7	4831	6012	4605	-0.5	3	3	9	4844	6035	4588	0.0	19.9	9.4	70.7	152	63	2.4	517	0.021	1.0	3	7.3	0.0	0.00	11.8	0.5	1.9	1.8
40.10	6.00	53.29	65.50	11.71	18.80	46.34	100.00	17.32	12.75	2.63	145.37	19.75	12.7	21.6	1.55	941	1012.5	206.6	177	177	46.46	9.1	9.6	0	34.4	4858	6046	4619	33.5	4813	6004	4590	-0.9	4	1	10	4827	6011	4581	0.0	19.7	9.6	70.7	153	61	2.5	472	0.019	2.0	2	7.1	0.0	0.00	11.7	0.5	1.8	1.9
39.14	5.49	48.03	58.99	10.24	18.30	41.50	100.00	16.61	12.34	2.55	137.49	18.77	13.0	21.4	1.54	939	998.5	209.8	177	178	45.23	9.9	10.1	0	34.2	4802	5990	4620	34.0	4786	5983	4614	-0.2	6	3	12	4783	5983	4597	0.0	19.5	10.2	70.3	149	59	2.5	490	0.021	0.7	2	7.2	0.0	0.00	11.9	0.6	1.8	2.1
38.63	5.30	46.87	57.56	9.93	18.07	40.60	100.00	16.34	12.12	2.43	135.81	18.57	13.3	21.4	1.54	940	1012.2	209.4	178	178	46.74	9.4	10.6	0	34.0	4858	6067	4677	32.8	4768	5964	4596	-1.2	7	4	13	4776	5995	4607	0.0	19.7	10.5	69.8	143	56	2.5	496	0.022	1.0	3	7.1	0.1	0.10	11.9	0.4	1.7	1.8
41.43	5.54	52.48	64.98	10.30	18.24	45.86	100.00	17.07	12.90	2.18	145.88	20.08	13.4	21.4	1.54	939	1007.6	208.9	177	177	45.54	9.0	9.4	0	34.4	4862	6036	4601	33.5	4832	6003	4578	-0.9	8	5	14	4841	6023	4581	0.0	19.8	9.3	70.9	151	60	2.5	470	0.019	1.3	2	7.2	0.0	0.00	12.0	0.5	1.8	1.9
40.96	5.54	52.29	64.61	10.32	18.52	45.76	100.00	17.05	12.82	2.19	145.59	20.02	13.3	21.1	1.54	936	1022.3	206.4	178	178	46.26	9.3	10.1	0	34.2	4843	6021	4597	33.2	4780	5971	4553	-1.0	9	6	15	4781	5983	4551	0.0	19.5	10.1	70.4	148	59	2.5	507	0.021	0.6	3	7.2	0.0	0.00	11.7	0.6	1.7	2.3
37.89	6.25	52.68	65.12	11.64	18.11	45.42	100.00	17.51	13.01	2.88	145.07	19.56	12.7	21.5	1.60	946	1005.0	210.5	178	178	45.76	8.3	9.8	0	34.6	4920	6093	4648	33.3	4797	5977	4570	-1.3	1	1	16	4812	6008	4567	0.0	19.7	9.7	70.6	154	63	2.5	500	0.020	0.8	3	7.4	0.0	0.00	11.6	0.7	1.9	1.9
37.42	6.25	52.68	65.12	11.64	18.11	45.42	100.00	17.51	13.01	2.88	145.07	19.56	12.7	21.7	1.59	939	998.0	205.5	177	177	44.90	9.5	9.5	0	34.3	4831	6017	4591	34.2	4821	6009	4586	-0.1	2	2	17	4864	6056	4600	0.0	20.0	9.0	71.0	155	63	2.4	493	0.020	1.8	2	7.3	0.0	0.00	11.3	0.5	1.9	2.1
37.51	6.25	52.68	65.12	11.64	18.11	45.42	100.00	17.51	13.01	2.88	145.07	19.56	12.7	21.3	1.60	934	997.1	206.2	178	178	45.30	9.5	9.4	0	34.4	4824	6003	4586	34.3	4832	6012	4589	-0.1	3	3	18	4839	6019	4585	0.0	19.7	9.3	70.9	154	61	2.5	501	0.020	0.7	3	7.3	0.1	0.10	10.7	0.7	1.8	2.1
37.92	6.22	52.76	65.13	11.50	18.55	45.48	100.00	17.57	13.08	2.88	145.10	19.64	12.7	21.2	1.60	936	1006.1	207.3	177	177	45.28	8.7	9.5	0	34.7	4887	6076	4637	33.9	4829	6012	4599	-0.8	4	4	19	4844	6025	4594	0.0	19.8	9.3	70.9	152	61	2.5	470	0.019	1.7	2	7.3	0.0	0.00	11.3	0.5	1.8	1.8
36.77	5.90	51.37	63.65	10.76	19.90	45.07	100.00	17.06	12.78	2.46	144.98	19.73	12.4	21.3	1.54	935	990.5	205.7	177	177	44.38	9.1	9.1	0	35.0	4850	6015	4611	34.9	4856	6027	4618	-0.1	7	3	3	4852	6031	4606	0.0	19.8	9.1	71.0	152	65	2.3	505	0.021	1.5	3	7.3	0.0	0.00	11.0	0.4	1.9	2.3
37.14	5.90	51.37	63.65	10.76	19.90	45.07	100.00	17.06	12.78	2.46	144.98	19.73	12.4	21.0	1.55	934	1002.4	206.2	178	178	44.25	9.1	9.3	0	34.9	4854	6014	4617	34.6	4834	6009	4601	-0.3	8	4	4	4839	6015	4594	0.0	19.7	9.3	71.0	154	64	2.4	517	0.021	0.4	3	7.3	0.1	0.20	10.9	0.5	1.9	2.3
37.73	5.90	51.37	63.65	10.76	19.90	45.07	100.00	17.06	12.78	2.46	144.98	19.73	12.4	21.2	1.55	940	984.3	204.3	177	177	44.38	9.0	9.7	0	35.3	4861	6025	4611	34.8	4805	5975	4570	-0.5	9	5	5	4862	6060	4609	0.0	20.1	9.1	70.8	156	63	2.5	502	0.020	1.7	3	7.3	0.0	0.00	12.0	0.5	1.8	2.3
38.03	5.70	52.77	66.25	10.50	15.18	47.07	100.00	16.67	12.84	2.17	144.39	19.93	12.1	21.4	1.55	938	1005.3	207.1	178	178	44.86	8.5	8.8	0	34.8	4868	6012	4561	34.5	4844	5982	4542	-0.3	10	1	6	4852	6009	4556	0.0	19.4	8.8	71.8	156	64	2.5	502	0.020	0.9	3	7.3	0.0	0.00	12.1	0.3	1.8	2.3
37.86	5.70	52.77	66.25	10.50	15.18	47.07	100.00	16.67	12.84	2.17	144.39	19.93	12.1	21.3	1.55	936	1003.8	206.2	177	177	44.09	7.7	8.1	0	35.3	4924	6050	4597	34.8	4897	6022	4586	-0.5	11	2	7	4906	6040	4586	0.0	19.7	8.0	72.3	158	65	2.4	481	0.019	1.3	2	7.2	0.0	0.00	12.1	0.7	1.8	2.1
38.31	5.70	52.77	66.25	10.50	15.18	47.07	100.00	16.67	12.84	2.17	144.39	19.93	12.1	21.5	1.54	934	1009.5	205.7	178	178	44.62	7.8	8.6	0	35.2	4925	6058	4601	34.6	4863	6016	4594	-0.6	12	3	8	4906	6047	4585	0.0	19.7	8.1	72.2	158	63	2.5	506	0.020	0.9	3	7.3	0.0	0.00	12.0	0.3	1.8	2.3
38.66	5.97	53.13	66.58	11.00	16.55	46.77	100.00	16.97	13.00	2.40	145.03	19.88	12.1	21.1	1.54	938	1015.1	207.5	178	178	45.51	8.8	9.2	0	34.5	4856	6022	4579	34.1	4820	5989	4544	-0.4	2	1	10	4846	6015	4541	0.0	19.6	9.0	71.4	158	64	2.5	520	0.021	0.5	3	7.4	0.0	0.00	11.7	0.3	1.9	1.8
38.65	6.39	53.72	67.13	11.75	18.61	46.32	100.00	17.44	13.25	2.75	145.96	19.80	12.4	21.0	1.55	936	1020.8	206.6	177	177	46.22	8.8	9.2	0	34.7	4866	6040	4571	34.0	4844	6028	4553	-0.7	9	5	17	4855	6055	4581	0.0	19.9	9.1	71.0	158	65	2.4	482	0.019	1.6	3	7.3	0.0	0.00	11.5	0.6	1.8	2.2
38.67	5.36	53.39	65.30	12.05	18.57	46.91	100.00	16.42	12.32	2.31	144.29	19.71	11.5	21.2	1.55	939	1014.5	206.6	177	177	45.87	9.3	9.9	0	34.4	4833	6020	4569	34.1	4789	5973	4532	-0.3	1	1	19	4809	5990	4536	0.0	19.5	9.6	70.9	155	61	2.5	488	0.020	1.0	3	7.3	0.1	0.10	11.8	0.4	1.8	2.4
38.42	5.27	52.45	64.09	10.84	18.10	46.02	100.00	16.35	12.22	1.77	143.59	19.66	11.4	21.2	1.55	933	1029.0	205.9	178	177	47.70	10.5	10.3	0	33.5	4750	5952	4516	33.5	4765	5956	4520	0.0	2	2	20	4790	5993	4541	0.0	19.5	10.1	70.4	149	59	2.5	512	0.021	1.8	3	7.3	0.0	0.00	11.6	0.4	1.8	2.3
39.15	4.58	49.56	61.08	9.84	18.68	43.53	100.00	16.16	12.14	1.99	139.64	18.80	11.6	21.2	1.55	933	1008.0	205.5	177	178	45.51	9.3	10.1	0	34.4	4795	5982	4600	34.4	4785	5984	4587	0.0	3	3	21	4815	5991	4597	0.0	19.6	9.7	70.7	151	61	2.5	511	0.021	1.1	3	7.2	0.0	0.00	11.4	0.8	1.7	2.2
38.82	4.58	49.56	61.08	9.84	18.68	43.53	100.00	16.16	12.14	1.99	139.64	18.80	11.6	21.2	1.55	934	999.9	208.2	178	178	45.13	9.4	9.6	0	34.8	4832	6024	4625	34.7	4816	6001	4601	-0.1	4	4	22	4837	6024	4608	0.0	19.8	9.4	70.8	151	60	2.5	507	0.021	1.9	3	7.3	0.0	0.00	11.4	0.7	1.8	2.4
39.08	4.58	49.56	61.08	9.84	18.68	43.53	100.00	16.16	12.14	1.99	139.64	18.80	11.6	20.0	1.55	928	1003.2	206.6	177	177	45.73	9.7	10.0	0	34.4	4815	6010	4618	34.4	4786	5986	4590	0.0	5	5	23	4815	6011	4594	0.0	19.7	9.7	70.5	150	60	2.5	518	0.022	1.6	3	7.3	0.0	0.00	11.3	0.5	1.7	2.1
38.90	7.70	62.92	75.91	13.49	16.10	55.29	100.00	18.28	13.83	3.20	149.91	21.23	11.3	20.8	1.55	934	1014.6	208.7	177	177	46.04	8.2	8.2	0	34.8	4882	6022	4606	34.8	4884	6026	4613	0.0	7	1	1	4903	6034	4606	0.0	19.5	8.0	72.5	158	66	2.4	475	0.019	1.6	3	0.0	0.0	0.00	11.8	0.2	1.8	2.2
39.62	6.39	59.10	71.04	11.52	21.82	53.53	100.00	17.93	12.92	2.38	155.77	20.76	12.5	19.9	1.55	933	1005.1	205.2	178	177	45.31	8.8	9.1	0	34.9	4878	6058	4641	34.6	4853	6047	4630	-0.3	8	2	2	4872	6058	4629	0.0	20.1	8.9	71.0	160	66	2.4	496	0.019	1.4	3	0.0	0.0	0.00	11.6	0.3	1.8	2.5
39.77	6.63	59.81	71.94	11.89	20.76	53.86	100.00	17.99	13.09	2.53	154.68	20.85	14.1	20.0	1.54	936	1029.7	206.8	177	178	44.77	9.0	9.4	0	34.7	4860	6041	4617	34.0	4822	6000	4588	-0.7	9	3	3	4832	6013	4585	0.0	19.7	9.4	71.0	160	63	2.5	496	0.019	0.6	3	0.0	0.0	0.00	11.7	0.5	1.8	2.2
39.66	6.71	56.32	66.19	12.35	20.02	50.26	100.00	17.54	12.50	2.82	143.45	20.32	12.8	21.5	1.54	935	1027.0	206.2	178	177	46.78	9.6	9.6	0	33.9	4829	6026	4621	33.5	4815	6011	4612	-0.4	2	2	8	NA	NA	NA	NA	NA	NA	NA	156	NA	NA	NA	NA	2.3	0	0.0	0.0	0.00	0.0	0.6	0.0	0.0
39.68	6.87	56.74	66.61	12.55	20.18	50.80	100.00	17.48	12.41	2.82	143.10	20.24	12.8	21.5	1.56	933	1032.0	206.6	177	178	46.51	9.5	9.9	0	34.0	4833	6029	4608	33.5	4807	6001	4584	-0.5	0	0	0	NA	NA	NA	NA	NA	NA	NA	158	NA	NA	NA	NA	1.0	0	0.0	0.0	0.00	0.0	0.6	0.0	0.0
42.23	7.50	58.41	68.30	13.33	20.81	52.96	100.00	17.23	12.04	2.83	141.72	19.92	13.0	20.4	1.55	930	1040.0	208.7	178	177	48.05	10.1	10.4	0	33.1	4795	6000	4557	32.8	4764	5977	4538	-0.3	0	0	0	NA	NA	NA	NA	NA	NA	NA	167	NA	NA	NA	NA	1.3	0	0.0	0.0	0.00	0.0	0.6	0.0	0.0
38.48	7.53	58.36	69.25	14.35	20.57	51.31	100.00	17.87	12.77	3.55	145.56	20.04	14.1	21.6	1.55	935	1044.8	208.0	177	177	48.11	10.2	10.3	0	32.9	4793	6029	4600	32.4	4787	6030	4592	-0.5	0	0	0	NA	NA	NA	NA	NA	NA	NA	156	NA	NA	NA	NA	2.3	0	0.0	0.0	0.00	0.0	0.5	0.0	0.0
39.49	7.53	58.36	69.25	14.35	20.57	51.31	100.00	17.87	12.77	3.55	145.56	20.04	14.1	20.8	1.55	932	1053.8	207.5	178	178	48.13	9.9	10.3	0	32.9	4824	6068	4630	32.0	4795	6050	4607	-0.9	0	0	0	NA	NA	NA	NA	NA	NA	NA	160	NA	NA	NA	NA	0.9	0	0.0	0.0	0.00	0.0	0.6	0.0	0.0

Imputation Process

K Nearest Neighbor Technique

I have adopted the KNN approach for imputation for the current data set using the knnImputation() fruntion from DMwR package. I am using k=10 as a tuning parameter.

ChemicalManProcessImputed <- knnImputation(ChemicalManufacturingProcess, k = 10)

Pre-Processing

Excluding Near Zero Variance Predictors

nZVIndices <- nearZeroVar(ChemicalManProcessImputed)

ChemicalManProcessTransformed <- ChemicalManProcessImputed[,-nZVIndices]

Excluding highly Correlated Predictors

corThresh <- 0.9
tooHigh <- findCorrelation(cor(ChemicalManProcessTransformed), corThresh)
corrPred <- names(ChemicalManProcessTransformed)[tooHigh]
ChemicalManProcessTransformed <- ChemicalManProcessTransformed[,-tooHigh]

Test/Train Split

trainingRows <- createDataPartition(ChemicalManProcessTransformed$Yield, times = 1, p = 0.8, list = FALSE)

train_df <- ChemicalManProcessTransformed[trainingRows,]
test_df <- ChemicalManProcessTransformed[-trainingRows,]

Linear PLS Model

pls_model <- train(
  Yield ~ ., data = train_df, method = "pls",
  center = TRUE,
  scale = TRUE,
  trControl = trainControl("cv", number = 10),
  tuneLength = 25
)

pls_model

## Partial Least Squares 
## 
## 144 samples
##  46 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 131, 128, 130, 130, 129, 130, ... 
## Resampling results across tuning parameters:
## 
##   ncomp  RMSE      Rsquared   MAE     
##    1     1.531690  0.4496446  1.163700
##    2     1.547220  0.4873139  1.119879
##    3     1.607266  0.5376517  1.104752
##    4     1.825444  0.5178039  1.152061
##    5     2.146480  0.4823536  1.240617
##    6     2.416900  0.4635793  1.325782
##    7     2.852780  0.4486778  1.445887
##    8     3.272619  0.4452845  1.567872
##    9     3.659073  0.4409455  1.661615
##   10     3.989016  0.4380995  1.755627
##   11     4.292909  0.4396050  1.831603
##   12     4.487305  0.4395146  1.869397
##   13     4.704699  0.4392074  1.928953
##   14     4.795230  0.4389353  1.947829
##   15     4.910591  0.4395564  1.980380
##   16     5.053330  0.4415422  2.010911
##   17     5.176641  0.4419146  2.040667
##   18     5.266161  0.4396490  2.064457
##   19     5.386007  0.4401243  2.097618
##   20     5.476649  0.4416685  2.120398
##   21     5.627034  0.4423000  2.156865
##   22     5.729360  0.4425671  2.185449
##   23     5.857944  0.4406855  2.222400
##   24     5.955727  0.4371396  2.251753
##   25     5.991281  0.4348859  2.263472
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was ncomp = 1.

pls_predictions <- predict(pls_model, test_df)

results_ <- data.frame(t(postResample(pred = pls_predictions, obs = test_df$Yield))) %>%
  mutate("Model"= "PLS")

Which nonlinear regression model gives the optimal resampling and test set performance?

KNN Model

KNN_Model <- train(
                  Yield ~ ., data = train_df, method = "knn",
                  center = TRUE,
                  scale = TRUE,
                  trControl = trainControl("cv", number = 10),
                  tuneLength = 25)
KNN_Model

## k-Nearest Neighbors 
## 
## 144 samples
##  46 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 130, 128, 129, 130, 129, 128, ... 
## Resampling results across tuning parameters:
## 
##   k   RMSE      Rsquared   MAE     
##    5  1.538830  0.2979078  1.233104
##    7  1.583751  0.2574915  1.262188
##    9  1.591300  0.2514446  1.267891
##   11  1.601359  0.2420973  1.289624
##   13  1.579840  0.2613202  1.268824
##   15  1.606836  0.2413608  1.294699
##   17  1.611544  0.2405870  1.286746
##   19  1.622715  0.2373953  1.290872
##   21  1.624339  0.2362901  1.296138
##   23  1.632503  0.2332327  1.301691
##   25  1.657831  0.2168566  1.338169
##   27  1.669691  0.2151954  1.341896
##   29  1.676987  0.2144434  1.345498
##   31  1.677858  0.2240214  1.350751
##   33  1.684099  0.2291292  1.351920
##   35  1.694858  0.2213519  1.359842
##   37  1.701904  0.2250788  1.367375
##   39  1.711609  0.2208245  1.375639
##   41  1.713360  0.2247125  1.374744
##   43  1.714253  0.2359008  1.373852
##   45  1.718399  0.2485101  1.380568
##   47  1.728350  0.2375636  1.386594
##   49  1.730080  0.2434653  1.389811
##   51  1.741987  0.2280506  1.401164
##   53  1.750291  0.2051428  1.408994
## 
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was k = 5.

KNN_Pred <- predict(KNN_Model, newdata = test_df)

results_ <- data.frame(t(postResample(pred = KNN_Pred, obs = test_df$Yield))) %>%
  mutate("Model"= "KNN") %>% rbind(results_)

MARS Model

MARS_grid <- expand.grid(.degree = 1:2, .nprune = 2:15)

MARS_model <- train(
  Yield ~ ., data = train_df, method = "earth",
  tuneGrid = MARS_grid,
  preProcess = c("center", "scale"),
  trControl = trainControl("cv", number = 10),
  tuneLength = 25
)
MARS_model

## Multivariate Adaptive Regression Spline 
## 
## 144 samples
##  46 predictor
## 
## Pre-processing: centered (46), scaled (46) 
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 130, 130, 129, 130, 129, 129, ... 
## Resampling results across tuning parameters:
## 
##   degree  nprune  RMSE      Rsquared   MAE      
##   1        2      1.452867  0.3930060  1.1473125
##   1        3      1.344280  0.4868924  1.0806599
##   1        4      1.187584  0.5905783  0.9440652
##   1        5      1.188432  0.5856458  0.9360078
##   1        6      1.202950  0.5813013  0.9521411
##   1        7      1.142291  0.6122901  0.9158920
##   1        8      1.159583  0.6145650  0.9199298
##   1        9      1.147710  0.6191827  0.9289759
##   1       10      1.157172  0.6153554  0.9265057
##   1       11      1.158571  0.6159548  0.9339272
##   1       12      1.185477  0.6007300  0.9403314
##   1       13      1.215585  0.5856282  0.9579280
##   1       14      1.189262  0.6031387  0.9457467
##   1       15      1.157069  0.6239886  0.9218104
##   2        2      1.482178  0.3815296  1.1581702
##   2        3      1.284403  0.5278808  1.0342802
##   2        4      1.233108  0.5686280  1.0080982
##   2        5      1.185967  0.5950316  0.9809369
##   2        6      1.286354  0.5442740  1.0237503
##   2        7      1.368844  0.4932049  1.0688890
##   2        8      1.352108  0.5191945  1.0442623
##   2        9      1.353261  0.5214838  1.0526310
##   2       10      1.330361  0.5498477  1.0374128
##   2       11      1.337156  0.5484313  1.0471669
##   2       12      1.346348  0.5475549  1.0524228
##   2       13      1.353769  0.5406293  1.0548767
##   2       14      1.572854  0.5096394  1.1236550
##   2       15      1.569258  0.5126063  1.1256675
## 
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nprune = 7 and degree = 1.

MARS_predictions <- predict(MARS_model, test_df)

results_ <- data.frame(t(postResample(pred = MARS_predictions, obs = test_df$Yield))) %>%
  mutate("Model"= "MARS") %>% rbind(results_)

SVM Model

SVM_model <- train(
  Yield ~ ., data = train_df, method = "svmRadial",
  center = TRUE,
  scale = TRUE,
  trControl = trainControl(method = "cv"),
  tuneLength = 25
)
SVM_model

## Support Vector Machines with Radial Basis Function Kernel 
## 
## 144 samples
##  46 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 131, 130, 130, 129, 131, 130, ... 
## Resampling results across tuning parameters:
## 
##   C           RMSE      Rsquared   MAE      
##         0.25  1.378402  0.5400946  1.1367177
##         0.50  1.262877  0.5894361  1.0459388
##         1.00  1.165623  0.6394720  0.9645419
##         2.00  1.152515  0.6422550  0.9359212
##         4.00  1.163228  0.6362881  0.9219777
##         8.00  1.169604  0.6266254  0.9333011
##        16.00  1.190450  0.6138222  0.9466976
##        32.00  1.190450  0.6138222  0.9466976
##        64.00  1.190450  0.6138222  0.9466976
##       128.00  1.190450  0.6138222  0.9466976
##       256.00  1.190450  0.6138222  0.9466976
##       512.00  1.190450  0.6138222  0.9466976
##      1024.00  1.190450  0.6138222  0.9466976
##      2048.00  1.190450  0.6138222  0.9466976
##      4096.00  1.190450  0.6138222  0.9466976
##      8192.00  1.190450  0.6138222  0.9466976
##     16384.00  1.190450  0.6138222  0.9466976
##     32768.00  1.190450  0.6138222  0.9466976
##     65536.00  1.190450  0.6138222  0.9466976
##    131072.00  1.190450  0.6138222  0.9466976
##    262144.00  1.190450  0.6138222  0.9466976
##    524288.00  1.190450  0.6138222  0.9466976
##   1048576.00  1.190450  0.6138222  0.9466976
##   2097152.00  1.190450  0.6138222  0.9466976
##   4194304.00  1.190450  0.6138222  0.9466976
## 
## Tuning parameter 'sigma' was held constant at a value of 0.01396929
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were sigma = 0.01396929 and C = 2.

SVM_predictions <- predict(SVM_model, test_df)

results_ <- data.frame(t(postResample(pred = SVM_predictions, obs = test_df$Yield))) %>%
  mutate("Model"= "SVM") %>% rbind(results_)

Neural Network Model

nnet_grid <- expand.grid(.decay = c(0, 0.01, .1), .size = c(1:10), .bag = FALSE)
nnet_maxnwts <- 5 * ncol(train_df) + 5 + 1
nnet_model <- train(
  Yield ~ ., data = train_df, method = "avNNet",
  center = TRUE,
  scale = TRUE,
  tuneGrid = nnet_grid,
  trControl = trainControl(method = "cv"),
  linout = TRUE,
  trace = FALSE,
  MaxNWts = nnet_maxnwts,
  maxit = 500
)

nnet_model

## Model Averaged Neural Network 
## 
## 144 samples
##  46 predictor
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 130, 129, 129, 130, 129, 131, ... 
## Resampling results across tuning parameters:
## 
##   decay  size  RMSE      Rsquared    MAE     
##   0.00    1    1.809706  0.08812244  1.480013
##   0.00    2    1.820483  0.08589216  1.497667
##   0.00    3    1.780551  0.15860016  1.455172
##   0.00    4    1.891101  0.10362611  1.526117
##   0.00    5    1.823663  0.14284429  1.498001
##   0.00    6         NaN         NaN       NaN
##   0.00    7         NaN         NaN       NaN
##   0.00    8         NaN         NaN       NaN
##   0.00    9         NaN         NaN       NaN
##   0.00   10         NaN         NaN       NaN
##   0.01    1    1.553301  0.36918045  1.269452
##   0.01    2    1.590486  0.33655573  1.304391
##   0.01    3    1.345266  0.49924923  1.098344
##   0.01    4    1.785037  0.29566943  1.390056
##   0.01    5    1.468346  0.47781605  1.122994
##   0.01    6         NaN         NaN       NaN
##   0.01    7         NaN         NaN       NaN
##   0.01    8         NaN         NaN       NaN
##   0.01    9         NaN         NaN       NaN
##   0.01   10         NaN         NaN       NaN
##   0.10    1    1.496125  0.48701870  1.221730
##   0.10    2    1.413803  0.51602464  1.122862
##   0.10    3    1.846219  0.36080409  1.254734
##   0.10    4    1.689693  0.46744030  1.249126
##   0.10    5    1.924868  0.38450680  1.328834
##   0.10    6         NaN         NaN       NaN
##   0.10    7         NaN         NaN       NaN
##   0.10    8         NaN         NaN       NaN
##   0.10    9         NaN         NaN       NaN
##   0.10   10         NaN         NaN       NaN
## 
## Tuning parameter 'bag' was held constant at a value of FALSE
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 3, decay = 0.01 and bag = FALSE.

nnet_predictions <- predict(nnet_model, test_df)

results_ <- data.frame(t(postResample(pred = nnet_predictions, obs = test_df$Yield))) %>%
  mutate("Model"= "Neural Network") %>% rbind(results_)

Model Summary

results_ %>%
  select(Model, RMSE, Rsquared, MAE) %>%
  arrange(RMSE) %>%
  kable() %>%
  kable_styling()

Model	RMSE	Rsquared	MAE
SVM	1.238321	0.5818865	0.9681929
MARS	1.238899	0.5703493	0.9959592
KNN	1.524593	0.3296098	1.1638750
PLS	1.550276	0.3325869	1.2398936
Neural Network	1.917466	0.1368724	1.5143792

The SVM Model was the best non-linear model.

Which predictors are most important in the optimal nonlinear regression model? Do either the biological or process variables dominate the list? How do the top ten important predictors compare to the top ten predictors from the optimal linear model?

Variable Importance: Non-Linear Model

varImp(SVM_model, 10)

## loess r-squared variable importance
## 
##   only 20 most important variables shown (out of 46)
## 
##                        Overall
## ManufacturingProcess13  100.00
## ManufacturingProcess32   96.61
## BiologicalMaterial06     92.92
## ManufacturingProcess09   88.09
## BiologicalMaterial03     83.39
## ManufacturingProcess36   79.25
## ManufacturingProcess17   78.66
## ManufacturingProcess06   68.82
## ManufacturingProcess11   61.52
## BiologicalMaterial08     60.12
## BiologicalMaterial11     55.75
## BiologicalMaterial01     49.44
## ManufacturingProcess33   47.60
## ManufacturingProcess30   45.36
## BiologicalMaterial09     34.40
## ManufacturingProcess12   29.04
## ManufacturingProcess35   27.96
## BiologicalMaterial10     26.21
## ManufacturingProcess04   24.93
## ManufacturingProcess10   24.15

13 out of 20 most important variables are process related. So in a way they dominate the list over biological variables.

Variable Importance: Linear Model

varImp(pls_model, 10)

## pls variable importance
## 
##   only 20 most important variables shown (out of 46)
## 
##                        Overall
## ManufacturingProcess32  100.00
## ManufacturingProcess09   94.85
## ManufacturingProcess36   90.51
## ManufacturingProcess13   89.51
## BiologicalMaterial06     82.19
## BiologicalMaterial03     77.84
## BiologicalMaterial08     76.45
## ManufacturingProcess17   73.17
## ManufacturingProcess11   71.46
## ManufacturingProcess06   70.82
## ManufacturingProcess33   70.00
## BiologicalMaterial01     68.23
## BiologicalMaterial11     65.26
## ManufacturingProcess12   54.55
## ManufacturingProcess28   47.91
## ManufacturingProcess10   46.64
## ManufacturingProcess30   43.46
## ManufacturingProcess04   39.14
## ManufacturingProcess02   39.03
## BiologicalMaterial10     38.78

Observations

ManufacturingProcess13 is the most important variable in the SVM model, whereas ManufacturingProcess32 shows up as most important in the PLS model.
For both the models, process variables dominate the top 10 variables list.
Most of the variables in top 10 overlap in both the models.

Explore the relationships between the top predictors and the response for the predictors that are unique to the optimal nonlinear regression model. Do these plots reveal intuition about the biological or process predictors and their relationship with yield?

train_imp_df <- train_df %>% 
              dplyr::select(ManufacturingProcess13, ManufacturingProcess32, BiologicalMaterial06, 
                            ManufacturingProcess09, BiologicalMaterial03, ManufacturingProcess36, 
                            ManufacturingProcess17, ManufacturingProcess06, ManufacturingProcess11, 
                            BiologicalMaterial08, Yield)

correlations <- cor(train_imp_df)
corrplot::corrplot(correlations, order = "FPC", diag = TRUE)

The correlation plot illustrates some insight to these important variables.
Yield has a mostly positive correlation with many of the processes but with only Manufacturing processes have Yield with the strongest negative correlation seen in top variables.
Some very high correlations is noticed which it may implicate for possible issues of multi-collinearity.

DATA 624 - Homework8 - Non-Linear Regression

Soumya Ghosh

November 06, 2020

Libraries

Applied Predictive Modeling

Exercise 7.2

KNN Model

MARS Model

Variable Importance Check

SVM Model

Variable Importance Check

Neural Network Model

Variable Importance Check

Model Summary

Exercise 7.5

Data set Preview

Imputation Process

K Nearest Neighbor Technique

Pre-Processing

Excluding Near Zero Variance Predictors

Excluding highly Correlated Predictors

Test/Train Split

Linear PLS Model

KNN Model

MARS Model

SVM Model

Neural Network Model

Model Summary

Variable Importance: Non-Linear Model

Variable Importance: Linear Model

Observations