Project Requirements

The website https://data.cityofnewyork.us provides several datasets related to New York city in various domains such as Health, Transportation, Education etc. One such data set related to health category is the “Restaurants” dataset. As per the web site (https://data.cityofnewyork.us), the Department of Health and Mental Hygiene conducts unannounced inspections of restaurants at least once a year. Inspectors check for compliance in food handling, food temperature, personal hygiene and vermin control. Each violation of a regulation gets a certain number of points. At the end of the inspection, the inspector totals the points, and this number is the restaurant’s inspection score. The lower the score, the better the Grade

The major requirements of this project are given below:

R1. Which factors affect the number of days between the inspections visit?

R2. Which factors affect the restaurant’s closure (in the current inspection)?

R3. Which factors affect the restaurant’s closure in future (in the next inspection)?

These three business requirements if correctly implemented, will help the restaurant owners to focus on the areas of improvement, and be prepared for sudden inspection by the food inspectors, and avoid possible closure of the restaurant.

Data gathering

The data is publicly available at the following web site: https://data.cityofnewyork.us/Health/DOHMH-New-York-City-Restaurant-Inspection-Results/xx67-kt59

Technical design/requirements

T1. We will be implementing a RDBMS (Relational Database Management System) database in postgreSQL, following the principles of 1st, 2nd and 3rd normal forms. The RDBMS will help us to maintain the data in a structured format. Also given that the restaurant’s data is updated frequently (almost daily) at https://data.cityofnewyork.us, a RDBMS database will help us to store the data consistently for performing up to date analysis using our data analysis algorithms, and thus provide accurate predictions. NOTE that in the current scope of the project we will not be doing any on the fly analysis of the data. See T3 requirement given below, for more information.

T2. ERWIN will be used to perform the RDBMS database design

T3. The required CSV files for analysis are produced from the RDBMS database tables, and these CSV files will be stored at github.com. These files will be further used by R programs to transform the data to a proper format and perform data analysis and provide predictions (listed in R1 and R2 requirements)

T4. Produce separate R data frames, for training, and one data frame (2015 data) for algorithm testing. The idea is to select the best statistical learning algorithms with the least MSE (Mean Squared Error)

T5. Statistical models will be implemented using the training data, and their performance is evaluated using the test data. The training data belongs to all years before 2015, and the test data belongs to 2015 year

T1/T2/T3 - RDBMS Design, implementation and final CSV files generation

We will be creating the following RDBMS tables:

  • ALL_DATA_DUMP - Contains all the data from https://data.cityofnewyork.us/Health/DOHMH-New-York-City-Restaurant-Inspection-Results/xx67-kt59

  • VIOLATION - Contains all the possible violation codes, violation’s description, and the level (critical/non-critical)

  • CUISINE - Contains all the available cuisines

  • RESTAURANT - Contains all the restaurants available, identified by restaurant ID

  • RESTAURANT_CUISINE - Contains restaurant’s cuisine information (mapping between restaurants and cuisines)

  • INSPECTION - Restaurant’s inspecton, violation, inspection date, score received, number of the visit (denoted by LEVEL)

Here is the database design (created using ERWIN):

DB Design

Figure: 1 Database Design

The SQL Statements can be found at the following location (under the file: SQL_Statements.txt): https://github.com/msekhar12/MSDA_FINAL_PROJECT

The following three files are created (CSV Files) out of the SQL Statements provided at https://github.com/msekhar12/MSDA_FINAL_PROJECT (SQL_Statements.txt)

The file names along with the attributes of the files data is described below:

  • inspection file has the attributes visit_level, restaurant_id, cuisine_id, inspection_date, closed, violation_code, score

visit_level - contains the inspection number

restaurant_id - unique ID of the restaurant

cuisine_id - Cuisine identifier

inspection_date - inspection date

closed - contains 1 or 0. 1 means the restaurant is closed and 0 mean the restaurant is not closed

violation_code - violation code

score - score received in the inspection

  • violation_codes file with the attributes code,description,level

code - violation code

description - violation long description

level - Can be “Critical” or “Not critical”

  • cuisine file with the attributes id,name

id - cuisine identifier

name - cuisine name

The above three files are loaded at https://github.com/msekhar12/MSDA_FINAL_PROJECT. These files will be accessed and loaded into R data frames for further analysis.

T4 - Implementation (Data transformation and the creation of training and test datasets)

Required R libraries

We need the following R libraries to perform data transformation and analysis

library(RCurl)
## Loading required package: bitops
library(dplyr)
## 
## Attaching package: 'dplyr'
## 
## The following object is masked from 'package:stats':
## 
##     filter
## 
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(tidyr)
library(knitr)
library(leaps)
library(boot)
library(FNN)
library(ggplot2)
library(MASS)
## 
## Attaching package: 'MASS'
## 
## The following object is masked from 'package:dplyr':
## 
##     select

Reading the data from github

The following R code reads data from the 3 files (cuisine.txt, inspection.txt, violation_codes.txt) placed at “https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/”

URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/inspection.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
inspection_data <- read.csv(textConnection(x))

kable(head(inspection_data))
visit_level restaurant_id cuisine_id inspection_date closed violation_code score
8 30075445 8 2015-02-09 0 06C 6
7 30075445 8 2014-03-03 0 10F 2
6 30075445 8 2013-10-10 0 000 0
5 30075445 8 2013-09-11 0 04L 6
5 30075445 8 2013-09-11 0 04N 6
4 30075445 8 2013-08-14 0 04L 32 
URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/cuisine.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
cuisine_data <- read.csv(textConnection(x))

kable(head(cuisine_data))
id name
1 Afghan
2 African
3 American
4 Armenian
5 Asian
6 Australian 
URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/violation_codes.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
violation_data <- read.csv(textConnection(x))

kable(head(violation_data))
code description level
18I Choking first aid poster not posted. Alcohol and Pregnancy warning sign, inspection report sign; not posted. CPR sign not posted, equipment (resuscitation masks, adult & pediatric, latex gloves) not provided. Not Critical
04G Unprotected potentially hazardous food re-served. Critical
10D Mechanical or natural ventilation system not provided, improperly installed, in disrepair and/or fails to prevent excessive build-up of grease, heat, steam condensation vapors, odors, smoke, and fumes. Not Critical
04A Food Protection Certificate not held by supervisor of food operations. Critical
10E Accurate thermometer not provided in refrigerated or hot holding equipment. Not Critical
99B Other general violation. Not Critical

The inspection_data data frame contains the inspection details with the following variables:

  • visit_level - The inspection number

  • restaurant_id - The restaurant ID (We are not concerned about the name)

  • cuisine_id - The cuisine of the restaurant. (we are not concerned with the name of the cuisine)

  • inspection_date - The date, when the restaurant was inspected

  • closed - will be either 1 or 0. 1 represents that the restaurant is closed, and 0 represents that it is not closed

  • violation_code - violation code (encoded)

  • score - restaurant’s current score. Lesser the score, better is the restaurant

The cuisine_data has the following cuisine details (variables):

  • id - Cuisine ID

  • name - Cuisine name

The violation_data has the following variables:

  • code - Violation ID

  • description - Violation description

  • level - “Critical” and “Not Critical”

Data transfomation

Creating inspection_spread data frame. This data frame will have the violation codes as the columns

inspection_data$citation <- 1

inspection_spread <- spread(inspection_data,violation_code, citation)

#head(inspection_spread)
#head(inspection_spread,100)

#Filling NA values with 0, in the citation
inspection_spread[is.na(inspection_spread)] <- 0

names(inspection_spread) 
##  [1] "visit_level"     "restaurant_id"   "cuisine_id"     
##  [4] "inspection_date" "closed"          "score"          
##  [7] "000 "            "02A "            "02B "           
## [10] "02C "            "02D "            "02E "           
## [13] "02F "            "02G "            "02H "           
## [16] "02I "            "02J "            "03A "           
## [19] "03B "            "03C "            "03D "           
## [22] "03E "            "03F "            "03G "           
## [25] "04A "            "04B "            "04C "           
## [28] "04D "            "04E "            "04F "           
## [31] "04G "            "04H "            "04I "           
## [34] "04J "            "04K "            "04L "           
## [37] "04M "            "04N "            "04O "           
## [40] "05A "            "05B "            "05C "           
## [43] "05D "            "05E "            "05F "           
## [46] "05H "            "05I "            "06A "           
## [49] "06B "            "06C "            "06D "           
## [52] "06E "            "06F "            "06G "           
## [55] "06H "            "06I "            "07A "           
## [58] "08A "            "08B "            "08C "           
## [61] "09A "            "09B "            "09C "           
## [64] "10A "            "10B "            "10C "           
## [67] "10D "            "10E "            "10F "           
## [70] "10G "            "10H "            "10I "           
## [73] "10J "            "10K "            "16B "           
## [76] "18D "            "18F "            "18G "           
## [79] "18I "            "20A "            "20D "           
## [82] "20E "            "20F "            "22A "           
## [85] "22C "            "99B "
dim(inspection_spread)
## [1] 160536     86
#Changing the names of the columns where the names begin with a number. The citation codes begin with a number, but R does not support data frame column names beginning with a number
#Execute this statement only once.  

names(inspection_spread)[7:86] <- paste("C_",names(inspection_spread)[7:86],sep="")
kable(head(inspection_spread))
visit_level restaurant_id cuisine_id inspection_date closed score C_000 C_02A C_02B C_02C C_02D C_02E C_02F C_02G C_02H C_02I C_02J C_03A C_03B C_03C C_03D C_03E C_03F C_03G C_04A C_04B C_04C C_04D C_04E C_04F C_04G C_04H C_04I C_04J C_04K C_04L C_04M C_04N C_04O C_05A C_05B C_05C C_05D C_05E C_05F C_05H C_05I C_06A C_06B C_06C C_06D C_06E C_06F C_06G C_06H C_06I C_07A C_08A C_08B C_08C C_09A C_09B C_09C C_10A C_10B C_10C C_10D C_10E C_10F C_10G C_10H C_10I C_10J C_10K C_16B C_18D C_18F C_18G C_18I C_20A C_20D C_20E C_20F C_22A C_22C C_99B
8 30075445 8 2015-02-09 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
7 30075445 8 2014-03-03 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
6 30075445 8 2013-10-10 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
5 30075445 8 2013-09-11 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
4 30075445 8 2013-08-14 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
3 30075445 8 2013-01-24 0 10 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

To predict if a restaurant is closed in the next visit, we need to add a variable which contains the information, if a restaurant is closed in the next visit. R code to add closed_next variable to inspection_spread data frame is given below.

closed_next <- vector(length=length(inspection_spread$closed))
closed_next[1] <- NA
closed_next[2:(length(inspection_spread$closed))] <- inspection_spread$closed[1:(length(inspection_spread$closed)-1)]
#data.frame(inspection_spread$closed,closed_next)

restaurant_id_tmp <- vector(length=length(inspection_spread$restaurant_id))
restaurant_id_tmp[1] <- NA
restaurant_id_tmp[2:(length(inspection_spread$restaurant_id))] <- inspection_spread$restaurant_id[1:(length(inspection_spread$restaurant_id)-1)]
#data.frame(inspection_spread$restaurant_id,restaurant_id_tmp)

inspection_spread <- cbind(inspection_spread, closed_next, restaurant_id_tmp)

inspection_spread$closed_next[which(inspection_spread$restaurant_id != inspection_spread$restaurant_id_tmp)] <- NA

#head(inspection_spread)
#dim(inspection_spread) 
inspection_spread <- inspection_spread[,-88]
names(inspection_spread)[87] <- "closed_next"
#names(inspection_spread)
#head(inspection_spread,100)

To Predict the days between the visits…we need to add another variable called days_diff to inspection_spread data frame

date_temp <- vector(length=length(inspection_spread$inspection_date))
date_temp[1] <- NA
date_temp[2:(length(inspection_spread$inspection_date))] <- as.character(inspection_spread$inspection_date[1:(length(inspection_spread$inspection_date)-1)])
#data.frame(inspection_spread$inspection_date,date_temp)

restaurant_id_tmp <- vector(length=length(inspection_spread$restaurant_id))
restaurant_id_tmp[1] <- NA
restaurant_id_tmp[2:(length(inspection_spread$restaurant_id))] <- inspection_spread$restaurant_id[1:(length(inspection_spread$restaurant_id)-1)]
#data.frame(inspection_spread$restaurant_id,restaurant_id_tmp)

df_tmp <- data.frame(inspection_spread$inspection_date,date_temp,inspection_spread$restaurant_id,restaurant_id_tmp)
#names(df_tmp)
#head(df_tmp)

#Populating NA values to date_temp, when restaurant_id_tmp and inspection_spread.restaurant_id are not equal
df_tmp$date_temp[which(df_tmp$inspection_spread.restaurant_id != df_tmp$restaurant_id_tmp)] <- NA
#dim(df_tmp)
#dim(inspection_spread)


#Adding a days_diff variable to inspection_spread

inspection_spread$days_diff <- as.integer(difftime(strptime(df_tmp$date_temp, format = "%Y-%m-%d"),
strptime(df_tmp$inspection_spread.inspection_date, format = "%Y-%m-%d"),units="days")
)

#head(inspection_spread,10)

Adding the year and month variables to inspection_spread data frame. These variables will help us to separate the data into different data frames (for training and test data) depending on the year/month of the inspection

y <- format(strptime(inspection_spread$inspection_date, format = "%Y-%m-%d"),"%Y")
m <- format(strptime(inspection_spread$inspection_date, format = "%Y-%m-%d"),"%m")

inspection_spread <- cbind(inspection_spread,year=y,month=m)

#head(inspection_spread)

Displaying new columns (closed_next, days_diff, year, month) along with some other columns of the inspection_spread

head(data.frame(visit_level=inspection_spread$visit_level, 
restaurant_id=inspection_spread$restaurant_id, 
cuisine_id=inspection_spread$cuisine_id, 
inspection_date=inspection_spread$inspection_date,
closed=inspection_spread$closed,
closed_next=inspection_spread$closed_next,
days_diff=inspection_spread$days_diff,
score=inspection_spread$score,
year=inspection_spread$year,
month=inspection_spread$month),10) 
##    visit_level restaurant_id cuisine_id inspection_date closed closed_next
## 1            8      30075445          8      2015-02-09      0          NA
## 2            7      30075445          8      2014-03-03      0           0
## 3            6      30075445          8      2013-10-10      0           0
## 4            5      30075445          8      2013-09-11      0           0
## 5            4      30075445          8      2013-08-14      0           0
## 6            3      30075445          8      2013-01-24      0           0
## 7            3      30075445          8      2013-01-24      0           0
## 8            2      30075445          8      2012-12-31      0           0
## 9            1      30075445          8      2012-05-03      0           0
## 10           7      30112340         37      2015-05-07      0          NA
##    days_diff score year month
## 1         NA     6 2015    02
## 2        343     2 2014    03
## 3        144     0 2013    10
## 4         29     6 2013    09
## 5         28    32 2013    08
## 6        201    10 2013    01
## 7          0     0 2013    01
## 8         24    25 2012    12
## 9        242     0 2012    05
## 10        NA    12 2015    05

Preparing R data frames for training and testing of statistical models

The following R code creates days_diff_training data frame, and days_diff_testing data frame. The days_diff_training data frame will have the daya related to all the years except the 2015 data. We will also eliminate unnecessary variables, which are not needed for creating the statistical models to predict days_diff variable output. The days_diff variable will have the predicted number of days, after which a sudden food inspection could happen. An error of 1 week is allowed in the prediction.

days_diff_training <- inspection_spread[inspection_spread$year != 2015,]
#days_diff_training$month <- as.integer(days_diff_training$month)
#days_diff_training$year <- as.integer(as.character(days_diff_training$year))


#Elimination variables restaurant_id, inspection_date and closed_next from the training data, since these are not needed for training
#names(days_diff_training)
days_diff_training <- days_diff_training[,c(-2, -4, -5, -87, -89, -90)]
#names(days_diff_training)

#Omitting the NA values from training data set:
days_diff_training <- na.omit(days_diff_training)
dim(days_diff_training)
## [1] 129293     84
#Plotting the box plot
ggplot(days_diff_training,aes(x=score,y=days_diff))+
 geom_boxplot()

#eliminating outliers (days_diff <= 365 is only correct data)
days_diff_training <- days_diff_training[days_diff_training$days_diff <= 365,]
days_diff_training <- days_diff_training[days_diff_training$days_diff != 0,]
dim(days_diff_training)
## [1] 117541     84
days_diff_testing <- inspection_spread[inspection_spread$year == 2015,]
#days_diff_training$month <- as.integer(days_diff_training$month)
#days_diff_training$year <- as.integer(as.character(days_diff_training$year))


#Elimination variables restaurant_id, inspection_date and closed_next from the training data, since these are not needed for training
#names(days_diff_testing)
days_diff_testing <- days_diff_testing[,c(-2, -4, -5, -87, -89, -90)]
#names(days_diff_testing)

#

days_diff_testing <- na.omit(days_diff_testing)
#dim(days_diff_testing)

days_diff_testing <- days_diff_testing[days_diff_testing$days_diff <= 365,]
days_diff_testing <- days_diff_testing[days_diff_testing$days_diff != 0,]
dim(days_diff_testing)
## [1] 6413   84
kable(head(days_diff_testing))
visit_level cuisine_id score C_000 C_02A C_02B C_02C C_02D C_02E C_02F C_02G C_02H C_02I C_02J C_03A C_03B C_03C C_03D C_03E C_03F C_03G C_04A C_04B C_04C C_04D C_04E C_04F C_04G C_04H C_04I C_04J C_04K C_04L C_04M C_04N C_04O C_05A C_05B C_05C C_05D C_05E C_05F C_05H C_05I C_06A C_06B C_06C C_06D C_06E C_06F C_06G C_06H C_06I C_07A C_08A C_08B C_08C C_09A C_09B C_09C C_10A C_10B C_10C C_10D C_10E C_10F C_10G C_10H C_10I C_10J C_10K C_16B C_18D C_18F C_18G C_18I C_20A C_20D C_20E C_20F C_22A C_22C C_99B days_diff
79 11 18 30 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 27
96 7 39 18 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13
104 7 48 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29
115 5 41 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 23
142 5 26 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 20
171 3 3 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 
kable(head(days_diff_training))
visit_level cuisine_id score C_000 C_02A C_02B C_02C C_02D C_02E C_02F C_02G C_02H C_02I C_02J C_03A C_03B C_03C C_03D C_03E C_03F C_03G C_04A C_04B C_04C C_04D C_04E C_04F C_04G C_04H C_04I C_04J C_04K C_04L C_04M C_04N C_04O C_05A C_05B C_05C C_05D C_05E C_05F C_05H C_05I C_06A C_06B C_06C C_06D C_06E C_06F C_06G C_06H C_06I C_07A C_08A C_08B C_08C C_09A C_09B C_09C C_10A C_10B C_10C C_10D C_10E C_10F C_10G C_10H C_10I C_10J C_10K C_16B C_18D C_18F C_18G C_18I C_20A C_20D C_20E C_20F C_22A C_22C C_99B days_diff
2 7 8 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 343
3 6 8 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 144
4 5 8 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29
5 4 8 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 28
6 3 8 10 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 201
8 2 8 25 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 
ggplot(days_diff_training,aes(x=score,y=days_diff))+
 geom_boxplot()

From the bar plot display, we can conclude that on an average the restaurant is visited for every 100 days (approximately)

Thus we produced the following 2 data frames needed for the model implementation and testing for days_diff prediction.

  • days_diff_training

  • days_diff_testing

T5. Statistical model implementation and evaluation

Predicting the “days_diff” variable

In the days_diff_training we have 84 variables, including days_diff variable. We will use the forward selection method, and evaluate which variables are optimal for prediction, and eliminate the variables which are not really needed. The main draw back of this approach is we are assuming a linear model. But this will at least help us to eliminate some irrevelant variables. Once the important variables are obtained, we can create more models for various degrees of freedom and select the optimal method. We will also evaluate KNN’s (K-Nearest Neighbors method) performance.

set.seed(18)
regfit.fwd <- regsubsets (days_diff~., data=days_diff_training, nvmax =83, method ="forward")
## Warning in leaps.setup(x, y, wt = wt, nbest = nbest, nvmax = nvmax,
## force.in = force.in, : 6 linear dependencies found
## Reordering variables and trying again:
reg.summary <- summary(regfit.fwd)

summary(regfit.fwd)
## Subset selection object
## Call: regsubsets.formula(days_diff ~ ., data = days_diff_training, 
##     nvmax = 83, method = "forward")
## 83 Variables  (and intercept)
##             Forced in Forced out
## visit_level     FALSE      FALSE
## cuisine_id      FALSE      FALSE
## score           FALSE      FALSE
## `C_000 `        FALSE      FALSE
## `C_02A `        FALSE      FALSE
## `C_02B `        FALSE      FALSE
## `C_02C `        FALSE      FALSE
## `C_02D `        FALSE      FALSE
## `C_02E `        FALSE      FALSE
## `C_02F `        FALSE      FALSE
## `C_02G `        FALSE      FALSE
## `C_02H `        FALSE      FALSE
## `C_02I `        FALSE      FALSE
## `C_03A `        FALSE      FALSE
## `C_03B `        FALSE      FALSE
## `C_03C `        FALSE      FALSE
## `C_03D `        FALSE      FALSE
## `C_03E `        FALSE      FALSE
## `C_03G `        FALSE      FALSE
## `C_04A `        FALSE      FALSE
## `C_04B `        FALSE      FALSE
## `C_04C `        FALSE      FALSE
## `C_04D `        FALSE      FALSE
## `C_04E `        FALSE      FALSE
## `C_04F `        FALSE      FALSE
## `C_04G `        FALSE      FALSE
## `C_04H `        FALSE      FALSE
## `C_04I `        FALSE      FALSE
## `C_04J `        FALSE      FALSE
## `C_04K `        FALSE      FALSE
## `C_04L `        FALSE      FALSE
## `C_04M `        FALSE      FALSE
## `C_04N `        FALSE      FALSE
## `C_04O `        FALSE      FALSE
## `C_05A `        FALSE      FALSE
## `C_05B `        FALSE      FALSE
## `C_05C `        FALSE      FALSE
## `C_05D `        FALSE      FALSE
## `C_05E `        FALSE      FALSE
## `C_05F `        FALSE      FALSE
## `C_05H `        FALSE      FALSE
## `C_05I `        FALSE      FALSE
## `C_06A `        FALSE      FALSE
## `C_06B `        FALSE      FALSE
## `C_06C `        FALSE      FALSE
## `C_06D `        FALSE      FALSE
## `C_06E `        FALSE      FALSE
## `C_06F `        FALSE      FALSE
## `C_06G `        FALSE      FALSE
## `C_06H `        FALSE      FALSE
## `C_06I `        FALSE      FALSE
## `C_07A `        FALSE      FALSE
## `C_08A `        FALSE      FALSE
## `C_08B `        FALSE      FALSE
## `C_08C `        FALSE      FALSE
## `C_09A `        FALSE      FALSE
## `C_09B `        FALSE      FALSE
## `C_09C `        FALSE      FALSE
## `C_10A `        FALSE      FALSE
## `C_10B `        FALSE      FALSE
## `C_10C `        FALSE      FALSE
## `C_10D `        FALSE      FALSE
## `C_10E `        FALSE      FALSE
## `C_10F `        FALSE      FALSE
## `C_10G `        FALSE      FALSE
## `C_10H `        FALSE      FALSE
## `C_10I `        FALSE      FALSE
## `C_10J `        FALSE      FALSE
## `C_10K `        FALSE      FALSE
## `C_18F `        FALSE      FALSE
## `C_18G `        FALSE      FALSE
## `C_20A `        FALSE      FALSE
## `C_20E `        FALSE      FALSE
## `C_20F `        FALSE      FALSE
## `C_22A `        FALSE      FALSE
## `C_22C `        FALSE      FALSE
## `C_99B `        FALSE      FALSE
## `C_02J `        FALSE      FALSE
## `C_03F `        FALSE      FALSE
## `C_16B `        FALSE      FALSE
## `C_18D `        FALSE      FALSE
## `C_18I `        FALSE      FALSE
## `C_20D `        FALSE      FALSE
## 1 subsets of each size up to 77
## Selection Algorithm: forward
##           visit_level cuisine_id score `C_000 ` `C_02A ` `C_02B ` `C_02C `
## 1  ( 1 )  " "         " "        "*"   " "      " "      " "      " "     
## 2  ( 1 )  " "         " "        "*"   "*"      " "      " "      " "     
## 3  ( 1 )  " "         " "        "*"   "*"      " "      " "      " "     
## 4  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 5  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 6  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 7  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 8  ( 1 )  "*"         " "        "*"   "*"      " "      "*"      " "     
## 9  ( 1 )  "*"         " "        "*"   "*"      " "      "*"      " "     
## 10  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 11  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 12  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 13  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 14  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 15  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 16  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 17  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 18  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 19  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 20  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 21  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 22  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 23  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 24  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 25  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 26  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 27  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 28  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 29  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 30  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 31  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 32  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 33  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 34  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 35  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 36  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 37  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 38  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 39  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 40  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 41  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 42  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 43  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 44  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 45  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 46  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 47  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 48  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 49  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 50  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 51  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 52  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 53  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 54  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 55  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 56  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 57  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 58  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 59  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 60  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 61  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 62  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 63  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 64  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 65  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 71  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 72  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 73  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 74  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 75  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
##           `C_02D ` `C_02E ` `C_02F ` `C_02G ` `C_02H ` `C_02I ` `C_02J `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      "*"      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 32  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 33  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 34  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 35  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 36  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 37  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 41  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 42  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 43  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 44  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 45  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 46  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 47  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 48  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 49  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 50  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 51  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 52  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 53  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 54  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 55  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 56  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 57  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 58  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 59  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 60  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 61  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 62  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 63  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 64  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 65  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
##           `C_03A ` `C_03B ` `C_03C ` `C_03D ` `C_03E ` `C_03F ` `C_03G `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 41  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 42  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 43  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 44  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 45  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 46  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 47  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 48  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 49  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 50  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 51  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 52  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 53  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 54  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 55  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 56  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 57  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 58  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 59  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 60  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 61  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 62  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 63  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 64  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
##           `C_04A ` `C_04B ` `C_04C ` `C_04D ` `C_04E ` `C_04F ` `C_04G `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 28  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 29  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 30  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 31  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 32  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 33  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 34  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 35  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 36  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 37  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 38  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 39  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 40  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 41  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 42  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 43  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 44  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 45  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 46  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 47  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 48  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 49  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 50  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 51  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 52  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 53  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 54  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 55  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 56  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 57  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 58  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 59  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 60  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 61  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 62  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_04H ` `C_04I ` `C_04J ` `C_04K ` `C_04L ` `C_04M ` `C_04N `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      "*"      " "      "*"     
## 7  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 8  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 9  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 10  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 11  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 12  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 13  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 14  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 15  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 16  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 17  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 18  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 19  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 20  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 21  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 22  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 23  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 24  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 25  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 26  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 27  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 28  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 47  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 48  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 49  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_04O ` `C_05A ` `C_05B ` `C_05C ` `C_05D ` `C_05E ` `C_05F `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 34  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 35  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 36  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 37  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 38  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 39  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 40  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 41  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 42  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 43  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 44  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 45  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 46  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 47  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 48  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 49  ( 1 ) " "      " "      " "      "*"      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_05H ` `C_05I ` `C_06A ` `C_06B ` `C_06C ` `C_06D ` `C_06E `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      "*"      "*"     
## 19  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 20  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 21  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 22  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 23  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 24  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 25  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 26  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 27  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 28  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_06F ` `C_06G ` `C_06H ` `C_06I ` `C_07A ` `C_08A ` `C_08B `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 25  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 26  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 27  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 28  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_08C ` `C_09A ` `C_09B ` `C_09C ` `C_10A ` `C_10B ` `C_10C `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 32  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 33  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 34  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 35  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 36  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 37  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 41  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 42  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 43  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 44  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 45  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 46  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 47  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 48  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 49  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 50  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 51  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 52  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 53  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 54  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 55  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 56  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 57  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_10D ` `C_10E ` `C_10F ` `C_10G ` `C_10H ` `C_10I ` `C_10J `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 22  ( 1 ) " "      "*"      "*"      " "      " "      " "      " "     
## 23  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 24  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 25  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 26  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 27  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 28  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 29  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 30  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 31  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 32  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 33  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 34  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 35  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 36  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 37  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 38  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 39  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 58  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_10K ` `C_16B ` `C_18D ` `C_18F ` `C_18G ` `C_18I ` `C_20A `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 38  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 39  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 40  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 41  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 42  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 43  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 44  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 45  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 46  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 47  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 48  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 49  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 50  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 51  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 52  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 53  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 54  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 55  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 56  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 57  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 58  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 59  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 60  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 61  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 62  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 63  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 64  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 65  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 66  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 67  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 68  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 69  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 70  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 71  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 72  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 73  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 74  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 75  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 76  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 77  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
##           `C_20D ` `C_20E ` `C_20F ` `C_22A ` `C_22C ` `C_99B `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      " "      " "      " "     
## 38  ( 1 ) " "      " "      " "      " "      " "      " "     
## 39  ( 1 ) " "      " "      " "      " "      " "      " "     
## 40  ( 1 ) " "      " "      " "      " "      " "      " "     
## 41  ( 1 ) " "      " "      " "      " "      " "      " "     
## 42  ( 1 ) " "      " "      " "      " "      " "      " "     
## 43  ( 1 ) " "      " "      " "      " "      " "      " "     
## 44  ( 1 ) " "      " "      " "      " "      " "      " "     
## 45  ( 1 ) " "      " "      " "      " "      " "      " "     
## 46  ( 1 ) " "      " "      " "      " "      " "      " "     
## 47  ( 1 ) " "      " "      " "      " "      " "      " "     
## 48  ( 1 ) " "      " "      " "      " "      " "      " "     
## 49  ( 1 ) " "      " "      " "      " "      " "      " "     
## 50  ( 1 ) " "      " "      " "      " "      " "      " "     
## 51  ( 1 ) " "      " "      " "      " "      " "      " "     
## 52  ( 1 ) " "      " "      " "      " "      " "      "*"     
## 53  ( 1 ) " "      " "      " "      " "      " "      "*"     
## 54  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 55  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 56  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 57  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 58  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 59  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 60  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 61  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 62  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 63  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 64  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 65  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 66  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 67  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 68  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 69  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 70  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 71  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 72  ( 1 ) " "      " "      "*"      " "      "*"      "*"     
## 73  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 74  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 75  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 76  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 77  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"
reg.summary$rsq
##  [1] 0.1792078 0.1964987 0.2011161 0.2042560 0.2068072 0.2080653 0.2091026
##  [8] 0.2099496 0.2109281 0.2117098 0.2122451 0.2126110 0.2129357 0.2132283
## [15] 0.2135256 0.2138151 0.2141021 0.2143693 0.2146069 0.2147869 0.2149185
## [22] 0.2150206 0.2151150 0.2152053 0.2152829 0.2153583 0.2153999 0.2154358
## [29] 0.2154727 0.2155123 0.2155411 0.2155676 0.2155885 0.2156101 0.2156289
## [36] 0.2156432 0.2156573 0.2156700 0.2156819 0.2156937 0.2157051 0.2157162
## [43] 0.2157266 0.2157358 0.2157433 0.2157496 0.2157559 0.2157619 0.2157675
## [50] 0.2157729 0.2157778 0.2157824 0.2157868 0.2157897 0.2157924 0.2157950
## [57] 0.2157970 0.2157980 0.2157989 0.2157997 0.2158005 0.2158012 0.2158016
## [64] 0.2158020 0.2158024 0.2158028 0.2158031 0.2158033 0.2158035 0.2158037
## [71] 0.2158038 0.2158039 0.2158039 0.2158039 0.2158040 0.2158040 0.2158040
plot(reg.summary$rsq,xlab="Number of variables", ylab="R-Square", type="l")
points(1:83,reg.summary$rsq[1:83], col="red",cex=2,pch=20)

Figure 2A: Plot between Number of variables and R-Square (forward variable selection method)

set.seed(18)
regfit.bkw <- regsubsets (days_diff~., data=days_diff_training, nvmax =83, method ="backward")
## Warning in leaps.setup(x, y, wt = wt, nbest = nbest, nvmax = nvmax,
## force.in = force.in, : 6 linear dependencies found
## Reordering variables and trying again:
reg.summary.bkw <- summary(regfit.bkw)

summary(regfit.bkw)
## Subset selection object
## Call: regsubsets.formula(days_diff ~ ., data = days_diff_training, 
##     nvmax = 83, method = "backward")
## 83 Variables  (and intercept)
##             Forced in Forced out
## visit_level     FALSE      FALSE
## cuisine_id      FALSE      FALSE
## score           FALSE      FALSE
## `C_000 `        FALSE      FALSE
## `C_02A `        FALSE      FALSE
## `C_02B `        FALSE      FALSE
## `C_02C `        FALSE      FALSE
## `C_02D `        FALSE      FALSE
## `C_02E `        FALSE      FALSE
## `C_02F `        FALSE      FALSE
## `C_02G `        FALSE      FALSE
## `C_02H `        FALSE      FALSE
## `C_02I `        FALSE      FALSE
## `C_03A `        FALSE      FALSE
## `C_03B `        FALSE      FALSE
## `C_03C `        FALSE      FALSE
## `C_03D `        FALSE      FALSE
## `C_03E `        FALSE      FALSE
## `C_03G `        FALSE      FALSE
## `C_04A `        FALSE      FALSE
## `C_04B `        FALSE      FALSE
## `C_04C `        FALSE      FALSE
## `C_04D `        FALSE      FALSE
## `C_04E `        FALSE      FALSE
## `C_04F `        FALSE      FALSE
## `C_04G `        FALSE      FALSE
## `C_04H `        FALSE      FALSE
## `C_04I `        FALSE      FALSE
## `C_04J `        FALSE      FALSE
## `C_04K `        FALSE      FALSE
## `C_04L `        FALSE      FALSE
## `C_04M `        FALSE      FALSE
## `C_04N `        FALSE      FALSE
## `C_04O `        FALSE      FALSE
## `C_05A `        FALSE      FALSE
## `C_05B `        FALSE      FALSE
## `C_05C `        FALSE      FALSE
## `C_05D `        FALSE      FALSE
## `C_05E `        FALSE      FALSE
## `C_05F `        FALSE      FALSE
## `C_05H `        FALSE      FALSE
## `C_05I `        FALSE      FALSE
## `C_06A `        FALSE      FALSE
## `C_06B `        FALSE      FALSE
## `C_06C `        FALSE      FALSE
## `C_06D `        FALSE      FALSE
## `C_06E `        FALSE      FALSE
## `C_06F `        FALSE      FALSE
## `C_06G `        FALSE      FALSE
## `C_06H `        FALSE      FALSE
## `C_06I `        FALSE      FALSE
## `C_07A `        FALSE      FALSE
## `C_08A `        FALSE      FALSE
## `C_08B `        FALSE      FALSE
## `C_08C `        FALSE      FALSE
## `C_09A `        FALSE      FALSE
## `C_09B `        FALSE      FALSE
## `C_09C `        FALSE      FALSE
## `C_10A `        FALSE      FALSE
## `C_10B `        FALSE      FALSE
## `C_10C `        FALSE      FALSE
## `C_10D `        FALSE      FALSE
## `C_10E `        FALSE      FALSE
## `C_10F `        FALSE      FALSE
## `C_10G `        FALSE      FALSE
## `C_10H `        FALSE      FALSE
## `C_10I `        FALSE      FALSE
## `C_10J `        FALSE      FALSE
## `C_10K `        FALSE      FALSE
## `C_18F `        FALSE      FALSE
## `C_18G `        FALSE      FALSE
## `C_20A `        FALSE      FALSE
## `C_20E `        FALSE      FALSE
## `C_20F `        FALSE      FALSE
## `C_22A `        FALSE      FALSE
## `C_22C `        FALSE      FALSE
## `C_99B `        FALSE      FALSE
## `C_02J `        FALSE      FALSE
## `C_03F `        FALSE      FALSE
## `C_16B `        FALSE      FALSE
## `C_18D `        FALSE      FALSE
## `C_18I `        FALSE      FALSE
## `C_20D `        FALSE      FALSE
## 1 subsets of each size up to 77
## Selection Algorithm: backward
##           visit_level cuisine_id score `C_000 ` `C_02A ` `C_02B ` `C_02C `
## 1  ( 1 )  " "         " "        "*"   " "      " "      " "      " "     
## 2  ( 1 )  " "         " "        "*"   "*"      " "      " "      " "     
## 3  ( 1 )  " "         " "        "*"   "*"      " "      " "      " "     
## 4  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 5  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 6  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 7  ( 1 )  "*"         " "        "*"   "*"      " "      " "      " "     
## 8  ( 1 )  "*"         " "        "*"   "*"      " "      "*"      " "     
## 9  ( 1 )  "*"         " "        "*"   "*"      " "      "*"      " "     
## 10  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 11  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 12  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 13  ( 1 ) "*"         " "        "*"   "*"      " "      "*"      " "     
## 14  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 15  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 16  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 17  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 18  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 19  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 20  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 21  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 22  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 23  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 24  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 25  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 26  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 27  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 28  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 29  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 30  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 31  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 32  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 33  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 34  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 35  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 36  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 37  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 38  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 39  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 40  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 41  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 42  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 43  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 44  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 45  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 46  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 47  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 48  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 49  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 50  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 51  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 52  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 53  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 54  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 55  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 56  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 57  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 58  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 59  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 60  ( 1 ) "*"         "*"        "*"   "*"      " "      "*"      " "     
## 61  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 62  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 63  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 64  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 65  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 71  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 72  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 73  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 74  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      " "     
## 75  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"         "*"        "*"   "*"      "*"      "*"      "*"     
##           `C_02D ` `C_02E ` `C_02F ` `C_02G ` `C_02H ` `C_02I ` `C_02J `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      "*"      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 32  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 33  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 34  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 35  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 36  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 37  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 41  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 42  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 43  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 44  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 45  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 46  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 47  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 48  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 49  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 50  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 51  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 52  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 53  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 54  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 55  ( 1 ) " "      "*"      "*"      "*"      "*"      " "      " "     
## 56  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 57  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 58  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 59  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 60  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 61  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 62  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 63  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 64  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 65  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
##           `C_03A ` `C_03B ` `C_03C ` `C_03D ` `C_03E ` `C_03F ` `C_03G `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      "*"      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 41  ( 1 ) " "      " "      " "      "*"      "*"      " "      " "     
## 42  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 43  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 44  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 45  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 46  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 47  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 48  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 49  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 50  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 51  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 52  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 53  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 54  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 55  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 56  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 57  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 58  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 59  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 60  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 61  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 62  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 63  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 64  ( 1 ) "*"      " "      "*"      "*"      "*"      " "      " "     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      "*"     
##           `C_04A ` `C_04B ` `C_04C ` `C_04D ` `C_04E ` `C_04F ` `C_04G `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 28  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 29  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 30  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 31  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 32  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 33  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 34  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 35  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 36  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 37  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 38  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 39  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 40  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 41  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 42  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 43  ( 1 ) "*"      " "      "*"      " "      " "      " "      " "     
## 44  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 45  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 46  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 47  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 48  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 49  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 50  ( 1 ) "*"      "*"      "*"      " "      " "      " "      " "     
## 51  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 52  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 53  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 54  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 55  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 56  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 57  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 58  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 59  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 60  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 61  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 62  ( 1 ) "*"      "*"      "*"      "*"      " "      " "      " "     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      " "      " "     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      " "     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_04H ` `C_04I ` `C_04J ` `C_04K ` `C_04L ` `C_04M ` `C_04N `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      "*"      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      "*"      " "      "*"     
## 7  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 8  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 9  ( 1 )  " "      " "      " "      " "      "*"      "*"      "*"     
## 10  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 11  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 12  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 13  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 14  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 15  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 16  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 17  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 18  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 19  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 20  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 21  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 22  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 23  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 24  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 25  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 26  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 27  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 28  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 47  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 48  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 49  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_04O ` `C_05A ` `C_05B ` `C_05C ` `C_05D ` `C_05E ` `C_05F `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 34  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 35  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 36  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 37  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 38  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 39  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 40  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 41  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 42  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 43  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 44  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 45  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 46  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 47  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 48  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 49  ( 1 ) " "      " "      " "      "*"      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      " "      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      " "      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_05H ` `C_05I ` `C_06A ` `C_06B ` `C_06C ` `C_06D ` `C_06E `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      "*"      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      "*"      "*"     
## 19  ( 1 ) " "      " "      " "      " "      "*"      "*"      "*"     
## 20  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 21  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 22  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 23  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 24  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 25  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 26  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 27  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 28  ( 1 ) " "      " "      "*"      " "      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 58  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      " "      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_06F ` `C_06G ` `C_06H ` `C_06I ` `C_07A ` `C_08A ` `C_08B `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) "*"      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) "*"      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 25  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 26  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 27  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 28  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 29  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 30  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 31  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 32  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 33  ( 1 ) "*"      " "      " "      " "      "*"      "*"      "*"     
## 34  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 35  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 36  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 37  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 38  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 39  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      " "      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_08C ` `C_09A ` `C_09B ` `C_09C ` `C_10A ` `C_10B ` `C_10C `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 32  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 33  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 34  ( 1 ) " "      " "      "*"      " "      "*"      " "      " "     
## 35  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 36  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 37  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 38  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 39  ( 1 ) " "      " "      "*"      "*"      "*"      " "      " "     
## 40  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 41  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 42  ( 1 ) " "      " "      "*"      "*"      "*"      " "      "*"     
## 43  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 44  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 45  ( 1 ) " "      " "      "*"      "*"      "*"      "*"      "*"     
## 46  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 47  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 48  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 49  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 50  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 51  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 52  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 53  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 54  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 55  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 56  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 57  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 58  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 59  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 60  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 61  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 62  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 63  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 64  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 65  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 66  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 67  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 68  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 69  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 70  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 71  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_10D ` `C_10E ` `C_10F ` `C_10G ` `C_10H ` `C_10I ` `C_10J `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      "*"      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      "*"      " "      " "      " "      " "     
## 22  ( 1 ) " "      "*"      "*"      " "      " "      " "      " "     
## 23  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 24  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 25  ( 1 ) " "      "*"      "*"      " "      " "      " "      "*"     
## 26  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 27  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 28  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 29  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 30  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 31  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 32  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 33  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 34  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 35  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 36  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 37  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 38  ( 1 ) " "      "*"      "*"      " "      "*"      " "      "*"     
## 39  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 40  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 41  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 42  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 43  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 44  ( 1 ) " "      "*"      "*"      " "      "*"      "*"      "*"     
## 45  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 46  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 47  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 48  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 49  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 50  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 51  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 52  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 53  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 54  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 55  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 56  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 57  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 58  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 59  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 60  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 61  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 62  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 63  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 64  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 65  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 66  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 67  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 68  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 69  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 70  ( 1 ) "*"      "*"      "*"      " "      "*"      "*"      "*"     
## 71  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 72  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 73  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 74  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 75  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 76  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
## 77  ( 1 ) "*"      "*"      "*"      "*"      "*"      "*"      "*"     
##           `C_10K ` `C_16B ` `C_18D ` `C_18F ` `C_18G ` `C_18I ` `C_20A `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 38  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 39  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 40  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 41  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 42  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 43  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 44  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 45  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 46  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 47  ( 1 ) " "      " "      " "      " "      " "      " "      " "     
## 48  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 49  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 50  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 51  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 52  ( 1 ) " "      " "      " "      " "      "*"      " "      " "     
## 53  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 54  ( 1 ) " "      " "      " "      " "      "*"      " "      "*"     
## 55  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 56  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 57  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 58  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 59  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 60  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 61  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 62  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 63  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 64  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 65  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 66  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 67  ( 1 ) "*"      " "      " "      " "      "*"      " "      "*"     
## 68  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 69  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 70  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 71  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 72  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 73  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 74  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 75  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 76  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
## 77  ( 1 ) "*"      " "      " "      "*"      "*"      " "      "*"     
##           `C_20D ` `C_20E ` `C_20F ` `C_22A ` `C_22C ` `C_99B `
## 1  ( 1 )  " "      " "      " "      " "      " "      " "     
## 2  ( 1 )  " "      " "      " "      " "      " "      " "     
## 3  ( 1 )  " "      " "      " "      " "      " "      " "     
## 4  ( 1 )  " "      " "      " "      " "      " "      " "     
## 5  ( 1 )  " "      " "      " "      " "      " "      " "     
## 6  ( 1 )  " "      " "      " "      " "      " "      " "     
## 7  ( 1 )  " "      " "      " "      " "      " "      " "     
## 8  ( 1 )  " "      " "      " "      " "      " "      " "     
## 9  ( 1 )  " "      " "      " "      " "      " "      " "     
## 10  ( 1 ) " "      " "      " "      " "      " "      " "     
## 11  ( 1 ) " "      " "      " "      " "      " "      " "     
## 12  ( 1 ) " "      " "      " "      " "      " "      " "     
## 13  ( 1 ) " "      " "      " "      " "      " "      " "     
## 14  ( 1 ) " "      " "      " "      " "      " "      " "     
## 15  ( 1 ) " "      " "      " "      " "      " "      " "     
## 16  ( 1 ) " "      " "      " "      " "      " "      " "     
## 17  ( 1 ) " "      " "      " "      " "      " "      " "     
## 18  ( 1 ) " "      " "      " "      " "      " "      " "     
## 19  ( 1 ) " "      " "      " "      " "      " "      " "     
## 20  ( 1 ) " "      " "      " "      " "      " "      " "     
## 21  ( 1 ) " "      " "      " "      " "      " "      " "     
## 22  ( 1 ) " "      " "      " "      " "      " "      " "     
## 23  ( 1 ) " "      " "      " "      " "      " "      " "     
## 24  ( 1 ) " "      " "      " "      " "      " "      " "     
## 25  ( 1 ) " "      " "      " "      " "      " "      " "     
## 26  ( 1 ) " "      " "      " "      " "      " "      " "     
## 27  ( 1 ) " "      " "      " "      " "      " "      " "     
## 28  ( 1 ) " "      " "      " "      " "      " "      " "     
## 29  ( 1 ) " "      " "      " "      " "      " "      " "     
## 30  ( 1 ) " "      " "      " "      " "      " "      " "     
## 31  ( 1 ) " "      " "      " "      " "      " "      " "     
## 32  ( 1 ) " "      " "      " "      " "      " "      " "     
## 33  ( 1 ) " "      " "      " "      " "      " "      " "     
## 34  ( 1 ) " "      " "      " "      " "      " "      " "     
## 35  ( 1 ) " "      " "      " "      " "      " "      " "     
## 36  ( 1 ) " "      " "      " "      " "      " "      " "     
## 37  ( 1 ) " "      " "      " "      " "      " "      " "     
## 38  ( 1 ) " "      " "      " "      " "      " "      " "     
## 39  ( 1 ) " "      " "      " "      " "      " "      " "     
## 40  ( 1 ) " "      " "      " "      " "      " "      " "     
## 41  ( 1 ) " "      " "      " "      " "      " "      " "     
## 42  ( 1 ) " "      " "      " "      " "      " "      " "     
## 43  ( 1 ) " "      " "      " "      " "      " "      " "     
## 44  ( 1 ) " "      " "      " "      " "      " "      " "     
## 45  ( 1 ) " "      " "      " "      " "      " "      " "     
## 46  ( 1 ) " "      " "      " "      " "      " "      " "     
## 47  ( 1 ) " "      " "      " "      " "      " "      " "     
## 48  ( 1 ) " "      " "      " "      " "      " "      " "     
## 49  ( 1 ) " "      " "      " "      " "      " "      " "     
## 50  ( 1 ) " "      " "      " "      " "      " "      " "     
## 51  ( 1 ) " "      " "      " "      " "      " "      " "     
## 52  ( 1 ) " "      " "      " "      " "      " "      "*"     
## 53  ( 1 ) " "      " "      " "      " "      " "      "*"     
## 54  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 55  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 56  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 57  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 58  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 59  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 60  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 61  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 62  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 63  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 64  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 65  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 66  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 67  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 68  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 69  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 70  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 71  ( 1 ) " "      " "      "*"      " "      " "      "*"     
## 72  ( 1 ) " "      " "      "*"      " "      "*"      "*"     
## 73  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 74  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 75  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 76  ( 1 ) " "      "*"      "*"      " "      "*"      "*"     
## 77  ( 1 ) " "      "*"      "*"      "*"      "*"      "*"
reg.summary.bkw$rsq
##  [1] 0.1792078 0.1964987 0.2011161 0.2042560 0.2068072 0.2080653 0.2091026
##  [8] 0.2099496 0.2109281 0.2117098 0.2122451 0.2126110 0.2129357 0.2132283
## [15] 0.2135256 0.2138151 0.2141021 0.2143693 0.2146069 0.2147869 0.2149185
## [22] 0.2150206 0.2151150 0.2152053 0.2152829 0.2153583 0.2153999 0.2154358
## [29] 0.2154727 0.2155123 0.2155411 0.2155676 0.2155885 0.2156101 0.2156289
## [36] 0.2156432 0.2156573 0.2156700 0.2156819 0.2156937 0.2157051 0.2157162
## [43] 0.2157266 0.2157358 0.2157433 0.2157496 0.2157559 0.2157619 0.2157675
## [50] 0.2157729 0.2157778 0.2157824 0.2157868 0.2157897 0.2157924 0.2157950
## [57] 0.2157970 0.2157980 0.2157989 0.2157997 0.2158005 0.2158012 0.2158016
## [64] 0.2158020 0.2158024 0.2158028 0.2158031 0.2158033 0.2158035 0.2158037
## [71] 0.2158038 0.2158039 0.2158039 0.2158039 0.2158040 0.2158040 0.2158040
plot(reg.summary.bkw$rsq,xlab="Number of variables", ylab="R-Square", type="l")
points(1:83,reg.summary.bkw$rsq[1:83], col="red",cex=2,pch=20)

Figure 2B: Plot between Number of variables and R-Square (backward variable selection)

The Figure-2A/B above display that after approximately 12 variables, there is not really significant improvement in the R-Square (Greater the R-Square, better is the model). Now we will evaluate various models on the following 12 variables to improve the R-Square further. The following variables are obtained from the display obtained from the command “summary(regfit.fwd and regfit.bkw)”. At the 12th row, where ever we see an “*“, those 12 variables are selected. Here are those 12 variables. The same 12 variables are obtained in both forward and backward selection

The following R code gives the description of the violation codes (which will be used as variables for evaluating days_diff)

rbind(
       violation_data[violation_data$code == "000 ",],
       violation_data[violation_data$code == "10F ",],
       violation_data[violation_data$code == "06F ",],
       violation_data[violation_data$code == "04A ",],
       violation_data[violation_data$code == "04H ",],
       violation_data[violation_data$code == "04L ",],
       violation_data[violation_data$code == "04M ",],
       violation_data[violation_data$code == "04N ",],
       violation_data[violation_data$code == "02B ",],
       violation_data[violation_data$code == "02G ",]
)
##    code
## 45 000 
## 55 10F 
## 81 06F 
## 4  04A 
## 70 04H 
## 56 04L 
## 94 04M 
## 66 04N 
## 64 02B 
## 29 02G 
##                                                                                                                                                                                                                                                                                             description
## 45                                                                                                                                                                                                                                                                                           NONE FOUND
## 55                      Non-food contact surface improperly constructed. Unacceptable material used. Non-food contact surface or equipment improperly maintained and/or not properly sealed, raised, spaced or movable to allow accessibility for cleaning on all sides, above and underneath the unit.
## 81                                                                                                                                                                                                                                           Wiping cloths soiled or not stored in sanitizing solution.
## 4                                                                                                                                                                                                                                Food Protection Certificate not held by supervisor of food operations.
## 70                                                                                                                                                                       Raw, cooked or prepared food is adulterated, contaminated, cross-contaminated, or not discarded in accordance with HACCP plan.
## 56                                                                                                                                                                                                                      Evidence of mice or live mice present in facility's food and/or non-food areas.
## 94                                                                                                                                                                                                                                       Live roaches present in facility's food and/or non-food areas.
## 66 Filth flies or food/refuse/sewage-associated (FRSA) flies present in facility\032s food and/or non-food areas. Filth flies include house flies, little house flies, blow flies, bottle flies and flesh flies. Food/refuse/sewage-associated flies include fruit flies, drain flies and Phorid flies.
## 64                                                                                                                                                                                                                                                          Hot food item not held at or above 140º F.
## 29                                                                                                                                                                   Cold food item held above 41º F (smoked fish and reduced oxygen packaged foods above 38 ºF) except during necessary preparation.
##                                                 level
## 45 Not Applicable                                    
## 55 Not Critical                                      
## 81 Critical                                          
## 4  Critical                                          
## 70 Critical                                          
## 56 Critical                                          
## 94 Critical                                          
## 66 Critical                                          
## 64 Critical                                          
## 29 Critical

Let us pair plot the 12 variables along with the days_diff. This will help us, if we have any non-linear relationship with the days_diff and any of the 5 variables identified above.

plot(days_diff_training$visit_level,days_diff_training$days_diff,xlab="Visit Level",ylab="days_diff",col="green")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$visit_level)
abline(temp_lm,col="red")

plot(days_diff_training$score,days_diff_training$days_diff,xlab="Score",ylab="days_diff",col="green")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$score)
abline(temp_lm,col="red")

plot(days_diff_training$C_000,days_diff_training$days_diff,xlab="C_000",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_000)
abline(temp_lm,col="red")

plot(days_diff_training$C_04L,days_diff_training$days_diff,xlab="C_04L",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_04L)
abline(temp_lm,col="red")

plot(days_diff_training$C_10F,days_diff_training$days_diff,xlab="C_10F",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_10F)
abline(temp_lm,col="red")

plot(days_diff_training$C_06F,days_diff_training$days_diff,xlab="C_06F",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_06F)
abline(temp_lm,col="red")

plot(days_diff_training$C_04A,days_diff_training$days_diff,xlab="C_04A",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_04A)
abline(temp_lm,col="red")

plot(days_diff_training$C_04H,days_diff_training$days_diff,xlab="C_04H",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_04H)
abline(temp_lm,col="red")

plot(days_diff_training$C_04M,days_diff_training$days_diff,xlab="C_04M",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_04M)
abline(temp_lm,col="red")

plot(days_diff_training$C_04N,days_diff_training$days_diff,xlab="C_04N",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_04N)
abline(temp_lm,col="red")

plot(days_diff_training$C_02B,days_diff_training$days_diff,xlab="C_02B",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_02B)
abline(temp_lm,col="red")

plot(days_diff_training$C_02G,days_diff_training$days_diff,xlab="C_02G",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_02G)
abline(temp_lm,col="red")

Figure 3: Graphs showing the plots between days_diff and other selected variables

From the above graphs display we can conclude that on an average, only score variable and days_diff have some strong relationship (inversely proportional). All other variables are not strongly related to days_diff variable. So let us just consider the score variable to predict the days_diff. This might be a very simple assumption for modelling a complex phenomenon, and thus increasing the bias. But as per the business requirement (R1) we have to find the most important variable to predict the days_diff value.

Cross validation to select a model with least Cross Validation error

Now I will use the Cross Validation to get the Cross Validation error for a simple linear relationship and polynomial relationship between score and days_diff. We will consider the degree of polynomial for which the CV error is minimal. We will just run this testing on score variable only.

NOTE: This R Code may run for a while

set.seed(18)
#par(mfrow=c(1,1))

cv.error.5 <- rep(0,5)
for (i in 1:5)
  {
glm.fit <- glm((days_diff_training$days_diff~(I(days_diff_training$score^i))))

#round(predict(glm.fit,data.frame(days_diff_training$score[1:10]))[1:10])
#days_diff_training$days_diff[1:10]
cv.error.5[i] <-  cv.glm(days_diff_training,glm.fit,K=5)$delta[1] 

print(cv.error.5[i])
}
## [1] 13604.2
## [1] 12724.35
## [1] 12026.98
## [1] 11702.26
## [1] 11599.74
plot(1:5,cv.error.5,xlab="Degrees of freedom of score", ylab="CV Error", type="l")
points(1:5,cv.error.5, col="red",cex=2,pch=20)

Figure 4: Cross validation error vs. “Score” variable’s degree of freedom

From Figure 4, after 4 degrees of freedom, there is no significant decrease in the CV error. Hence we will consider score with 4 degrees of freedom to model the days_diff variable (days_diff is nothing but number of days from the current inspection date, after which the next inspection could happen). But note that the CV error of 11500 is still very high, but we can at least conclude that lesser the score larger the gap between successive inspections. The model we will consider is given below:

  glm.fit <- glm((days_diff_training$days_diff~(I(days_diff_training$score^4))))

Predicting the factors infulencing “closure” of restaurant

Using logistic regression to predict if a restaurant is closed

R Code to prepare required data frames (preparation of 2 data frames for training and testing):

dim(inspection_spread)
## [1] 160536     90
names(inspection_spread)
##  [1] "visit_level"     "restaurant_id"   "cuisine_id"     
##  [4] "inspection_date" "closed"          "score"          
##  [7] "C_000 "          "C_02A "          "C_02B "         
## [10] "C_02C "          "C_02D "          "C_02E "         
## [13] "C_02F "          "C_02G "          "C_02H "         
## [16] "C_02I "          "C_02J "          "C_03A "         
## [19] "C_03B "          "C_03C "          "C_03D "         
## [22] "C_03E "          "C_03F "          "C_03G "         
## [25] "C_04A "          "C_04B "          "C_04C "         
## [28] "C_04D "          "C_04E "          "C_04F "         
## [31] "C_04G "          "C_04H "          "C_04I "         
## [34] "C_04J "          "C_04K "          "C_04L "         
## [37] "C_04M "          "C_04N "          "C_04O "         
## [40] "C_05A "          "C_05B "          "C_05C "         
## [43] "C_05D "          "C_05E "          "C_05F "         
## [46] "C_05H "          "C_05I "          "C_06A "         
## [49] "C_06B "          "C_06C "          "C_06D "         
## [52] "C_06E "          "C_06F "          "C_06G "         
## [55] "C_06H "          "C_06I "          "C_07A "         
## [58] "C_08A "          "C_08B "          "C_08C "         
## [61] "C_09A "          "C_09B "          "C_09C "         
## [64] "C_10A "          "C_10B "          "C_10C "         
## [67] "C_10D "          "C_10E "          "C_10F "         
## [70] "C_10G "          "C_10H "          "C_10I "         
## [73] "C_10J "          "C_10K "          "C_16B "         
## [76] "C_18D "          "C_18F "          "C_18G "         
## [79] "C_18I "          "C_20A "          "C_20D "         
## [82] "C_20E "          "C_20F "          "C_22A "         
## [85] "C_22C "          "C_99B "          "closed_next"    
## [88] "days_diff"       "year"            "month"
closed_training <- inspection_spread[(inspection_spread$year!=2015),c(-2,-4,-87,-88,-89,-90)]
closed_testing <- inspection_spread[(inspection_spread$year==2015),c(-2,-4,-87,-88,-89,-90)]

The R Code for logistic regression follows:

glm.fit=glm(closed~.,
data=closed_training,family =binomial )

summary(glm.fit)
## 
## Call:
## glm(formula = closed ~ ., family = binomial, data = closed_training)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -3.6261  -0.1245  -0.0652  -0.0512   3.9603  
## 
## Coefficients: (3 not defined because of singularities)
##               Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -6.803e+00  8.646e-02 -78.690  < 2e-16 ***
## visit_level  6.230e-02  8.272e-03   7.531 5.04e-14 ***
## cuisine_id   2.284e-03  1.018e-03   2.243 0.024890 *  
## score        1.619e-01  2.420e-03  66.895  < 2e-16 ***
## `C_000 `    -9.691e-01  4.526e-01  -2.141 0.032265 *  
## `C_02A `    -2.615e+00  5.550e-01  -4.712 2.45e-06 ***
## `C_02B `    -1.100e+00  6.588e-02 -16.695  < 2e-16 ***
## `C_02C `    -3.930e-01  3.613e-01  -1.088 0.276695    
## `C_02D `     1.588e-01  1.114e+00   0.143 0.886641    
## `C_02E `     1.122e+00  1.544e+00   0.727 0.467290    
## `C_02F `    -1.060e+01  4.471e+02  -0.024 0.981091    
## `C_02G `    -1.183e+00  5.976e-02 -19.791  < 2e-16 ***
## `C_02H `    -1.084e+00  1.433e-01  -7.570 3.73e-14 ***
## `C_02I `    -1.997e+00  1.399e+00  -1.428 0.153406    
## `C_02J `    -1.073e+01  1.455e+03  -0.007 0.994120    
## `C_03A `    -9.141e-01  2.929e-01  -3.121 0.001805 ** 
## `C_03B `    -1.722e+00  4.925e-01  -3.497 0.000470 ***
## `C_03C `    -2.034e+00  6.512e-01  -3.124 0.001784 ** 
## `C_03D `    -1.201e+01  1.656e+02  -0.073 0.942183    
## `C_03E `    -1.612e+00  1.026e+00  -1.571 0.116172    
## `C_03F `            NA         NA      NA       NA    
## `C_03G `    -1.296e+00  2.759e+00  -0.470 0.638383    
## `C_04A `    -1.057e+00  9.010e-02 -11.726  < 2e-16 ***
## `C_04B `     1.546e-01  6.925e-01   0.223 0.823332    
## `C_04C `    -1.257e+00  1.402e-01  -8.965  < 2e-16 ***
## `C_04D `    -7.523e-01  3.010e-01  -2.499 0.012445 *  
## `C_04E `    -5.162e-01  3.926e-01  -1.315 0.188585    
## `C_04F `     1.510e+00  3.937e-01   3.837 0.000125 ***
## `C_04G `    -3.353e-01  2.152e+00  -0.156 0.876207    
## `C_04H `    -5.851e-01  6.783e-02  -8.625  < 2e-16 ***
## `C_04I `     2.959e-01  1.145e+00   0.258 0.796160    
## `C_04J `    -1.098e+00  9.939e-02 -11.050  < 2e-16 ***
## `C_04K `     9.344e-01  9.746e-02   9.588  < 2e-16 ***
## `C_04L `     6.625e-01  5.826e-02  11.371  < 2e-16 ***
## `C_04M `     9.569e-01  5.749e-02  16.644  < 2e-16 ***
## `C_04N `    -6.007e-01  6.310e-02  -9.520  < 2e-16 ***
## `C_04O `    -6.124e-01  3.219e-01  -1.902 0.057111 .  
## `C_05A `     1.084e+00  3.109e-01   3.488 0.000487 ***
## `C_05B `    -2.083e-01  6.215e-01  -0.335 0.737500    
## `C_05C `    -2.322e+00  1.319e+00  -1.761 0.078274 .  
## `C_05D `    -1.527e+00  1.138e-01 -13.415  < 2e-16 ***
## `C_05E `     5.579e-01  3.264e-01   1.709 0.087408 .  
## `C_05F `     7.564e-01  1.387e-01   5.455 4.89e-08 ***
## `C_05H `    -1.256e+00  2.174e-01  -5.776 7.63e-09 ***
## `C_05I `    -1.146e+01  8.150e+02  -0.014 0.988785    
## `C_06A `    -6.293e-01  9.298e-02  -6.768 1.31e-11 ***
## `C_06B `    -4.008e-01  1.442e-01  -2.780 0.005443 ** 
## `C_06C `    -7.518e-01  6.318e-02 -11.899  < 2e-16 ***
## `C_06D `    -1.008e+00  6.698e-02 -15.052  < 2e-16 ***
## `C_06E `    -9.231e-01  9.737e-02  -9.480  < 2e-16 ***
## `C_06F `    -1.006e+00  1.080e-01  -9.313  < 2e-16 ***
## `C_06G `    -2.574e+00  6.545e-01  -3.933 8.38e-05 ***
## `C_06H `    -6.788e-01  1.145e+00  -0.593 0.553182    
## `C_06I `    -2.106e+01  2.612e+02  -0.081 0.935754    
## `C_07A `    -1.860e+00  3.473e-01  -5.357 8.47e-08 ***
## `C_08A `    -3.212e-01  7.226e-02  -4.446 8.77e-06 ***
## `C_08B `    -4.461e-01  1.804e-01  -2.473 0.013406 *  
## `C_08C `    -3.170e-01  9.975e-02  -3.178 0.001484 ** 
## `C_09A `    -4.758e-01  2.382e-01  -1.997 0.045782 *  
## `C_09B `    -1.586e-01  1.338e-01  -1.185 0.235927    
## `C_09C `    -6.261e-01  1.382e-01  -4.530 5.89e-06 ***
## `C_10A `    -4.700e-01  1.448e-01  -3.247 0.001167 ** 
## `C_10B `    -4.649e-01  6.548e-02  -7.100 1.25e-12 ***
## `C_10C `    -1.803e-01  4.422e-01  -0.408 0.683533    
## `C_10D `    -4.176e-01  1.326e-01  -3.149 0.001639 ** 
## `C_10E `    -4.817e-01  1.920e-01  -2.509 0.012095 *  
## `C_10F `    -4.492e-01  5.405e-02  -8.311  < 2e-16 ***
## `C_10G `     4.054e-01  6.372e-01   0.636 0.524646    
## `C_10H `    -9.471e-01  1.341e-01  -7.062 1.64e-12 ***
## `C_10I `    -2.373e-01  1.690e-01  -1.405 0.160092    
## `C_10J `    -6.581e-01  2.298e-01  -2.864 0.004185 ** 
## `C_10K `    -9.009e+00  1.455e+03  -0.006 0.995061    
## `C_16B `            NA         NA      NA       NA    
## `C_18D `    -1.130e+01  1.455e+03  -0.008 0.993806    
## `C_18F `    -8.428e+00  1.455e+03  -0.006 0.995380    
## `C_18G `    -8.392e+00  1.455e+03  -0.006 0.995399    
## `C_18I `            NA         NA      NA       NA    
## `C_20A `    -8.999e+00  1.455e+03  -0.006 0.995067    
## `C_20D `    -4.311e-01  2.058e+03   0.000 0.999833    
## `C_20E `     2.298e+00  2.129e+00   1.079 0.280552    
## `C_20F `    -8.887e+00  1.455e+03  -0.006 0.995128    
## `C_22A `     9.018e-01  1.656e+03   0.001 0.999566    
## `C_22C `    -1.020e+01  7.901e+02  -0.013 0.989701    
## `C_99B `    -6.207e-01  4.792e-01  -1.295 0.195179    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 25502  on 138191  degrees of freedom
## Residual deviance: 13588  on 138111  degrees of freedom
## AIC: 13750
## 
## Number of Fisher Scoring iterations: 14

The above display shows lots of variables, have P-Value greater than 0.5%. Let us filter them:

rownames(summary(glm.fit)$coeff[which(summary(glm.fit)$coeff[,4]<0.005),])[-1]
##  [1] "visit_level" "score"       "`C_02A `"    "`C_02B `"    "`C_02G `"   
##  [6] "`C_02H `"    "`C_03A `"    "`C_03B `"    "`C_03C `"    "`C_04A `"   
## [11] "`C_04C `"    "`C_04F `"    "`C_04H `"    "`C_04J `"    "`C_04K `"   
## [16] "`C_04L `"    "`C_04M `"    "`C_04N `"    "`C_05A `"    "`C_05D `"   
## [21] "`C_05F `"    "`C_05H `"    "`C_06A `"    "`C_06C `"    "`C_06D `"   
## [26] "`C_06E `"    "`C_06F `"    "`C_06G `"    "`C_07A `"    "`C_08A `"   
## [31] "`C_08C `"    "`C_09C `"    "`C_10A `"    "`C_10B `"    "`C_10D `"   
## [36] "`C_10F `"    "`C_10H `"    "`C_10J `"

Creating another model with just the above variables.

glm.fit=glm(closed~visit_level+score +`C_02A `+`C_02B `+`C_02G `+`C_02H `+`C_03A `+`C_03B `+`C_03C `+`C_04A `+`C_04C `+`C_04F `+`C_04H `+`C_04J `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_05A `+`C_05D `+`C_05F `+`C_05H `+`C_06A `+`C_06C `+`C_06D `+`C_06E `+`C_06F `+`C_06G `+`C_07A `+`C_08A `+`C_08C `+`C_09C `+`C_10A `+`C_10B `+`C_10D `+`C_10F `+`C_10H `+`C_10J `,
data=closed_training,family =binomial )

Logistic model evaluation

We will evaluate the model on the closed_training data.

#The following commented R Code is tested on closed_testing. Since we are interested on training


#glm.probs <- predict(glm.fit,type="response")
#closed_training[which(glm.probs > 0.5),]
#contrasts(as.factor(closed_training$closed))

#glm.pred <- rep(0,nrow(closed_training))
#glm.pred[glm.probs>0.7] <- 1
#prop.table(table(glm.pred,closed_training$closed))
#mean(glm.pred==closed_training$closed)

#Evaluating the model performance on test data set:

glm.probs <- predict(glm.fit,closed_testing,type="response")
glm.pred <- rep(0,nrow(closed_testing))
glm.pred[glm.probs>0.7] <- 1
prop.table(table(glm.pred,closed_testing$closed))
##         
## glm.pred           0           1
##        0 0.985230934 0.011412460
##        1 0.001208378 0.002148228
mean(glm.pred==closed_testing$closed)
## [1] 0.9873792

The above display confirms that on the test data, 98.73% of the times, we can expect the model to correctly predict if a restaurant is closed. But this is not really the case, since the probability of getting a restaurant closed is very low, and hence even if we predict that the restaurant would never be closed, we still have more than 98% of the probability that our guess is correct. In fact our model has predicted only 16% (100 * 0.002148228/(0.002148228 + 0.011412460) = 16) of the test cases correctly, and 84% of the cases incorrectly. We will have to consider other models to evaluate if we can correctly predict the closure of the restaurant.

Let us use Linear discriminant analysis

lda.fit <- lda(closed~visit_level+score +`C_02A `+`C_02B `+`C_02G `+`C_02H `+`C_03A `+`C_03B `+`C_03C `+`C_04A `+`C_04C `+`C_04F `+`C_04H `+`C_04J `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_05A `+`C_05D `+`C_05F `+`C_05H `+`C_06A `+`C_06C `+`C_06D `+`C_06E `+`C_06F `+`C_06G `+`C_07A `+`C_08A `+`C_08C `+`C_09C `+`C_10A `+`C_10B `+`C_10D `+`C_10F `+`C_10H `+`C_10J `,data=closed_training)

#predict(lda.fit,closed_testing)$class
prop.table(table(predict(lda.fit,closed_testing)$class,closed_testing$closed))
##    
##               0           1
##   0 0.964151450 0.003356606
##   1 0.022287863 0.010204082
mean(predict(lda.fit,closed_testing)$class==closed_testing$closed)
## [1] 0.9743555

LDA (Linear Discriminant Analysis) looks promising. Since it is correctly predicts 75% of the test cases correctly (100 * 0.010204082/(0.010204082+0.003356606)). This means, out of the 100 closed restaurant cases, this LDA model predicts 75 cases correctly, and 25 cases incorrectly. Also this model predicts 97.4% of the cases correctly.

Let us use Quadratic Discriminant Analysis

qda.fit <- qda(closed~visit_level+score +`C_02A `+`C_02B `+`C_02G `+`C_02H `+`C_03A `+`C_03B `+`C_03C `+`C_04A `+`C_04C `+`C_04F `+`C_04H `+`C_04J `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_05A `+`C_05D `+`C_05F `+`C_05H `+`C_06A `+`C_06C `+`C_06D `+`C_06E `+`C_06F `+`C_06G `+`C_07A `+`C_08A `+`C_08C `+`C_09C `+`C_10A `+`C_10B `+`C_10D `+`C_10F `+`C_10H `+`C_10J `,data=closed_training)

#predict(qda.fit,closed_testing)$class
prop.table(table(predict(qda.fit,closed_testing)$class,closed_testing$closed))
##    
##               0           1
##   0 0.938775510 0.004385965
##   1 0.047663802 0.009174723
mean(predict(qda.fit,closed_testing)$class==closed_testing$closed)
## [1] 0.9479502

The QDA model has shown that approximately 68% of the closed test cases are correctly predicted. So this is not optimal, when compared to LDA.

Hence we will use LDA model to predict if a restaurant will be closed, given the current variables

Our LDA model uses the following violation codes. These violation code descrption is also provided below:

  rbind(
       violation_data[violation_data$code == "02A ",],
       violation_data[violation_data$code == "02B ",],
       violation_data[violation_data$code == "02G ",],
       violation_data[violation_data$code == "02H ",],
       violation_data[violation_data$code == "03A ",],
       violation_data[violation_data$code == "03B ",],
       violation_data[violation_data$code == "03C ",],
       violation_data[violation_data$code == "04A ",],
       violation_data[violation_data$code == "04C ",],
       violation_data[violation_data$code == "04F ",],
       violation_data[violation_data$code == "04H ",],
       violation_data[violation_data$code == "04J ",],
       violation_data[violation_data$code == "04K ",],
       violation_data[violation_data$code == "04L ",],
       
       violation_data[violation_data$code == "04M ",],
       violation_data[violation_data$code == "04N ",],
       violation_data[violation_data$code == "05A ",],
       violation_data[violation_data$code == "05D ",], 
       
       violation_data[violation_data$code == "05F ",], 
       violation_data[violation_data$code == "05H ",], 
       violation_data[violation_data$code == "06A ",], 
       violation_data[violation_data$code == "06C ",], 
       violation_data[violation_data$code == "06D ",], 
       violation_data[violation_data$code == "06E ",],
       
       violation_data[violation_data$code == "06F ",],
       violation_data[violation_data$code == "06G ",],
       violation_data[violation_data$code == "07A ",],
       violation_data[violation_data$code == "08A ",],
       violation_data[violation_data$code == "08C ",],

       violation_data[violation_data$code == "09C ",],
       violation_data[violation_data$code == "10A ",],
       violation_data[violation_data$code == "10B ",],
       violation_data[violation_data$code == "10D ",],
       
       violation_data[violation_data$code == "10F ",],
       violation_data[violation_data$code == "10H ",],
       violation_data[violation_data$code == "10J ",]
       
       
)
##    code
## 98 02A 
## 64 02B 
## 29 02G 
## 24 02H 
## 52 03A 
## 80 03B 
## 78 03C 
## 4  04A 
## 39 04C 
## 62 04F 
## 70 04H 
## 34 04J 
## 47 04K 
## 56 04L 
## 94 04M 
## 66 04N 
## 76 05A 
## 23 05D 
## 86 05F 
## 42 05H 
## 9  06A 
## 19 06C 
## 15 06D 
## 96 06E 
## 81 06F 
## 50 06G 
## 83 07A 
## 32 08A 
## 63 08C 
## 28 09C 
## 13 10A 
## 41 10B 
## 3  10D 
## 55 10F 
## 49 10H 
## 74 10J 
##                                                                                                                                                                                                                                                                                             description
## 98                                                                                                                                                                                                                                                     Food not cooked to required minimum temperature.
## 64                                                                                                                                                                                                                                                          Hot food item not held at or above 140º F.
## 29                                                                                                                                                                   Cold food item held above 41º F (smoked fish and reduced oxygen packaged foods above 38 ºF) except during necessary preparation.
## 24                                                                                                Food not cooled by an approved method whereby the internal product temperature is reduced from 140º F to 70º F or less within 2 hours, and from 70º F to 41º F or less within 4 additional hours.
## 52                                                                                                               Food from unapproved or unknown source or home canned. Reduced oxygen packaged (ROP) fish not frozen before processing; or ROP foods prepared on premises transported to another site.
## 80                                                                                                                                                                                                        Shellfish not from approved source, improperly tagged/labeled; tags not retained for 90 days.
## 78                                                                                                                                                                                                                           Eggs found dirty/cracked; liquid, frozen or powdered eggs not pasteurized.
## 4                                                                                                                                                                                                                                Food Protection Certificate not held by supervisor of food operations.
## 39                                                                                                                                                           Food worker does not use proper utensil to eliminate bare hand contact with food that will not receive adequate additional heat treatment.
## 62                                                                                                                                                                           Food, food preparation area, food storage area, area used by employees or patrons, contaminated by sewage or liquid waste.
## 70                                                                                                                                                                       Raw, cooked or prepared food is adulterated, contaminated, cross-contaminated, or not discarded in accordance with HACCP plan.
## 34                                                                                                        Appropriately scaled metal stem-type thermometer or thermocouple not provided or used to evaluate temperatures of potentially hazardous foods during cooking, cooling, reheating and holding.
## 47                                                                                                                                                                                                                      Evidence of rats or live rats present in facility's food and/or non-food areas.
## 56                                                                                                                                                                                                                      Evidence of mice or live mice present in facility's food and/or non-food areas.
## 94                                                                                                                                                                                                                                       Live roaches present in facility's food and/or non-food areas.
## 66 Filth flies or food/refuse/sewage-associated (FRSA) flies present in facility\032s food and/or non-food areas. Filth flies include house flies, little house flies, blow flies, bottle flies and flesh flies. Food/refuse/sewage-associated flies include fruit flies, drain flies and Phorid flies.
## 76                                                                                                                                                                                                                                                       Sewage disposal system improper or unapproved.
## 23                                            Hand washing facility not provided in or near food preparation area and toilet room. Hot and cold running water at adequate pressure to enable cleanliness of employees not provided at facility. Soap and an acceptable hand-drying device not provided.
## 86                                                                                                                                                                               Insufficient or no refrigerated or hot holding equipment to keep potentially hazardous foods at required temperatures.
## 42                                                                                                                                                                                                                       No facilities available to wash, rinse and sanitize utensils and/or equipment.
## 9                                                                                                                                                Personal cleanliness inadequate. Outer garment soiled with possible contaminant.  Effective hair restraint not worn in an area where food is prepared.
## 19                                                                                                                                                                           Food not protected from potential source of contamination during storage, preparation, transportation, display or service.
## 15                                                                                                                                                       Food contact surface not properly washed, rinsed and sanitized after each use and following any activity when contamination may have occurred.
## 96                                                                                                                                                                                                 Sanitized equipment or utensil, including in-use food dispensing utensil, improperly used or stored.
## 81                                                                                                                                                                                                                                           Wiping cloths soiled or not stored in sanitizing solution.
## 50                                                                                                                                                                                                                           HACCP plan not approved or approved HACCP plan not maintained on premises.
## 83                                                                                                                                                                                                                                Duties of an officer of the Department interfered with or obstructed.
## 32                                                                                                                                                                   Facility not vermin proof. Harborage or conditions conducive to attracting vermin to the premises and/or allowing vermin to exist.
## 63                                                                                                                                                                              Pesticide use not in accordance with label or applicable laws. Prohibited chemical used/stored. Open bait station used.
## 28                                                                                                                                                                                                                                                        Food contact surface not properly maintained.
## 13                                                                                                                                                                                               Toilet facility not maintained and provided with toilet paper, waste receptacle and self-closing door.
## 41                                                                   Plumbing not properly installed or maintained; anti-siphonage or backflow prevention device not provided where required; equipment or floor not properly drained; sewage disposal system in disrepair or not functioning properly.
## 3                                                                                             Mechanical or natural ventilation system not provided, improperly installed, in disrepair and/or fails to prevent excessive build-up of grease, heat, steam condensation vapors, odors, smoke, and fumes.
## 55                      Non-food contact surface improperly constructed. Unacceptable material used. Non-food contact surface or equipment improperly maintained and/or not properly sealed, raised, spaced or movable to allow accessibility for cleaning on all sides, above and underneath the unit.
## 49                                                                                                                                                                                                                                 Proper sanitization not provided for utensil ware washing operation.
## 74                                                                                                                                                                                                                                            ''''Wash hands\032 sign not posted at hand wash facility.
##                                                 level
## 98 Critical                                          
## 64 Critical                                          
## 29 Critical                                          
## 24 Critical                                          
## 52 Critical                                          
## 80 Critical                                          
## 78 Critical                                          
## 4  Critical                                          
## 39 Critical                                          
## 62 Critical                                          
## 70 Critical                                          
## 34 Critical                                          
## 47 Critical                                          
## 56 Critical                                          
## 94 Critical                                          
## 66 Critical                                          
## 76 Critical                                          
## 23 Critical                                          
## 86 Critical                                          
## 42 Critical                                          
## 9  Critical                                          
## 19 Critical                                          
## 15 Critical                                          
## 96 Critical                                          
## 81 Critical                                          
## 50 Critical                                          
## 83 Critical                                          
## 32 Not Critical                                      
## 63 Not Critical                                      
## 28 Not Critical                                      
## 13 Not Critical                                      
## 41 Not Critical                                      
## 3  Not Critical                                      
## 55 Not Critical                                      
## 49 Not Critical                                      
## 74 Not Critical

Predicting the factors infulencing “closure” of restaurant in “future”

R Code to prepare required data frames (preparation of 2 data frames for training and testing):

dim(inspection_spread)
## [1] 160536     90
names(inspection_spread)
##  [1] "visit_level"     "restaurant_id"   "cuisine_id"     
##  [4] "inspection_date" "closed"          "score"          
##  [7] "C_000 "          "C_02A "          "C_02B "         
## [10] "C_02C "          "C_02D "          "C_02E "         
## [13] "C_02F "          "C_02G "          "C_02H "         
## [16] "C_02I "          "C_02J "          "C_03A "         
## [19] "C_03B "          "C_03C "          "C_03D "         
## [22] "C_03E "          "C_03F "          "C_03G "         
## [25] "C_04A "          "C_04B "          "C_04C "         
## [28] "C_04D "          "C_04E "          "C_04F "         
## [31] "C_04G "          "C_04H "          "C_04I "         
## [34] "C_04J "          "C_04K "          "C_04L "         
## [37] "C_04M "          "C_04N "          "C_04O "         
## [40] "C_05A "          "C_05B "          "C_05C "         
## [43] "C_05D "          "C_05E "          "C_05F "         
## [46] "C_05H "          "C_05I "          "C_06A "         
## [49] "C_06B "          "C_06C "          "C_06D "         
## [52] "C_06E "          "C_06F "          "C_06G "         
## [55] "C_06H "          "C_06I "          "C_07A "         
## [58] "C_08A "          "C_08B "          "C_08C "         
## [61] "C_09A "          "C_09B "          "C_09C "         
## [64] "C_10A "          "C_10B "          "C_10C "         
## [67] "C_10D "          "C_10E "          "C_10F "         
## [70] "C_10G "          "C_10H "          "C_10I "         
## [73] "C_10J "          "C_10K "          "C_16B "         
## [76] "C_18D "          "C_18F "          "C_18G "         
## [79] "C_18I "          "C_20A "          "C_20D "         
## [82] "C_20E "          "C_20F "          "C_22A "         
## [85] "C_22C "          "C_99B "          "closed_next"    
## [88] "days_diff"       "year"            "month"
closed_training <- inspection_spread[(inspection_spread$year!=2015),c(-2,-4,-88,-89,-90)]
closed_testing <- inspection_spread[(inspection_spread$year==2015),c(-2,-4, -88,-89,-90)]

Using logistic regression to predict if a restaurant is “closed in future”, given the current variables

glm.fit=glm(closed_next~.,
data=closed_training,family =binomial )

summary(glm.fit)
## 
## Call:
## glm(formula = closed_next ~ ., family = binomial, data = closed_training)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.4832  -0.1885  -0.1582  -0.1384   3.2599  
## 
## Coefficients: (5 not defined because of singularities)
##               Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -4.764e+00  6.408e-02 -74.345  < 2e-16 ***
## visit_level  9.884e-03  7.421e-03   1.332 0.182890    
## cuisine_id   2.183e-03  8.616e-04   2.533 0.011298 *  
## closed       1.613e+00  8.375e-02  19.263  < 2e-16 ***
## score        2.785e-02  3.320e-03   8.388  < 2e-16 ***
## `C_000 `     5.322e-01  1.014e-01   5.250 1.52e-07 ***
## `C_02A `    -1.171e-01  3.647e-01  -0.321 0.748058    
## `C_02B `    -1.432e-02  6.152e-02  -0.233 0.815946    
## `C_02C `    -5.631e-01  4.220e-01  -1.334 0.182109    
## `C_02D `    -1.196e+01  2.183e+02  -0.055 0.956327    
## `C_02E `     7.905e-01  1.212e+00   0.652 0.514140    
## `C_02F `    -1.121e+01  5.006e+02  -0.022 0.982140    
## `C_02G `    -1.519e-01  5.643e-02  -2.692 0.007109 ** 
## `C_02H `    -2.632e-02  1.279e-01  -0.206 0.837039    
## `C_02I `     5.869e-01  7.965e-01   0.737 0.461199    
## `C_02J `            NA         NA      NA       NA    
## `C_03A `    -7.202e-02  2.885e-01  -0.250 0.802897    
## `C_03B `     1.794e-01  3.764e-01   0.477 0.633531    
## `C_03C `    -1.123e+00  7.216e-01  -1.557 0.119514    
## `C_03D `     1.752e+00  4.749e-01   3.688 0.000226 ***
## `C_03E `     9.729e-01  8.665e-01   1.123 0.261516    
## `C_03F `            NA         NA      NA       NA    
## `C_03G `    -1.208e+01  3.291e+02  -0.037 0.970718    
## `C_04A `     8.887e-02  8.625e-02   1.030 0.302846    
## `C_04B `    -8.062e-01  1.059e+00  -0.762 0.446252    
## `C_04C `    -4.690e-01  1.413e-01  -3.320 0.000900 ***
## `C_04D `    -7.889e-02  3.138e-01  -0.251 0.801467    
## `C_04E `    -6.550e-01  4.771e-01  -1.373 0.169799    
## `C_04F `     6.024e-01  4.253e-01   1.416 0.156683    
## `C_04G `     1.136e+00  1.242e+00   0.915 0.360314    
## `C_04H `    -2.317e-01  7.405e-02  -3.129 0.001757 ** 
## `C_04I `    -1.178e+01  2.968e+02  -0.040 0.968337    
## `C_04J `    -2.784e-01  1.074e-01  -2.594 0.009492 ** 
## `C_04K `     4.563e-01  1.049e-01   4.348 1.38e-05 ***
## `C_04L `     4.223e-01  5.594e-02   7.549 4.40e-14 ***
## `C_04M `     6.351e-01  5.884e-02  10.794  < 2e-16 ***
## `C_04N `     1.933e-01  5.883e-02   3.286 0.001015 ** 
## `C_04O `    -4.432e-01  3.764e-01  -1.178 0.238994    
## `C_05A `    -6.170e-01  4.532e-01  -1.361 0.173428    
## `C_05B `    -1.227e+01  2.103e+02  -0.058 0.953488    
## `C_05C `    -3.935e-01  1.131e+00  -0.348 0.727876    
## `C_05D `    -3.615e-02  1.094e-01  -0.330 0.741075    
## `C_05E `    -3.700e-01  5.027e-01  -0.736 0.461746    
## `C_05F `    -4.241e-01  1.937e-01  -2.189 0.028562 *  
## `C_05H `     2.139e-01  2.208e-01   0.969 0.332636    
## `C_05I `    -1.166e+01  8.328e+02  -0.014 0.988825    
## `C_06A `    -1.880e-01  9.116e-02  -2.062 0.039169 *  
## `C_06B `    -2.667e-01  1.542e-01  -1.729 0.083839 .  
## `C_06C `    -7.533e-02  6.072e-02  -1.241 0.214771    
## `C_06D `    -2.826e-01  6.098e-02  -4.634 3.59e-06 ***
## `C_06E `    -1.368e-01  8.417e-02  -1.625 0.104126    
## `C_06F `    -4.573e-02  8.904e-02  -0.514 0.607563    
## `C_06G `    -1.161e+00  1.024e+00  -1.134 0.256988    
## `C_06H `    -1.148e+01  3.817e+02  -0.030 0.975999    
## `C_06I `    -1.080e+01  3.257e+02  -0.033 0.973536    
## `C_07A `    -1.008e+00  5.051e-01  -1.996 0.045890 *  
## `C_08A `    -8.129e-02  6.576e-02  -1.236 0.216411    
## `C_08B `     1.461e-01  1.547e-01   0.945 0.344786    
## `C_08C `    -2.798e-03  9.889e-02  -0.028 0.977432    
## `C_09A `     5.745e-02  1.743e-01   0.330 0.741732    
## `C_09B `     1.560e-01  1.151e-01   1.356 0.175000    
## `C_09C `     3.498e-02  1.105e-01   0.317 0.751524    
## `C_10A `     1.085e-01  1.142e-01   0.950 0.341873    
## `C_10B `    -2.338e-02  5.647e-02  -0.414 0.678845    
## `C_10C `     2.797e-01  3.641e-01   0.768 0.442395    
## `C_10D `    -1.247e-01  1.216e-01  -1.026 0.304992    
## `C_10E `    -3.135e-01  1.775e-01  -1.766 0.077463 .  
## `C_10F `    -8.809e-02  4.713e-02  -1.869 0.061629 .  
## `C_10G `    -2.172e-01  7.482e-01  -0.290 0.771596    
## `C_10H `    -2.662e-01  1.119e-01  -2.379 0.017345 *  
## `C_10I `    -2.687e-01  1.616e-01  -1.663 0.096337 .  
## `C_10J `    -1.523e-01  1.852e-01  -0.822 0.410856    
## `C_10K `    -1.104e+01  1.455e+03  -0.008 0.993949    
## `C_16B `            NA         NA      NA       NA    
## `C_18D `            NA         NA      NA       NA    
## `C_18F `    -1.054e+01  1.455e+03  -0.007 0.994222    
## `C_18G `    -1.069e+01  1.455e+03  -0.007 0.994139    
## `C_18I `            NA         NA      NA       NA    
## `C_20A `    -1.103e+01  1.455e+03  -0.008 0.993954    
## `C_20D `     4.874e-02  2.058e+03   0.000 0.999981    
## `C_20E `    -1.226e+01  6.708e+02  -0.018 0.985420    
## `C_20F `    -1.114e+01  1.455e+03  -0.008 0.993894    
## `C_22A `     7.067e-01  1.673e+03   0.000 0.999663    
## `C_22C `    -1.142e+01  8.260e+02  -0.014 0.988965    
## `C_99B `    -1.490e-03  4.285e-01  -0.003 0.997225    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 23557  on 129292  degrees of freedom
## Residual deviance: 21510  on 129213  degrees of freedom
##   (8899 observations deleted due to missingness)
## AIC: 21670
## 
## Number of Fisher Scoring iterations: 14

The above display shows lots of variables, which have P-Value greater than 0.5%. Let us filter them:

rownames(summary(glm.fit)$coeff[which(summary(glm.fit)$coeff[,4]<0.005),])[-1]
##  [1] "closed"   "score"    "`C_000 `" "`C_03D `" "`C_04C `" "`C_04H `"
##  [7] "`C_04K `" "`C_04L `" "`C_04M `" "`C_04N `" "`C_06D `"

Creating another model with just the above variables.

glm.fit=glm(closed_next~closed+score +`C_000 `+`C_03D `+`C_04C `+`C_04H `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_06D `,
data=closed_training,family =binomial )

Logistic model evaluation to predict if a restaurant will be closed in future

We will evaluate the model on the closed_training data.

#The following commented R Code is tested on closed_testing. Since we are interested on training


#glm.probs <- predict(glm.fit,type="response")
#closed_training[which(glm.probs > 0.5),]
#contrasts(as.factor(closed_training$closed))

#glm.pred <- rep(0,nrow(closed_training))
#glm.pred[glm.probs>0.7] <- 1
#prop.table(table(glm.pred,closed_training$closed))
#mean(glm.pred==closed_training$closed)

#Evaluating the model performance on test data set:

glm.probs <- predict(glm.fit,closed_testing,type="response")
glm.pred <- rep(0,nrow(closed_testing))
glm.pred[glm.probs>0.7] <- 1
prop.table(table(glm.pred,closed_testing$closed))
##         
## glm.pred          0          1
##        0 0.98643931 0.01356069
mean(glm.pred==closed_testing$closed)
## [1] 0.9864393

The above output shows that the model predicts 98.64% correctly. But this is not really a good model, since it has predicted that none of the restaurants will be closed in future. This happened because, the probability of getting a restaurant closed is very low.

Let us use LDA model

lda.fit <- lda(closed_next~closed+score +`C_000 `+`C_03D `+`C_04C `+`C_04H `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_06D `,
data=closed_training)

#predict(lda.fit,closed_testing)$class
prop.table(table(predict(lda.fit,closed_testing)$class,closed_testing$closed_next))
##    
##               0           1
##   0 0.950127303 0.008686536
##   1 0.031451251 0.009734911

The LDA model predicts only 52.8% of the closed cases correctly. This model is not great at prediction, since even if we guess randomly, we will be 50% correct. Let us try another model (QDA - Quadratic Descriminant Model).

qda.fit <- qda(closed_next~closed+score +`C_000 `+`C_03D `+`C_04C `+`C_04H `+`C_04K `+`C_04L `+`C_04M `+`C_04N `+`C_06D `,
data=closed_training)

#predict(qda.fit,closed_testing)$class
prop.table(table(predict(qda.fit,closed_testing)$class,closed_testing$closed_next))
##    
##               0           1
##   0 0.906994159 0.007787929
##   1 0.074584394 0.010633518

The *quadatric discriminant model is better than LDA, and it predicts approximately 58% of the closed cases correctly.

Using the KNN (K Nearest Neighbors)

names(closed_training)
##  [1] "visit_level" "cuisine_id"  "closed"      "score"       "C_000 "     
##  [6] "C_02A "      "C_02B "      "C_02C "      "C_02D "      "C_02E "     
## [11] "C_02F "      "C_02G "      "C_02H "      "C_02I "      "C_02J "     
## [16] "C_03A "      "C_03B "      "C_03C "      "C_03D "      "C_03E "     
## [21] "C_03F "      "C_03G "      "C_04A "      "C_04B "      "C_04C "     
## [26] "C_04D "      "C_04E "      "C_04F "      "C_04G "      "C_04H "     
## [31] "C_04I "      "C_04J "      "C_04K "      "C_04L "      "C_04M "     
## [36] "C_04N "      "C_04O "      "C_05A "      "C_05B "      "C_05C "     
## [41] "C_05D "      "C_05E "      "C_05F "      "C_05H "      "C_05I "     
## [46] "C_06A "      "C_06B "      "C_06C "      "C_06D "      "C_06E "     
## [51] "C_06F "      "C_06G "      "C_06H "      "C_06I "      "C_07A "     
## [56] "C_08A "      "C_08B "      "C_08C "      "C_09A "      "C_09B "     
## [61] "C_09C "      "C_10A "      "C_10B "      "C_10C "      "C_10D "     
## [66] "C_10E "      "C_10F "      "C_10G "      "C_10H "      "C_10I "     
## [71] "C_10J "      "C_10K "      "C_16B "      "C_18D "      "C_18F "     
## [76] "C_18G "      "C_18I "      "C_20A "      "C_20D "      "C_20E "     
## [81] "C_20F "      "C_22A "      "C_22C "      "C_99B "      "closed_next"
closed_training <- na.omit(closed_training)
closed_testing <- na.omit(closed_testing)

closed_testing_1 <- as.data.frame(cbind(closed_testing$closed_next,closed_testing$closed,closed_testing$score,closed_testing$C_000,closed_testing$C_03D,closed_testing$C_04C,closed_testing$C_04H,
closed_testing$C_04K,closed_testing$C_04L,closed_testing$C_04M,closed_testing$C_04N,closed_testing$C_06D))

names(closed_testing_1) <- c("closed_next","closed","score","C_000","C_03D","C_04C","C_04H","C_04K","C_04L","C_04M","C_04N","C_06D")

closed_training_1 <- as.data.frame(cbind(closed_training$closed_next,closed_training$closed,closed_training$score,closed_training$C_000,closed_training$C_03D,closed_training$C_04C,closed_training$C_04H,
closed_training$C_04K,closed_training$C_04L,closed_training$C_04M,closed_training$C_04N,closed_training$C_06D))

names(closed_training_1) <- c("closed_next","closed","score","C_000","C_03D","C_04C","C_04H","C_04K","C_04L","C_04M","C_04N","C_06D")

p <- vector(length=50)
for(i in 1:50){
knn.pred <- knn(closed_training_1[,-1],closed_testing_1[,-1],closed_training_1$closed_next,k=i)

if(nrow(table(knn.pred,closed_testing_1$closed_next))==2)
  {
p[i] <- (table(knn.pred,closed_testing_1$closed_next)[2,2] / (table(knn.pred,closed_testing_1$closed_next)[2,2] + table(knn.pred,closed_testing_1$closed_next)[1,2])) * 100
}
else p[i] <- 0

}

which(max(p)==p)
## [1] 1
plot(1:50,p,xlab="K", ylab="Correct Predictions %", type="l")
points(1:50,p, col="red",cex=2,pch=20)

Figure: 5 Selecting the optimal K for KNN algorithm

The above graph shows that a maximum of 13% cases be predicted correctly. This is pretty low, and the KNN model is not appropriate to predict if a restaurant will be closed in the next visit.

Among all the models evaluated till now, only QDA is the best with 58% of the closed cases (in future) being correctly predicted.

Conclusions

We can conclude the following:

  • We can use the score of the restaurant to predict when the next inspection could happen

  • Linear Discriminant Analysis is a good model, to predict if the restaurant gets closed (in the current inspection). It has successfully predicted 75% of the test cases (i. e., out of 100 closed restaurants, it has predicted 75 restaurants would be closed)

  • Quadratic Discriminant Analysis is a good model to predict if a restaurant would be closed in future, given the current restaurant score, and current citations

-~-End of Project Report-~-