MSDA 607 Final project - Restaurants data analysis
Sekhar Mekala
Thursday, May 21, 2015
Project Requirements
The website https://data.cityofnewyork.us provides several data related to New York city in various domains such as Health, Transportation, Education etc. One such data set related to health category is the “Restaurants” dataset. As per the web site (https://data.cityofnewyork.us), the Department of Health and Mental Hygiene conducts unannounced inspections of restaurants at least once a year. Inspectors check for compliance in food handling, food temperature, personal hygiene and vermin control. Each violation of a regulation gets a certain number of points. At the end of the inspection, the inspector totals the points, and this number is the restaurant’s inspection score—the lower the score, the better the Grade
The major requirements of this project are given below:
R1. Given the restaurant’s current score, violations, number of times the restaurant has been visited by the inspectors, the restaurant’s cuisine, is there any way we can predict the number of days after which the inspector inspects again? This need not be an exact estimation, an error of 1 week in the estimation is accetable.
R2. Given the current violations of a restaurant, can we predict which violations are most probably found during the next inspection?
These two business requirements if correctly implemented, will help the restaurant owners to focus on the areas of improvement, and be prepared for sudden inspection by the food inspectors. The second requirement also helps the food inspectors to focus on specific possible violations, given a restaurant’s current violations, score, cuisine etc.
Data gathering
The data is publicly available at the following web site: https://data.cityofnewyork.us/Health/DOHMH-New-York-City-Restaurant-Inspection-Results/xx67-kt59
Technical design/requirements
T1. We will be implementing a RDBMS (Relational Database Management System) database in postgreSQL, following the principles of 1st, 2nd and 3rd normal forms. The RDBMS will help us to maintain the data in a structured format. Also given that the restaurant’s data is updated frequently (almost daily) at https://data.cityofnewyork.us, a RDBMS database will help us to store the data consistently for performing up to date analysis using our data analysis algorithms, and thus provide accurate predictions.NOTE that in the current scope of the project we will not be doing any on the fly analysis of the data. See T3 requirement given below, for more information.
T2. ERWIN will be used to perform the RDBMS database design
T3. The required CSV files are produced from the RDBMS database tables, and will be stored at github.com. These files will be further used by R programs to transform the data to a proper format and perform data analysis and provide predictions (listed in R1 and R2 requirements)
T4. Produce separate R data frames, for training, and one data frame (2015 data) for algorithm testing. The idea is to select the best algorithms with the least MSE (Mean Squared Error)
T5. Statistical model implementation and evaluation
T1/T2/T3 - RDBMS Design, implementation and final CSV files generation
We will be creating the following RDBMS tables:
ALL_DATA_DUMP - Contains all the data from https://data.cityofnewyork.us/Health/DOHMH-New-York-City-Restaurant-Inspection-Results/xx67-kt59
VIOLATION - Contains all the possible violation codes, violation’s description, and the level (critical/non-critical)
CUISINE - Contains all the available cuisines
RESTAURANT - Contains all the restaurants available, identified by restaurant ID
RESTAURANT_CUISINE - Contains restaurant’s cuisine information (mapping between restaurants and cuisines)
INSPECTION - Restaurant’s inspecton, violation, inspection date, score received, number of the visit (denoted by LEVEL)
Here is the database design (created using ERWIN):
Figure: 1 Database Design
The SQL Statements can be found at the following location (under the file: SQL_Statements.txt): https://github.com/msekhar12/MSDA_FINAL_PROJECT
The following three files are created (CSV Files) out of the SQL Statements provided at https://github.com/msekhar12/MSDA_FINAL_PROJECT (SQL_Statements.txt)
The file names along with the attributes of the files data is described below:
inspection file has the attributes visit_level, restaurant_id, cuisine_id, inspection_date, closed, violation_code, score
visit_level - contains the inspection number
restaurant_id - unique ID of the restaurant
cuisine_id - Cuisine identifier
inspection_date - inspection date
closed - contains 1 or 0. 1 means the restaurant is closed and 0 mean the restaurant is not closed
violation_code - violation code
score - score received in the inspection
violation_codes file with the attributes code,description,level
code - violation code
description - violation long description
level - Can be “Critical” or “Not critical”
cuisine file with the attributes id,name
id - cuisine identifier
name - cuisine name
The above three files are loaded at https://github.com/msekhar12/MSDA_FINAL_PROJECT. These files will be accessed and loaded into R data frames for further analysis.
T4 - Implementation (Data transformation and the creation of training and test datasets)
Required R libraries
We need the following R libraries to perform data transformation and analysis
library(RCurl)## Loading required package: bitopslibrary(dplyr)##
## Attaching package: 'dplyr'
##
## The following object is masked from 'package:stats':
##
## filter
##
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(tidyr)
library(knitr)
library(leaps)
library(boot)Reading the data from github
The following R code reads data from the 3 files (cuisine.txt, inspection.txt, violation_codes.txt) placed at “https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/”
URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/inspection.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
inspection_data <- read.csv(textConnection(x))
kable(head(inspection_data))| visit_level | restaurant_id | cuisine_id | inspection_date | closed | violation_code | score |
|---|---|---|---|---|---|---|
| 8 | 30075445 | 8 | 2015-02-09 | 0 | 06C | 6 |
| 7 | 30075445 | 8 | 2014-03-03 | 0 | 10F | 2 |
| 6 | 30075445 | 8 | 2013-10-10 | 0 | 000 | 0 |
| 5 | 30075445 | 8 | 2013-09-11 | 0 | 04L | 6 |
| 5 | 30075445 | 8 | 2013-09-11 | 0 | 04N | 6 |
| 4 | 30075445 | 8 | 2013-08-14 | 0 | 04L | 32 |
URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/cuisine.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
cuisine_data <- read.csv(textConnection(x))
kable(head(cuisine_data))| id | name |
|---|---|
| 1 | Afghan |
| 2 | African |
| 3 | American |
| 4 | Armenian |
| 5 | Asian |
| 6 | Australian |
URL <- "https://raw.githubusercontent.com/msekhar12/MSDA_FINAL_PROJECT/master/violation_codes.txt"
x <- getURL(URL, ssl.verifypeer = FALSE )
violation_data <- read.csv(textConnection(x))
kable(head(violation_data))| code | description | level |
|---|---|---|
| 18I | Choking first aid poster not posted. Alcohol and Pregnancy warning sign, inspection report sign; not posted. CPR sign not posted, equipment (resuscitation masks, adult & pediatric, latex gloves) not provided. | Not Critical |
| 04G | Unprotected potentially hazardous food re-served. | Critical |
| 10D | Mechanical or natural ventilation system not provided, improperly installed, in disrepair and/or fails to prevent excessive build-up of grease, heat, steam condensation vapors, odors, smoke, and fumes. | Not Critical |
| 04A | Food Protection Certificate not held by supervisor of food operations. | Critical |
| 10E | Accurate thermometer not provided in refrigerated or hot holding equipment. | Not Critical |
| 99B | Other general violation. | Not Critical |
The inspection_data data frame contains the inspection details with the following variables:
visit_level - The inspection number
restaurant_id - The restaurant ID (We are not concerned about the name)
cuisine_id - The cuisine of the restaurant. (we are not concerned with the name of the cuisine)
inspection_date - The date, when the restaurant was inspected
closed - will be either 1 or 0. 1 represents that the restaurant is closed, and 0 represents that it is not closed
violation_code - violation code (encoded)
score - restaurant’s current score. Lesser the score, better is the restaurant
The cuisine_data has the following cuisine details (variables):
id - Cuisine ID
name - Cuisine name
The violation_data has the following variables:
code - Violation ID
description - Violation description
level - “Critical” and “Not Critical”
Data transfomation
Creating inspection_spread data frame. This data frame will have the violation codes as the columns
inspection_data$citation <- 1
inspection_spread <- spread(inspection_data,violation_code, citation)
#head(inspection_spread)
#head(inspection_spread,100)
#Filling NA values with 0, in the citation
inspection_spread[is.na(inspection_spread)] <- 0
names(inspection_spread) ## [1] "visit_level" "restaurant_id" "cuisine_id"
## [4] "inspection_date" "closed" "score"
## [7] "000 " "02A " "02B "
## [10] "02C " "02D " "02E "
## [13] "02F " "02G " "02H "
## [16] "02I " "02J " "03A "
## [19] "03B " "03C " "03D "
## [22] "03E " "03F " "03G "
## [25] "04A " "04B " "04C "
## [28] "04D " "04E " "04F "
## [31] "04G " "04H " "04I "
## [34] "04J " "04K " "04L "
## [37] "04M " "04N " "04O "
## [40] "05A " "05B " "05C "
## [43] "05D " "05E " "05F "
## [46] "05H " "05I " "06A "
## [49] "06B " "06C " "06D "
## [52] "06E " "06F " "06G "
## [55] "06H " "06I " "07A "
## [58] "08A " "08B " "08C "
## [61] "09A " "09B " "09C "
## [64] "10A " "10B " "10C "
## [67] "10D " "10E " "10F "
## [70] "10G " "10H " "10I "
## [73] "10J " "10K " "16B "
## [76] "18D " "18F " "18G "
## [79] "18I " "20A " "20D "
## [82] "20E " "20F " "22A "
## [85] "22C " "99B "dim(inspection_spread)## [1] 160536 86#Changing the names of the columns where the names begin with a number. The citation codes begin with a number, but R does not support data frame column names beginning with a number
#Execute this statement only once.
names(inspection_spread)[7:86] <- paste("C_",names(inspection_spread)[7:86],sep="")
kable(head(inspection_spread))| visit_level | restaurant_id | cuisine_id | inspection_date | closed | score | C_000 | C_02A | C_02B | C_02C | C_02D | C_02E | C_02F | C_02G | C_02H | C_02I | C_02J | C_03A | C_03B | C_03C | C_03D | C_03E | C_03F | C_03G | C_04A | C_04B | C_04C | C_04D | C_04E | C_04F | C_04G | C_04H | C_04I | C_04J | C_04K | C_04L | C_04M | C_04N | C_04O | C_05A | C_05B | C_05C | C_05D | C_05E | C_05F | C_05H | C_05I | C_06A | C_06B | C_06C | C_06D | C_06E | C_06F | C_06G | C_06H | C_06I | C_07A | C_08A | C_08B | C_08C | C_09A | C_09B | C_09C | C_10A | C_10B | C_10C | C_10D | C_10E | C_10F | C_10G | C_10H | C_10I | C_10J | C_10K | C_16B | C_18D | C_18F | C_18G | C_18I | C_20A | C_20D | C_20E | C_20F | C_22A | C_22C | C_99B |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 8 | 30075445 | 8 | 2015-02-09 | 0 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 7 | 30075445 | 8 | 2014-03-03 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 6 | 30075445 | 8 | 2013-10-10 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 5 | 30075445 | 8 | 2013-09-11 | 0 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 4 | 30075445 | 8 | 2013-08-14 | 0 | 32 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 30075445 | 8 | 2013-01-24 | 0 | 10 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
To predict if a restaurant is closed in the next visit, we need to add a variable which contains the information, if a restaurant is closed in the next visit. R code to add closed_next variable to inspection_spread data frame is given below.
closed_next <- vector(length=length(inspection_spread$closed))
closed_next[1] <- NA
closed_next[2:(length(inspection_spread$closed))] <- inspection_spread$closed[1:(length(inspection_spread$closed)-1)]
#data.frame(inspection_spread$closed,closed_next)
restaurant_id_tmp <- vector(length=length(inspection_spread$restaurant_id))
restaurant_id_tmp[1] <- NA
restaurant_id_tmp[2:(length(inspection_spread$restaurant_id))] <- inspection_spread$restaurant_id[1:(length(inspection_spread$restaurant_id)-1)]
#data.frame(inspection_spread$restaurant_id,restaurant_id_tmp)
inspection_spread <- cbind(inspection_spread, closed_next, restaurant_id_tmp)
inspection_spread$closed_next[which(inspection_spread$restaurant_id != inspection_spread$restaurant_id_tmp)] <- NA
#head(inspection_spread)
#dim(inspection_spread)
inspection_spread <- inspection_spread[,-88]
names(inspection_spread)[87] <- "closed_next"
#names(inspection_spread)
#head(inspection_spread,100)To Predict the days between the visits…we need to add another variable called days_diff to inspection_spread.
date_temp <- vector(length=length(inspection_spread$inspection_date))
date_temp[1] <- NA
date_temp[2:(length(inspection_spread$inspection_date))] <- as.character(inspection_spread$inspection_date[1:(length(inspection_spread$inspection_date)-1)])
#data.frame(inspection_spread$inspection_date,date_temp)
restaurant_id_tmp <- vector(length=length(inspection_spread$restaurant_id))
restaurant_id_tmp[1] <- NA
restaurant_id_tmp[2:(length(inspection_spread$restaurant_id))] <- inspection_spread$restaurant_id[1:(length(inspection_spread$restaurant_id)-1)]
#data.frame(inspection_spread$restaurant_id,restaurant_id_tmp)
df_tmp <- data.frame(inspection_spread$inspection_date,date_temp,inspection_spread$restaurant_id,restaurant_id_tmp)
#names(df_tmp)
#head(df_tmp)
#Populating NA values to date_temp, when restaurant_id_tmp and inspection_spread.restaurant_id are not equal
df_tmp$date_temp[which(df_tmp$inspection_spread.restaurant_id != df_tmp$restaurant_id_tmp)] <- NA
#dim(df_tmp)
#dim(inspection_spread)
#Adding a days_diff variable to inspection_spread
inspection_spread$days_diff <- as.integer(difftime(strptime(df_tmp$date_temp, format = "%Y-%m-%d"),
strptime(df_tmp$inspection_spread.inspection_date, format = "%Y-%m-%d"),units="days")
)
#head(inspection_spread,10)Adding the year and month variables to inspection_spread data frame. These variables will help us to separate the data into different data frames (for training and test data) depending on the year/month of the inspection
y <- format(strptime(inspection_spread$inspection_date, format = "%Y-%m-%d"),"%Y")
m <- format(strptime(inspection_spread$inspection_date, format = "%Y-%m-%d"),"%m")
inspection_spread <- cbind(inspection_spread,year=y,month=m)
#head(inspection_spread)Displaying new columns (closed_next, days_diff, year, month) along with some other columns of the inspection_spread
kable(head(data.frame(visit_level=inspection_spread$visit_level,
restaurant_id=inspection_spread$restaurant_id,
cuisine_id=inspection_spread$cuisine_id,
inspection_date=inspection_spread$inspection_date,
closed=inspection_spread$closed,
closed_next=inspection_spread$closed_next,
days_diff=inspection_spread$days_diff,
score=inspection_spread$score,
year=inspection_spread$year,
month=inspection_spread$month),10) )| visit_level | restaurant_id | cuisine_id | inspection_date | closed | closed_next | days_diff | score | year | month |
|---|---|---|---|---|---|---|---|---|---|
| 8 | 30075445 | 8 | 2015-02-09 | 0 | NA | NA | 6 | 2015 | 02 |
| 7 | 30075445 | 8 | 2014-03-03 | 0 | 0 | 343 | 2 | 2014 | 03 |
| 6 | 30075445 | 8 | 2013-10-10 | 0 | 0 | 144 | 0 | 2013 | 10 |
| 5 | 30075445 | 8 | 2013-09-11 | 0 | 0 | 29 | 6 | 2013 | 09 |
| 4 | 30075445 | 8 | 2013-08-14 | 0 | 0 | 28 | 32 | 2013 | 08 |
| 3 | 30075445 | 8 | 2013-01-24 | 0 | 0 | 201 | 10 | 2013 | 01 |
| 3 | 30075445 | 8 | 2013-01-24 | 0 | 0 | 0 | 0 | 2013 | 01 |
| 2 | 30075445 | 8 | 2012-12-31 | 0 | 0 | 24 | 25 | 2012 | 12 |
| 1 | 30075445 | 8 | 2012-05-03 | 0 | 0 | 242 | 0 | 2012 | 05 |
| 7 | 30112340 | 37 | 2015-05-07 | 0 | NA | NA | 12 | 2015 | 05 |
Preparing R data frames for training and testing of statistical models The following R code creates days_diff_training** data frame, and days_diff_testing data frame. The days_diff_training data frame will have the daya related to all the years except the 2015 data. We will also eliminate unnecessary variables, which are not needed for creating the statistical models to predict days_diff variable output. The days_diff variable will have the predicted number of days, after which a sudden food inspection could happen. An error of 1 week is allowed in the prediction.
days_diff_training <- inspection_spread[inspection_spread$year != 2015,]
#days_diff_training$month <- as.integer(days_diff_training$month)
#days_diff_training$year <- as.integer(as.character(days_diff_training$year))
#Elimination variables restaurant_id, inspection_date and closed_next from the training data, since these are not needed for training
#names(days_diff_training)
days_diff_training <- days_diff_training[,c(-2, -4, -5, -87, -89, -90)]
#names(days_diff_training)
#Omitting the NA values from training data set:
days_diff_training <- na.omit(days_diff_training)
dim(days_diff_training)## [1] 129293 84days_diff_testing <- inspection_spread[inspection_spread$year == 2015,]
#days_diff_training$month <- as.integer(days_diff_training$month)
#days_diff_training$year <- as.integer(as.character(days_diff_training$year))
#Elimination variables restaurant_id, inspection_date and closed_next from the training data, since these are not needed for training
#names(days_diff_testing)
days_diff_testing <- days_diff_testing[,c(-2, -4, -5, -87, -89, -90)]
#names(days_diff_testing)
#
days_diff_testing <- na.omit(days_diff_testing)
#dim(days_diff_testing)
kable(head(days_diff_testing))| visit_level | cuisine_id | score | C_000 | C_02A | C_02B | C_02C | C_02D | C_02E | C_02F | C_02G | C_02H | C_02I | C_02J | C_03A | C_03B | C_03C | C_03D | C_03E | C_03F | C_03G | C_04A | C_04B | C_04C | C_04D | C_04E | C_04F | C_04G | C_04H | C_04I | C_04J | C_04K | C_04L | C_04M | C_04N | C_04O | C_05A | C_05B | C_05C | C_05D | C_05E | C_05F | C_05H | C_05I | C_06A | C_06B | C_06C | C_06D | C_06E | C_06F | C_06G | C_06H | C_06I | C_07A | C_08A | C_08B | C_08C | C_09A | C_09B | C_09C | C_10A | C_10B | C_10C | C_10D | C_10E | C_10F | C_10G | C_10H | C_10I | C_10J | C_10K | C_16B | C_18D | C_18F | C_18G | C_18I | C_20A | C_20D | C_20E | C_20F | C_22A | C_22C | C_99B | days_diff | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 79 | 11 | 18 | 30 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 27 |
| 96 | 7 | 39 | 18 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 13 |
| 104 | 7 | 48 | 40 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 29 |
| 115 | 5 | 41 | 14 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 23 |
| 142 | 5 | 26 | 12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 20 |
| 171 | 3 | 3 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 20 |
kable(head(days_diff_training))| visit_level | cuisine_id | score | C_000 | C_02A | C_02B | C_02C | C_02D | C_02E | C_02F | C_02G | C_02H | C_02I | C_02J | C_03A | C_03B | C_03C | C_03D | C_03E | C_03F | C_03G | C_04A | C_04B | C_04C | C_04D | C_04E | C_04F | C_04G | C_04H | C_04I | C_04J | C_04K | C_04L | C_04M | C_04N | C_04O | C_05A | C_05B | C_05C | C_05D | C_05E | C_05F | C_05H | C_05I | C_06A | C_06B | C_06C | C_06D | C_06E | C_06F | C_06G | C_06H | C_06I | C_07A | C_08A | C_08B | C_08C | C_09A | C_09B | C_09C | C_10A | C_10B | C_10C | C_10D | C_10E | C_10F | C_10G | C_10H | C_10I | C_10J | C_10K | C_16B | C_18D | C_18F | C_18G | C_18I | C_20A | C_20D | C_20E | C_20F | C_22A | C_22C | C_99B | days_diff | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 7 | 8 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 343 |
| 3 | 6 | 8 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 144 |
| 4 | 5 | 8 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 29 |
| 5 | 4 | 8 | 32 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 28 |
| 6 | 3 | 8 | 10 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 201 |
| 7 | 3 | 8 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Thus we produced the following 2 data frames needed for the model implementation and testing for days_diff prediction.
days_diff_training
days_diff_testing
T5. Statistical model implementation and evaluation
Predicting the “days_diff” variable In the days_diff_training we have 84 variables, including days_diff variable. We will use the forward selection method, and evaluate which variables are optimal, and eliminate the variables which are not really needed. The main draw back of this approach is we are assuming linear model. But this will at least help us to eliminate some irrevelant variables. Once the important variables are obtained, we will create more models for various degrees of freedom and select the optimal method. We will also evaluate KNN (K-Nearest Neighbors method) performance.
set.seed(18)
regfit.fwd <- regsubsets (days_diff~., data=days_diff_training, nvmax =83, method ="forward")## Warning in leaps.setup(x, y, wt = wt, nbest = nbest, nvmax = nvmax,
## force.in = force.in, : 5 linear dependencies found## Reordering variables and trying again:reg.summary <- summary(regfit.fwd)
summary(regfit.fwd)## Subset selection object
## Call: regsubsets.formula(days_diff ~ ., data = days_diff_training,
## nvmax = 83, method = "forward")
## 83 Variables (and intercept)
## Forced in Forced out
## visit_level FALSE FALSE
## cuisine_id FALSE FALSE
## score FALSE FALSE
## `C_000 ` FALSE FALSE
## `C_02A ` FALSE FALSE
## `C_02B ` FALSE FALSE
## `C_02C ` FALSE FALSE
## `C_02D ` FALSE FALSE
## `C_02E ` FALSE FALSE
## `C_02F ` FALSE FALSE
## `C_02G ` FALSE FALSE
## `C_02H ` FALSE FALSE
## `C_02I ` FALSE FALSE
## `C_03A ` FALSE FALSE
## `C_03B ` FALSE FALSE
## `C_03C ` FALSE FALSE
## `C_03D ` FALSE FALSE
## `C_03E ` FALSE FALSE
## `C_03G ` FALSE FALSE
## `C_04A ` FALSE FALSE
## `C_04B ` FALSE FALSE
## `C_04C ` FALSE FALSE
## `C_04D ` FALSE FALSE
## `C_04E ` FALSE FALSE
## `C_04F ` FALSE FALSE
## `C_04G ` FALSE FALSE
## `C_04H ` FALSE FALSE
## `C_04I ` FALSE FALSE
## `C_04J ` FALSE FALSE
## `C_04K ` FALSE FALSE
## `C_04L ` FALSE FALSE
## `C_04M ` FALSE FALSE
## `C_04N ` FALSE FALSE
## `C_04O ` FALSE FALSE
## `C_05A ` FALSE FALSE
## `C_05B ` FALSE FALSE
## `C_05C ` FALSE FALSE
## `C_05D ` FALSE FALSE
## `C_05E ` FALSE FALSE
## `C_05F ` FALSE FALSE
## `C_05H ` FALSE FALSE
## `C_05I ` FALSE FALSE
## `C_06A ` FALSE FALSE
## `C_06B ` FALSE FALSE
## `C_06C ` FALSE FALSE
## `C_06D ` FALSE FALSE
## `C_06E ` FALSE FALSE
## `C_06F ` FALSE FALSE
## `C_06G ` FALSE FALSE
## `C_06H ` FALSE FALSE
## `C_06I ` FALSE FALSE
## `C_07A ` FALSE FALSE
## `C_08A ` FALSE FALSE
## `C_08B ` FALSE FALSE
## `C_08C ` FALSE FALSE
## `C_09A ` FALSE FALSE
## `C_09B ` FALSE FALSE
## `C_09C ` FALSE FALSE
## `C_10A ` FALSE FALSE
## `C_10B ` FALSE FALSE
## `C_10C ` FALSE FALSE
## `C_10D ` FALSE FALSE
## `C_10E ` FALSE FALSE
## `C_10F ` FALSE FALSE
## `C_10G ` FALSE FALSE
## `C_10H ` FALSE FALSE
## `C_10I ` FALSE FALSE
## `C_10J ` FALSE FALSE
## `C_10K ` FALSE FALSE
## `C_18F ` FALSE FALSE
## `C_18G ` FALSE FALSE
## `C_20A ` FALSE FALSE
## `C_20D ` FALSE FALSE
## `C_20E ` FALSE FALSE
## `C_20F ` FALSE FALSE
## `C_22A ` FALSE FALSE
## `C_22C ` FALSE FALSE
## `C_99B ` FALSE FALSE
## `C_02J ` FALSE FALSE
## `C_03F ` FALSE FALSE
## `C_16B ` FALSE FALSE
## `C_18D ` FALSE FALSE
## `C_18I ` FALSE FALSE
## 1 subsets of each size up to 78
## Selection Algorithm: forward
## visit_level cuisine_id score `C_000 ` `C_02A ` `C_02B ` `C_02C `
## 1 ( 1 ) " " " " "*" " " " " " " " "
## 2 ( 1 ) " " " " "*" "*" " " " " " "
## 3 ( 1 ) "*" " " "*" "*" " " " " " "
## 4 ( 1 ) "*" " " "*" "*" " " " " " "
## 5 ( 1 ) "*" " " "*" "*" " " " " " "
## 6 ( 1 ) "*" " " "*" "*" " " " " " "
## 7 ( 1 ) "*" " " "*" "*" " " " " " "
## 8 ( 1 ) "*" " " "*" "*" " " " " " "
## 9 ( 1 ) "*" " " "*" "*" " " "*" " "
## 10 ( 1 ) "*" " " "*" "*" " " "*" " "
## 11 ( 1 ) "*" " " "*" "*" " " "*" " "
## 12 ( 1 ) "*" " " "*" "*" " " "*" " "
## 13 ( 1 ) "*" " " "*" "*" " " "*" " "
## 14 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 15 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 16 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 17 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 18 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 19 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 20 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 21 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 22 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 23 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 24 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 25 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 26 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 27 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 28 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 29 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 30 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 31 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 32 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 33 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 34 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 35 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 36 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 37 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 38 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 39 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 40 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 41 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 42 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 43 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 44 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 45 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 46 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 47 ( 1 ) "*" "*" "*" "*" " " "*" " "
## 48 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 49 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 50 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 51 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 52 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 53 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 54 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 55 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 56 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 57 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 58 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 59 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 60 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 61 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 62 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 63 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 64 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 65 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 66 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 67 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 68 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 69 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 70 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_02D ` `C_02E ` `C_02F ` `C_02G ` `C_02H ` `C_02I ` `C_02J `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " "*" " " " " " "
## 9 ( 1 ) " " " " " " "*" " " " " " "
## 10 ( 1 ) " " " " " " "*" " " " " " "
## 11 ( 1 ) " " " " " " "*" " " " " " "
## 12 ( 1 ) " " " " " " "*" " " " " " "
## 13 ( 1 ) " " " " " " "*" " " " " " "
## 14 ( 1 ) " " " " " " "*" " " " " " "
## 15 ( 1 ) " " " " " " "*" " " " " " "
## 16 ( 1 ) " " " " " " "*" " " " " " "
## 17 ( 1 ) " " " " " " "*" " " " " " "
## 18 ( 1 ) " " " " " " "*" " " " " " "
## 19 ( 1 ) " " " " " " "*" " " " " " "
## 20 ( 1 ) " " " " " " "*" "*" " " " "
## 21 ( 1 ) " " " " " " "*" "*" " " " "
## 22 ( 1 ) " " " " " " "*" "*" " " " "
## 23 ( 1 ) " " " " " " "*" "*" " " " "
## 24 ( 1 ) " " " " " " "*" "*" " " " "
## 25 ( 1 ) " " " " " " "*" "*" " " " "
## 26 ( 1 ) " " " " " " "*" "*" " " " "
## 27 ( 1 ) " " " " " " "*" "*" " " " "
## 28 ( 1 ) " " " " " " "*" "*" " " " "
## 29 ( 1 ) " " " " " " "*" "*" " " " "
## 30 ( 1 ) " " " " " " "*" "*" " " " "
## 31 ( 1 ) " " " " " " "*" "*" " " " "
## 32 ( 1 ) " " " " " " "*" "*" " " " "
## 33 ( 1 ) " " " " " " "*" "*" " " " "
## 34 ( 1 ) " " " " " " "*" "*" " " " "
## 35 ( 1 ) " " " " " " "*" "*" " " " "
## 36 ( 1 ) " " " " " " "*" "*" " " " "
## 37 ( 1 ) " " " " " " "*" "*" " " " "
## 38 ( 1 ) " " " " " " "*" "*" " " " "
## 39 ( 1 ) " " " " " " "*" "*" " " " "
## 40 ( 1 ) " " " " " " "*" "*" " " " "
## 41 ( 1 ) " " " " " " "*" "*" " " " "
## 42 ( 1 ) " " " " " " "*" "*" " " " "
## 43 ( 1 ) " " " " " " "*" "*" " " " "
## 44 ( 1 ) " " " " " " "*" "*" " " " "
## 45 ( 1 ) " " " " " " "*" "*" " " " "
## 46 ( 1 ) " " "*" " " "*" "*" " " " "
## 47 ( 1 ) " " "*" " " "*" "*" " " " "
## 48 ( 1 ) " " "*" " " "*" "*" " " " "
## 49 ( 1 ) " " "*" " " "*" "*" " " " "
## 50 ( 1 ) " " "*" " " "*" "*" " " " "
## 51 ( 1 ) " " "*" " " "*" "*" " " " "
## 52 ( 1 ) " " "*" " " "*" "*" " " " "
## 53 ( 1 ) " " "*" " " "*" "*" " " " "
## 54 ( 1 ) " " "*" "*" "*" "*" " " " "
## 55 ( 1 ) " " "*" "*" "*" "*" " " " "
## 56 ( 1 ) " " "*" "*" "*" "*" " " " "
## 57 ( 1 ) " " "*" "*" "*" "*" " " " "
## 58 ( 1 ) " " "*" "*" "*" "*" " " " "
## 59 ( 1 ) " " "*" "*" "*" "*" " " " "
## 60 ( 1 ) " " "*" "*" "*" "*" " " " "
## 61 ( 1 ) " " "*" "*" "*" "*" " " " "
## 62 ( 1 ) " " "*" "*" "*" "*" " " " "
## 63 ( 1 ) " " "*" "*" "*" "*" " " " "
## 64 ( 1 ) " " "*" "*" "*" "*" " " " "
## 65 ( 1 ) " " "*" "*" "*" "*" " " " "
## 66 ( 1 ) " " "*" "*" "*" "*" " " " "
## 67 ( 1 ) " " "*" "*" "*" "*" " " " "
## 68 ( 1 ) " " "*" "*" "*" "*" " " " "
## 69 ( 1 ) " " "*" "*" "*" "*" " " " "
## 70 ( 1 ) " " "*" "*" "*" "*" " " " "
## 71 ( 1 ) " " "*" "*" "*" "*" " " " "
## 72 ( 1 ) " " "*" "*" "*" "*" " " " "
## 73 ( 1 ) "*" "*" "*" "*" "*" " " " "
## 74 ( 1 ) "*" "*" "*" "*" "*" " " " "
## 75 ( 1 ) "*" "*" "*" "*" "*" " " " "
## 76 ( 1 ) "*" "*" "*" "*" "*" " " " "
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" " "
## `C_03A ` `C_03B ` `C_03C ` `C_03D ` `C_03E ` `C_03F ` `C_03G `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " " " " " " "
## 16 ( 1 ) " " " " " " " " " " " " " "
## 17 ( 1 ) " " " " " " " " " " " " " "
## 18 ( 1 ) " " " " " " " " " " " " " "
## 19 ( 1 ) " " " " " " " " " " " " " "
## 20 ( 1 ) " " " " " " " " " " " " " "
## 21 ( 1 ) " " " " " " " " " " " " " "
## 22 ( 1 ) " " " " " " " " " " " " " "
## 23 ( 1 ) " " " " " " " " " " " " " "
## 24 ( 1 ) " " " " " " " " " " " " " "
## 25 ( 1 ) " " " " " " " " " " " " " "
## 26 ( 1 ) " " " " " " " " " " " " " "
## 27 ( 1 ) " " " " " " " " " " " " " "
## 28 ( 1 ) " " " " " " " " " " " " " "
## 29 ( 1 ) " " " " " " " " " " " " " "
## 30 ( 1 ) " " " " " " " " " " " " " "
## 31 ( 1 ) " " " " " " " " " " " " " "
## 32 ( 1 ) " " " " " " " " " " " " " "
## 33 ( 1 ) " " " " " " " " " " " " " "
## 34 ( 1 ) " " " " "*" " " " " " " " "
## 35 ( 1 ) " " " " "*" " " " " " " " "
## 36 ( 1 ) " " " " "*" "*" " " " " " "
## 37 ( 1 ) " " " " "*" "*" " " " " " "
## 38 ( 1 ) " " " " "*" "*" "*" " " " "
## 39 ( 1 ) " " " " "*" "*" "*" " " " "
## 40 ( 1 ) " " " " "*" "*" "*" " " " "
## 41 ( 1 ) " " " " "*" "*" "*" " " " "
## 42 ( 1 ) " " " " "*" "*" "*" " " " "
## 43 ( 1 ) " " " " "*" "*" "*" " " " "
## 44 ( 1 ) " " " " "*" "*" "*" " " " "
## 45 ( 1 ) " " " " "*" "*" "*" " " " "
## 46 ( 1 ) " " " " "*" "*" "*" " " " "
## 47 ( 1 ) " " " " "*" "*" "*" " " " "
## 48 ( 1 ) " " " " "*" "*" "*" " " " "
## 49 ( 1 ) " " " " "*" "*" "*" " " " "
## 50 ( 1 ) " " " " "*" "*" "*" " " " "
## 51 ( 1 ) " " " " "*" "*" "*" " " " "
## 52 ( 1 ) " " " " "*" "*" "*" " " " "
## 53 ( 1 ) "*" " " "*" "*" "*" " " " "
## 54 ( 1 ) "*" " " "*" "*" "*" " " " "
## 55 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 56 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 57 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 58 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 59 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 60 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 61 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 62 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 63 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 64 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 65 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 66 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 67 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 68 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 69 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 70 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 71 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 72 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 73 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 74 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## `C_04A ` `C_04B ` `C_04C ` `C_04D ` `C_04E ` `C_04F ` `C_04G `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) "*" " " " " " " " " " " " "
## 13 ( 1 ) "*" " " " " " " " " " " " "
## 14 ( 1 ) "*" " " " " " " " " " " " "
## 15 ( 1 ) "*" " " " " " " " " " " " "
## 16 ( 1 ) "*" " " " " " " " " " " " "
## 17 ( 1 ) "*" " " " " " " " " " " " "
## 18 ( 1 ) "*" " " " " " " " " " " " "
## 19 ( 1 ) "*" " " " " " " " " " " " "
## 20 ( 1 ) "*" " " " " " " " " " " " "
## 21 ( 1 ) "*" " " " " " " " " " " " "
## 22 ( 1 ) "*" " " "*" " " " " " " " "
## 23 ( 1 ) "*" " " "*" " " " " " " " "
## 24 ( 1 ) "*" " " "*" " " " " " " " "
## 25 ( 1 ) "*" " " "*" " " " " " " " "
## 26 ( 1 ) "*" " " "*" " " " " " " " "
## 27 ( 1 ) "*" " " "*" " " " " " " " "
## 28 ( 1 ) "*" " " "*" " " " " " " " "
## 29 ( 1 ) "*" " " "*" " " " " " " " "
## 30 ( 1 ) "*" " " "*" " " " " " " " "
## 31 ( 1 ) "*" " " "*" " " " " " " " "
## 32 ( 1 ) "*" " " "*" " " " " " " " "
## 33 ( 1 ) "*" " " "*" " " " " " " " "
## 34 ( 1 ) "*" " " "*" " " " " " " " "
## 35 ( 1 ) "*" " " "*" " " " " " " " "
## 36 ( 1 ) "*" " " "*" " " " " " " " "
## 37 ( 1 ) "*" " " "*" " " " " " " " "
## 38 ( 1 ) "*" " " "*" " " " " " " " "
## 39 ( 1 ) "*" " " "*" " " " " " " " "
## 40 ( 1 ) "*" " " "*" " " " " " " " "
## 41 ( 1 ) "*" " " "*" " " " " " " " "
## 42 ( 1 ) "*" " " "*" " " " " " " " "
## 43 ( 1 ) "*" " " "*" " " " " " " " "
## 44 ( 1 ) "*" " " "*" " " " " " " " "
## 45 ( 1 ) "*" " " "*" " " " " " " " "
## 46 ( 1 ) "*" " " "*" " " " " " " " "
## 47 ( 1 ) "*" " " "*" " " " " " " " "
## 48 ( 1 ) "*" " " "*" " " " " " " " "
## 49 ( 1 ) "*" " " "*" " " " " " " " "
## 50 ( 1 ) "*" " " "*" " " " " " " " "
## 51 ( 1 ) "*" " " "*" " " " " " " " "
## 52 ( 1 ) "*" "*" "*" " " " " " " " "
## 53 ( 1 ) "*" "*" "*" " " " " " " " "
## 54 ( 1 ) "*" "*" "*" " " " " " " " "
## 55 ( 1 ) "*" "*" "*" " " " " " " " "
## 56 ( 1 ) "*" "*" "*" " " " " " " " "
## 57 ( 1 ) "*" "*" "*" " " " " " " " "
## 58 ( 1 ) "*" "*" "*" " " " " " " " "
## 59 ( 1 ) "*" "*" "*" " " " " " " "*"
## 60 ( 1 ) "*" "*" "*" " " " " " " "*"
## 61 ( 1 ) "*" "*" "*" " " " " " " "*"
## 62 ( 1 ) "*" "*" "*" " " " " " " "*"
## 63 ( 1 ) "*" "*" "*" " " " " " " "*"
## 64 ( 1 ) "*" "*" "*" " " " " " " "*"
## 65 ( 1 ) "*" "*" "*" " " "*" " " "*"
## 66 ( 1 ) "*" "*" "*" " " "*" " " "*"
## 67 ( 1 ) "*" "*" "*" " " "*" " " "*"
## 68 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 69 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 70 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" " " "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_04H ` `C_04I ` `C_04J ` `C_04K ` `C_04L ` `C_04M ` `C_04N `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " "*" " " " "
## 5 ( 1 ) " " " " " " " " "*" " " " "
## 6 ( 1 ) " " " " " " " " "*" " " "*"
## 7 ( 1 ) " " " " " " " " "*" "*" "*"
## 8 ( 1 ) " " " " " " " " "*" "*" "*"
## 9 ( 1 ) " " " " " " " " "*" "*" "*"
## 10 ( 1 ) " " " " " " " " "*" "*" "*"
## 11 ( 1 ) "*" " " " " " " "*" "*" "*"
## 12 ( 1 ) "*" " " " " " " "*" "*" "*"
## 13 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 14 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 15 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 16 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 17 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 18 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 19 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 20 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 21 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 22 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 23 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 24 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 25 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 26 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 27 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 28 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 29 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 30 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 31 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 32 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 33 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 34 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 35 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 36 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 37 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 38 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 39 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 40 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 41 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 42 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 43 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 44 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 45 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 46 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 47 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 48 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 49 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 50 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 51 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 52 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 53 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 54 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 55 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 56 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 57 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 58 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 59 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 60 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 61 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 62 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 63 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 64 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 65 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 66 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 67 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 68 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 69 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 70 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_04O ` `C_05A ` `C_05B ` `C_05C ` `C_05D ` `C_05E ` `C_05F `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " "*" " " " "
## 16 ( 1 ) " " " " " " " " "*" " " " "
## 17 ( 1 ) " " " " " " " " "*" " " " "
## 18 ( 1 ) " " " " " " " " "*" " " " "
## 19 ( 1 ) " " " " " " " " "*" " " " "
## 20 ( 1 ) " " " " " " " " "*" " " " "
## 21 ( 1 ) " " " " " " " " "*" " " " "
## 22 ( 1 ) " " " " " " " " "*" " " " "
## 23 ( 1 ) " " " " " " " " "*" " " " "
## 24 ( 1 ) " " " " " " " " "*" " " " "
## 25 ( 1 ) " " " " " " " " "*" " " " "
## 26 ( 1 ) " " " " " " " " "*" " " " "
## 27 ( 1 ) " " " " " " " " "*" " " " "
## 28 ( 1 ) " " " " " " " " "*" " " " "
## 29 ( 1 ) " " " " " " " " "*" " " " "
## 30 ( 1 ) " " " " " " " " "*" " " "*"
## 31 ( 1 ) " " " " " " " " "*" " " "*"
## 32 ( 1 ) " " " " " " " " "*" " " "*"
## 33 ( 1 ) " " " " " " " " "*" " " "*"
## 34 ( 1 ) " " " " " " " " "*" " " "*"
## 35 ( 1 ) " " " " " " " " "*" "*" "*"
## 36 ( 1 ) " " " " " " " " "*" "*" "*"
## 37 ( 1 ) " " " " " " " " "*" "*" "*"
## 38 ( 1 ) " " " " " " " " "*" "*" "*"
## 39 ( 1 ) " " " " " " " " "*" "*" "*"
## 40 ( 1 ) " " " " " " " " "*" "*" "*"
## 41 ( 1 ) " " " " " " " " "*" "*" "*"
## 42 ( 1 ) " " " " " " " " "*" "*" "*"
## 43 ( 1 ) " " " " " " "*" "*" "*" "*"
## 44 ( 1 ) " " " " " " "*" "*" "*" "*"
## 45 ( 1 ) " " " " " " "*" "*" "*" "*"
## 46 ( 1 ) " " " " " " "*" "*" "*" "*"
## 47 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 48 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 49 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 50 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 51 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 52 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 53 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 54 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 55 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 56 ( 1 ) "*" " " " " "*" "*" "*" "*"
## 57 ( 1 ) "*" "*" " " "*" "*" "*" "*"
## 58 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 59 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 60 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 61 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 62 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 63 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 64 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 65 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 66 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 67 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 68 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 69 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 70 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_05H ` `C_05I ` `C_06A ` `C_06B ` `C_06C ` `C_06D ` `C_06E `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " " " " " " "
## 16 ( 1 ) " " " " " " " " " " " " " "
## 17 ( 1 ) " " " " " " " " " " " " "*"
## 18 ( 1 ) " " " " "*" " " " " " " "*"
## 19 ( 1 ) " " " " "*" " " "*" " " "*"
## 20 ( 1 ) " " " " "*" " " "*" " " "*"
## 21 ( 1 ) " " " " "*" " " "*" "*" "*"
## 22 ( 1 ) " " " " "*" " " "*" "*" "*"
## 23 ( 1 ) " " " " "*" " " "*" "*" "*"
## 24 ( 1 ) " " " " "*" " " "*" "*" "*"
## 25 ( 1 ) " " " " "*" " " "*" "*" "*"
## 26 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 27 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 28 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 29 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 30 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 31 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 32 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 33 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 34 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 35 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 36 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 37 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 38 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 39 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 40 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 41 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 42 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 43 ( 1 ) "*" " " "*" " " "*" "*" "*"
## 44 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 45 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 46 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 47 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 48 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 49 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 50 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 51 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 52 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 53 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 54 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 55 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 56 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 57 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 58 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 59 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 60 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 61 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 62 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 63 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 64 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 65 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 66 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 67 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 68 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 69 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 70 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_06F ` `C_06G ` `C_06H ` `C_06I ` `C_07A ` `C_08A ` `C_08B `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) "*" " " " " " " " " " " " "
## 11 ( 1 ) "*" " " " " " " " " " " " "
## 12 ( 1 ) "*" " " " " " " " " " " " "
## 13 ( 1 ) "*" " " " " " " " " " " " "
## 14 ( 1 ) "*" " " " " " " " " " " " "
## 15 ( 1 ) "*" " " " " " " " " " " " "
## 16 ( 1 ) "*" " " " " " " " " " " " "
## 17 ( 1 ) "*" " " " " " " " " " " " "
## 18 ( 1 ) "*" " " " " " " " " " " " "
## 19 ( 1 ) "*" " " " " " " " " " " " "
## 20 ( 1 ) "*" " " " " " " " " " " " "
## 21 ( 1 ) "*" " " " " " " " " " " " "
## 22 ( 1 ) "*" " " " " " " " " " " " "
## 23 ( 1 ) "*" " " " " " " " " " " " "
## 24 ( 1 ) "*" " " " " " " "*" " " " "
## 25 ( 1 ) "*" " " " " " " "*" " " " "
## 26 ( 1 ) "*" " " " " " " "*" " " " "
## 27 ( 1 ) "*" " " " " " " "*" " " " "
## 28 ( 1 ) "*" " " " " " " "*" " " " "
## 29 ( 1 ) "*" " " " " " " "*" " " "*"
## 30 ( 1 ) "*" " " " " " " "*" " " "*"
## 31 ( 1 ) "*" " " " " " " "*" " " "*"
## 32 ( 1 ) "*" " " " " " " "*" " " "*"
## 33 ( 1 ) "*" " " " " " " "*" " " "*"
## 34 ( 1 ) "*" " " " " " " "*" " " "*"
## 35 ( 1 ) "*" " " " " " " "*" " " "*"
## 36 ( 1 ) "*" " " " " " " "*" " " "*"
## 37 ( 1 ) "*" " " " " " " "*" " " "*"
## 38 ( 1 ) "*" " " " " " " "*" " " "*"
## 39 ( 1 ) "*" " " " " " " "*" " " "*"
## 40 ( 1 ) "*" " " " " " " "*" " " "*"
## 41 ( 1 ) "*" " " "*" " " "*" " " "*"
## 42 ( 1 ) "*" " " "*" " " "*" " " "*"
## 43 ( 1 ) "*" " " "*" " " "*" " " "*"
## 44 ( 1 ) "*" " " "*" " " "*" " " "*"
## 45 ( 1 ) "*" " " "*" " " "*" " " "*"
## 46 ( 1 ) "*" " " "*" " " "*" " " "*"
## 47 ( 1 ) "*" " " "*" " " "*" " " "*"
## 48 ( 1 ) "*" " " "*" " " "*" " " "*"
## 49 ( 1 ) "*" " " "*" " " "*" " " "*"
## 50 ( 1 ) "*" " " "*" " " "*" " " "*"
## 51 ( 1 ) "*" " " "*" " " "*" " " "*"
## 52 ( 1 ) "*" " " "*" " " "*" " " "*"
## 53 ( 1 ) "*" " " "*" " " "*" " " "*"
## 54 ( 1 ) "*" " " "*" " " "*" " " "*"
## 55 ( 1 ) "*" " " "*" " " "*" " " "*"
## 56 ( 1 ) "*" " " "*" " " "*" " " "*"
## 57 ( 1 ) "*" " " "*" " " "*" " " "*"
## 58 ( 1 ) "*" " " "*" " " "*" " " "*"
## 59 ( 1 ) "*" " " "*" " " "*" " " "*"
## 60 ( 1 ) "*" " " "*" " " "*" " " "*"
## 61 ( 1 ) "*" " " "*" " " "*" " " "*"
## 62 ( 1 ) "*" " " "*" " " "*" " " "*"
## 63 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 64 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 65 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 66 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 67 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 68 ( 1 ) "*" " " "*" "*" "*" " " "*"
## 69 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 70 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" " " "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_08C ` `C_09A ` `C_09B ` `C_09C ` `C_10A ` `C_10B ` `C_10C `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " " " " " " "
## 16 ( 1 ) " " " " " " " " "*" " " " "
## 17 ( 1 ) " " " " " " " " "*" " " " "
## 18 ( 1 ) " " " " " " " " "*" " " " "
## 19 ( 1 ) " " " " " " " " "*" " " " "
## 20 ( 1 ) " " " " " " " " "*" " " " "
## 21 ( 1 ) " " " " " " " " "*" " " " "
## 22 ( 1 ) " " " " " " " " "*" " " " "
## 23 ( 1 ) " " " " " " " " "*" " " " "
## 24 ( 1 ) " " " " " " " " "*" " " " "
## 25 ( 1 ) " " " " " " "*" "*" " " " "
## 26 ( 1 ) " " " " " " "*" "*" " " " "
## 27 ( 1 ) " " " " " " "*" "*" " " " "
## 28 ( 1 ) " " " " " " "*" "*" " " " "
## 29 ( 1 ) " " " " " " "*" "*" " " " "
## 30 ( 1 ) " " " " " " "*" "*" " " " "
## 31 ( 1 ) " " " " " " "*" "*" " " " "
## 32 ( 1 ) " " " " " " "*" "*" " " " "
## 33 ( 1 ) " " " " " " "*" "*" " " " "
## 34 ( 1 ) " " " " " " "*" "*" " " " "
## 35 ( 1 ) " " " " " " "*" "*" " " " "
## 36 ( 1 ) " " " " " " "*" "*" " " " "
## 37 ( 1 ) " " " " "*" "*" "*" " " " "
## 38 ( 1 ) " " " " "*" "*" "*" " " " "
## 39 ( 1 ) " " " " "*" "*" "*" " " " "
## 40 ( 1 ) " " " " "*" "*" "*" " " "*"
## 41 ( 1 ) " " " " "*" "*" "*" " " "*"
## 42 ( 1 ) " " " " "*" "*" "*" " " "*"
## 43 ( 1 ) " " " " "*" "*" "*" " " "*"
## 44 ( 1 ) " " " " "*" "*" "*" " " "*"
## 45 ( 1 ) " " " " "*" "*" "*" " " "*"
## 46 ( 1 ) " " " " "*" "*" "*" " " "*"
## 47 ( 1 ) " " " " "*" "*" "*" " " "*"
## 48 ( 1 ) " " " " "*" "*" "*" " " "*"
## 49 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 50 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 51 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 52 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 53 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 54 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 55 ( 1 ) " " " " "*" "*" "*" "*" "*"
## 56 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 57 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 58 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 59 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 60 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 61 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 62 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 63 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 64 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 65 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 66 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 67 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 68 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 69 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 70 ( 1 ) " " "*" "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_10D ` `C_10E ` `C_10F ` `C_10G ` `C_10H ` `C_10I ` `C_10J `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " "*" " " " " " " " "
## 6 ( 1 ) " " " " "*" " " " " " " " "
## 7 ( 1 ) " " " " "*" " " " " " " " "
## 8 ( 1 ) " " " " "*" " " " " " " " "
## 9 ( 1 ) " " " " "*" " " " " " " " "
## 10 ( 1 ) " " " " "*" " " " " " " " "
## 11 ( 1 ) " " " " "*" " " " " " " " "
## 12 ( 1 ) " " " " "*" " " " " " " " "
## 13 ( 1 ) " " " " "*" " " " " " " " "
## 14 ( 1 ) " " " " "*" " " " " " " " "
## 15 ( 1 ) " " " " "*" " " " " " " " "
## 16 ( 1 ) " " " " "*" " " " " " " " "
## 17 ( 1 ) " " " " "*" " " " " " " " "
## 18 ( 1 ) " " " " "*" " " " " " " " "
## 19 ( 1 ) " " " " "*" " " " " " " " "
## 20 ( 1 ) " " " " "*" " " " " " " " "
## 21 ( 1 ) " " " " "*" " " " " " " " "
## 22 ( 1 ) " " " " "*" " " " " " " " "
## 23 ( 1 ) " " "*" "*" " " " " " " " "
## 24 ( 1 ) " " "*" "*" " " " " " " " "
## 25 ( 1 ) " " "*" "*" " " " " " " " "
## 26 ( 1 ) " " "*" "*" " " " " " " " "
## 27 ( 1 ) " " "*" "*" " " " " " " " "
## 28 ( 1 ) " " "*" "*" " " " " " " "*"
## 29 ( 1 ) " " "*" "*" " " " " " " "*"
## 30 ( 1 ) " " "*" "*" " " " " " " "*"
## 31 ( 1 ) "*" "*" "*" " " " " " " "*"
## 32 ( 1 ) "*" "*" "*" " " " " " " "*"
## 33 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 34 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 35 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 36 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 37 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 38 ( 1 ) "*" "*" "*" " " " " "*" "*"
## 39 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 40 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 41 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 42 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 43 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 44 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 45 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 46 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 47 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 48 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 49 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 50 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 51 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 52 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 53 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 54 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 55 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 56 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 57 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 58 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 59 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 60 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 61 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 62 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 63 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 64 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 65 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 66 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 67 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 68 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 69 ( 1 ) "*" "*" "*" " " "*" "*" "*"
## 70 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 71 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 72 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 73 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 74 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 75 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*" "*"
## `C_10K ` `C_16B ` `C_18D ` `C_18F ` `C_18G ` `C_18I ` `C_20A `
## 1 ( 1 ) " " " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " " " " " " "
## 16 ( 1 ) " " " " " " " " " " " " " "
## 17 ( 1 ) " " " " " " " " " " " " " "
## 18 ( 1 ) " " " " " " " " " " " " " "
## 19 ( 1 ) " " " " " " " " " " " " " "
## 20 ( 1 ) " " " " " " " " " " " " " "
## 21 ( 1 ) " " " " " " " " " " " " " "
## 22 ( 1 ) " " " " " " " " " " " " " "
## 23 ( 1 ) " " " " " " " " " " " " " "
## 24 ( 1 ) " " " " " " " " " " " " " "
## 25 ( 1 ) " " " " " " " " " " " " " "
## 26 ( 1 ) " " " " " " " " " " " " " "
## 27 ( 1 ) " " " " " " " " " " " " " "
## 28 ( 1 ) " " " " " " " " " " " " " "
## 29 ( 1 ) " " " " " " " " " " " " " "
## 30 ( 1 ) " " " " " " " " " " " " " "
## 31 ( 1 ) " " " " " " " " " " " " " "
## 32 ( 1 ) " " " " " " " " " " " " " "
## 33 ( 1 ) " " " " " " " " " " " " " "
## 34 ( 1 ) " " " " " " " " " " " " " "
## 35 ( 1 ) " " " " " " " " " " " " " "
## 36 ( 1 ) " " " " " " " " " " " " " "
## 37 ( 1 ) " " " " " " " " " " " " " "
## 38 ( 1 ) " " " " " " " " " " " " " "
## 39 ( 1 ) " " " " " " " " " " " " " "
## 40 ( 1 ) " " " " " " " " " " " " " "
## 41 ( 1 ) " " " " " " " " " " " " " "
## 42 ( 1 ) " " " " " " " " " " " " " "
## 43 ( 1 ) " " " " " " " " " " " " " "
## 44 ( 1 ) " " " " " " " " " " " " " "
## 45 ( 1 ) " " " " " " " " "*" " " " "
## 46 ( 1 ) " " " " " " " " "*" " " " "
## 47 ( 1 ) " " " " " " " " "*" " " " "
## 48 ( 1 ) " " " " " " " " "*" " " " "
## 49 ( 1 ) " " " " " " " " "*" " " " "
## 50 ( 1 ) " " " " " " " " "*" " " " "
## 51 ( 1 ) "*" " " " " " " "*" " " " "
## 52 ( 1 ) "*" " " " " " " "*" " " " "
## 53 ( 1 ) "*" " " " " " " "*" " " " "
## 54 ( 1 ) "*" " " " " " " "*" " " " "
## 55 ( 1 ) "*" " " " " " " "*" " " " "
## 56 ( 1 ) "*" " " " " " " "*" " " " "
## 57 ( 1 ) "*" " " " " " " "*" " " " "
## 58 ( 1 ) "*" " " " " " " "*" " " " "
## 59 ( 1 ) "*" " " " " " " "*" " " " "
## 60 ( 1 ) "*" " " " " " " "*" " " "*"
## 61 ( 1 ) "*" " " " " " " "*" " " "*"
## 62 ( 1 ) "*" " " " " "*" "*" " " "*"
## 63 ( 1 ) "*" " " " " "*" "*" " " "*"
## 64 ( 1 ) "*" " " " " "*" "*" " " "*"
## 65 ( 1 ) "*" " " " " "*" "*" " " "*"
## 66 ( 1 ) "*" " " " " "*" "*" " " "*"
## 67 ( 1 ) "*" " " " " "*" "*" " " "*"
## 68 ( 1 ) "*" " " " " "*" "*" " " "*"
## 69 ( 1 ) "*" " " " " "*" "*" " " "*"
## 70 ( 1 ) "*" " " " " "*" "*" " " "*"
## 71 ( 1 ) "*" " " " " "*" "*" " " "*"
## 72 ( 1 ) "*" " " " " "*" "*" " " "*"
## 73 ( 1 ) "*" " " " " "*" "*" " " "*"
## 74 ( 1 ) "*" " " " " "*" "*" " " "*"
## 75 ( 1 ) "*" " " " " "*" "*" " " "*"
## 76 ( 1 ) "*" " " " " "*" "*" " " "*"
## 77 ( 1 ) "*" " " " " "*" "*" " " "*"
## 78 ( 1 ) "*" " " " " "*" "*" " " "*"
## `C_20D ` `C_20E ` `C_20F ` `C_22A ` `C_22C ` `C_99B `
## 1 ( 1 ) " " " " " " " " " " " "
## 2 ( 1 ) " " " " " " " " " " " "
## 3 ( 1 ) " " " " " " " " " " " "
## 4 ( 1 ) " " " " " " " " " " " "
## 5 ( 1 ) " " " " " " " " " " " "
## 6 ( 1 ) " " " " " " " " " " " "
## 7 ( 1 ) " " " " " " " " " " " "
## 8 ( 1 ) " " " " " " " " " " " "
## 9 ( 1 ) " " " " " " " " " " " "
## 10 ( 1 ) " " " " " " " " " " " "
## 11 ( 1 ) " " " " " " " " " " " "
## 12 ( 1 ) " " " " " " " " " " " "
## 13 ( 1 ) " " " " " " " " " " " "
## 14 ( 1 ) " " " " " " " " " " " "
## 15 ( 1 ) " " " " " " " " " " " "
## 16 ( 1 ) " " " " " " " " " " " "
## 17 ( 1 ) " " " " " " " " " " " "
## 18 ( 1 ) " " " " " " " " " " " "
## 19 ( 1 ) " " " " " " " " " " " "
## 20 ( 1 ) " " " " " " " " " " " "
## 21 ( 1 ) " " " " " " " " " " " "
## 22 ( 1 ) " " " " " " " " " " " "
## 23 ( 1 ) " " " " " " " " " " " "
## 24 ( 1 ) " " " " " " " " " " " "
## 25 ( 1 ) " " " " " " " " " " " "
## 26 ( 1 ) " " " " " " " " " " " "
## 27 ( 1 ) " " " " " " " " " " " "
## 28 ( 1 ) " " " " " " " " " " " "
## 29 ( 1 ) " " " " " " " " " " " "
## 30 ( 1 ) " " " " " " " " " " " "
## 31 ( 1 ) " " " " " " " " " " " "
## 32 ( 1 ) "*" " " " " " " " " " "
## 33 ( 1 ) "*" " " " " " " " " " "
## 34 ( 1 ) "*" " " " " " " " " " "
## 35 ( 1 ) "*" " " " " " " " " " "
## 36 ( 1 ) "*" " " " " " " " " " "
## 37 ( 1 ) "*" " " " " " " " " " "
## 38 ( 1 ) "*" " " " " " " " " " "
## 39 ( 1 ) "*" " " " " " " " " " "
## 40 ( 1 ) "*" " " " " " " " " " "
## 41 ( 1 ) "*" " " " " " " " " " "
## 42 ( 1 ) "*" " " " " " " " " "*"
## 43 ( 1 ) "*" " " " " " " " " "*"
## 44 ( 1 ) "*" " " " " " " " " "*"
## 45 ( 1 ) "*" " " " " " " " " "*"
## 46 ( 1 ) "*" " " " " " " " " "*"
## 47 ( 1 ) "*" " " " " " " " " "*"
## 48 ( 1 ) "*" " " " " " " " " "*"
## 49 ( 1 ) "*" " " " " " " " " "*"
## 50 ( 1 ) "*" " " " " " " " " "*"
## 51 ( 1 ) "*" " " " " " " " " "*"
## 52 ( 1 ) "*" " " " " " " " " "*"
## 53 ( 1 ) "*" " " " " " " " " "*"
## 54 ( 1 ) "*" " " " " " " " " "*"
## 55 ( 1 ) "*" " " " " " " " " "*"
## 56 ( 1 ) "*" " " " " " " " " "*"
## 57 ( 1 ) "*" " " " " " " " " "*"
## 58 ( 1 ) "*" " " " " " " " " "*"
## 59 ( 1 ) "*" " " " " " " " " "*"
## 60 ( 1 ) "*" " " " " " " " " "*"
## 61 ( 1 ) "*" " " " " " " " " "*"
## 62 ( 1 ) "*" " " " " " " " " "*"
## 63 ( 1 ) "*" " " " " " " " " "*"
## 64 ( 1 ) "*" "*" " " " " " " "*"
## 65 ( 1 ) "*" "*" " " " " " " "*"
## 66 ( 1 ) "*" "*" " " " " "*" "*"
## 67 ( 1 ) "*" "*" "*" " " "*" "*"
## 68 ( 1 ) "*" "*" "*" " " "*" "*"
## 69 ( 1 ) "*" "*" "*" " " "*" "*"
## 70 ( 1 ) "*" "*" "*" " " "*" "*"
## 71 ( 1 ) "*" "*" "*" " " "*" "*"
## 72 ( 1 ) "*" "*" "*" " " "*" "*"
## 73 ( 1 ) "*" "*" "*" " " "*" "*"
## 74 ( 1 ) "*" "*" "*" " " "*" "*"
## 75 ( 1 ) "*" "*" "*" " " "*" "*"
## 76 ( 1 ) "*" "*" "*" "*" "*" "*"
## 77 ( 1 ) "*" "*" "*" "*" "*" "*"
## 78 ( 1 ) "*" "*" "*" "*" "*" "*"reg.summary$rsq## [1] 0.1653968 0.2091791 0.2232770 0.2293543 0.2336068 0.2355835 0.2371433
## [8] 0.2386633 0.2399349 0.2406206 0.2413019 0.2418432 0.2422798 0.2426984
## [15] 0.2430421 0.2433268 0.2435413 0.2437368 0.2439413 0.2440861 0.2442462
## [22] 0.2444203 0.2445362 0.2446401 0.2447428 0.2448317 0.2449112 0.2449841
## [29] 0.2450419 0.2450911 0.2451286 0.2451560 0.2451831 0.2452064 0.2452294
## [36] 0.2452524 0.2452714 0.2452893 0.2453051 0.2453199 0.2453332 0.2453437
## [43] 0.2453538 0.2453621 0.2453700 0.2453770 0.2453840 0.2453900 0.2453960
## [50] 0.2454022 0.2454069 0.2454111 0.2454151 0.2454185 0.2454216 0.2454242
## [57] 0.2454267 0.2454285 0.2454303 0.2454316 0.2454327 0.2454339 0.2454349
## [64] 0.2454358 0.2454366 0.2454374 0.2454380 0.2454385 0.2454389 0.2454393
## [71] 0.2454396 0.2454398 0.2454400 0.2454401 0.2454402 0.2454402 0.2454402
## [78] 0.2454402plot(reg.summary$rsq,xlab="Number of variables", ylab="R-Square", type="l")
points(1:20,reg.summary$rsq[1:20], col="red",cex=2,pch=20)
Figure 2: Plot between Number of variables and R-Square
The Figure-2 above displays that after approximately 5 variables, there is not really significant improvement in R-Square (Greater the R-Square, better is the model). Now we will evaluate various models on the following 5 variables to improve the R-Square further. The following variables are obtained from the display obtained from the command “summary(regfit.fwd)”. At the 5th row, where ever we see an “*“, those 5 variables are selected. Here are those 5 variables
- visit_level (This represents the number of the visit/inspection)
- score (Restaurant’s inspection score)
- C_000 (If no violations are found, then it will have 1 or else 0)
- C_04L (Evidence of mice or live mice present in facility’s food and/or non-food areas)
- C_10F (Non-food contact surface improperly constructed. Unacceptable material used. Non-food contact surface or equipment improperly maintained and/or not properly sealed, raised, spaced or movable to allow accessibility for cleaning on all sides, above and underneath the unit)
NOTE The violation_data data frame has all the violation codes and descriptions. You may use the commands, as given below to find the violation codes (as per the need)
violation_data[violation_data$code == "04L ",]## code
## 56 04L
## description
## 56 Evidence of mice or live mice present in facility's food and/or non-food areas.
## level
## 56 Criticalviolation_data[violation_data$code == "10F ",]## code
## 55 10F
## description
## 55 Non-food contact surface improperly constructed. Unacceptable material used. Non-food contact surface or equipment improperly maintained and/or not properly sealed, raised, spaced or movable to allow accessibility for cleaning on all sides, above and underneath the unit.
## level
## 55 Not CriticalLet us pair plot the 5 variables along with the days_diff. This will help us, if we have any non-linear relationship with the days_diff and any of the 5 variables identified above.
par(mfrow=c(3,2))
plot(days_diff_training$visit_level,days_diff_training$days_diff,xlab="Visit Level",ylab="days_diff",col="green")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$visit_level)
abline(temp_lm,col="red")
plot(days_diff_training$score,days_diff_training$days_diff,xlab="Score",ylab="days_diff",col="green")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$score)
abline(temp_lm,col="red")
plot(days_diff_training$C_000,days_diff_training$days_diff,xlab="C_000",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_000)
abline(temp_lm,col="red")
plot(days_diff_training$C_04L,days_diff_training$days_diff,xlab="C_04L",ylab="days_diff",col="blue")
plot(days_diff_training$C_10F,days_diff_training$days_diff,xlab="C_10F",ylab="days_diff",col="blue")
temp_lm <- lm(days_diff_training$days_diff~days_diff_training$C_10F)
abline(temp_lm,col="red")
Figure 3: Graphs showing the plots between days_diff and other selected variables
From the above display we can conclude that the score and days_diff are not linearly related. Also we can see that there is some strong relationship between the score, visit_level and the days_diff variables. Let us use a functional form by introducing interaction terms between score and other variables. We will use the glm() function (glm stands for Generalized Additive Models), evaluate the model with different degrees of freedom (by using the K-Fold cross validation), and choose the optimal degree of freedom. The R code is given below:
The following R-Code is commented completely, since this will execute for a while, given the amount of training data we have.
#cv.error.5 <- rep(0,5)
#for (i in 1:5)
#{
# print(i)
# glm.fit <-glm((days_diff_training$days_diff~(days_diff_training$visit_level+days_diff_training$C_000+days_diff_training$C_04L+days_diff_training$C_10F+days_diff_training$visit_level+days_diff_training$visit_level)*poly(days_diff_training$score,i, raw=TRUE)))
# cv.error.5[i] <- cv.glm(days_diff_training,glm.fit,K=5)$delta[1]
# print(cv.error.5[i])
#}
#cv.error.5
#which(min(cv.error.5) == cv.error.5)The above R code has given the following output:
[1] 1 [1] 19807.2 [1] 2 [1] 20314.38 [1] 3 [1] 20343.02 [1] 4 [1] 20428.05 [1] 5 [1] 20662.53
cv.error.5 [1] 19807.20 20314.38 20343.02 20428.05 20662.53
which(min(cv.error.5) == cv.error.5) [1] 1
The above output shows that the polynomial level of degree has the minimal cross validation error. So we will be using the following function to predict days_diff variable:
glm.fit <-glm((days_diff_training$days_diff~(days_diff_training$visit_level+days_diff_training$C_000+days_diff_training$C_04L+days_diff_training$C_10F+days_diff_training$visit_level+days_diff_training$visit_level)*poly(days_diff_training$score,1, raw=TRUE)))
glm.fit##
## Call: glm(formula = (days_diff_training$days_diff ~ (days_diff_training$visit_level +
## days_diff_training$C_000 + days_diff_training$C_04L + days_diff_training$C_10F +
## days_diff_training$visit_level + days_diff_training$visit_level) *
## poly(days_diff_training$score, 1, raw = TRUE)))
##
## Coefficients:
## (Intercept)
## 268.8084
## days_diff_training$visit_level
## -10.1992
## days_diff_training$C_000
## -115.3469
## days_diff_training$C_04L
## -72.8362
## days_diff_training$C_10F
## 25.2543
## poly(days_diff_training$score, 1, raw = TRUE)
## -6.7881
## days_diff_training$visit_level:poly(days_diff_training$score, 1, raw = TRUE)
## 0.3233
## days_diff_training$C_000:poly(days_diff_training$score, 1, raw = TRUE)
## NA
## days_diff_training$C_04L:poly(days_diff_training$score, 1, raw = TRUE)
## 2.6036
## days_diff_training$C_10F:poly(days_diff_training$score, 1, raw = TRUE)
## -0.6635
##
## Degrees of Freedom: 129292 Total (i.e. Null); 129284 Residual
## Null Deviance: 2.04e+09
## Residual Deviance: 1.521e+09 AIC: 1579000\[f(daysdiff)=268.8084-10.1992(visitlevel)-115.3469(C000)-72.8362(C04L)+25.2543(C10F)-6.7881(score)+0.3233(visit_level)(score)+2.6036(C04L)(score)-0.6635(C10F)(score)\]