Andmestiku flights.csv puhastamine ja analüüs
Veronica Vinkler ja Dominic Arkhan Baum
2025-05-26
Käesolevas õppematerjalis on esitatud andmestiku puhastamise, ettevalmistamise ja kirjeldava analüüsi läbiviimise skeem andmestiku flights.csv näitel, mis sisaldab Ameerika Ühendriikide transpordiministeeriumi (DOT) transpordistatistika büroo poolt jälgitud suurte lennuettevõtjate 2015 aasta siselendude hilinemiste ja tühistamiste andmeid. Antud töös olevad lahendatud ülesanded vastavad Seminaritöö 1 ja 2 ülesannetele.
Flights.csv andmestik on kättesaadav lehelt: https://www.kaggle.com/datasets/usdot/flight-delays?resource=download
UA- United Air Lines Inc. AA- American Airlines Inc. US- US Airways Inc. F9- Frontier Airlines Inc. B6- JetBlue Airways 00- Skywest Airlines Inc. AS- Alaska Airlines Inc. NK- Spirit Air Lines WN- Southwest Airlines Co. DL- Delta Air Lines Inc. EV- Atlantic Southeast Airlines HA- Hawaiian Airlines Inc. MQ- American Eagle Airlines Inc. VX- Virgin America
Andmete eeltöötlus
Andmete eeltöötluse etapil toimub andmete edasiseks analüüsiks ettevalmistamine: andmete lugemine, tunnuste tüüpide kontrollimine, puhastamine (vigaste väärtuste kontroll ja eemaldamine, duplikaatide eemaldamine, puuduvate väärtuste kontroll ja asendamine või eemaldamine, mittesobivate tunnuste kustutamine) ja ka tunnuste teisendamine ning uute tunnuste moodustamine.
Andmete lugemine
flight failist Loeme andmeid R-i failist flights.csv, andmestik flights.csv üle 1500nda rea. Üle rea lugemiseks kasutame seq() funktsiooni.
flight <- read.csv("flights.csv", header=TRUE, stringsAsFactors = FALSE)
flight <- flight[seq(1, nrow(flight), by = 1500), ]Andmetabeli ülevaade
Et saada andmestikust ülevaadet väljastame andmestiku tabelina. Selleks kasutame funktsiooni datatable() paketist “DT” ja library(“DT”).
library(DT)
datatable(flight, options=list(scrollx=1, pageLenght=10, searching=FALSE, scroller=TRUE, scrolly=200))Ülevaade andmestiku struktuurist
Kasutame funktsiooni str() andmestiku struktuuri ülevaatamiseks
str(flight)## 'data.frame': 3880 obs. of 31 variables:
## $ YEAR : int 2015 2015 2015 2015 2015 2015 2015 2015 2015 2015 ...
## $ MONTH : int 1 1 1 1 1 1 1 1 1 1 ...
## $ DAY : int 1 1 1 1 1 1 1 1 1 1 ...
## $ DAY_OF_WEEK : int 4 4 4 4 4 4 4 4 4 4 ...
## $ AIRLINE : chr "AS" "UA" "WN" "UA" ...
## $ FLIGHT_NUMBER : int 98 1440 2002 1220 340 2620 554 960 5434 3344 ...
## $ TAIL_NUMBER : chr "N407AS" "N47414" "N8601C" "N63820" ...
## $ ORIGIN_AIRPORT : chr "ANC" "ORD" "LAS" "DEN" ...
## $ DESTINATION_AIRPORT: chr "SEA" "FLL" "TPA" "SEA" ...
## $ SCHEDULED_DEPARTURE: int 5 754 940 1119 1252 1425 1600 1740 1923 2145 ...
## $ DEPARTURE_TIME : int 2354 754 939 1255 1254 1424 1556 1814 1915 2146 ...
## $ DEPARTURE_DELAY : int -11 0 -1 96 2 -1 -4 34 -8 1 ...
## $ TAXI_OUT : int 21 15 12 12 12 15 12 9 8 12 ...
## $ WHEELS_OFF : int 15 809 951 1307 1306 1439 1608 1823 1923 2158 ...
## $ SCHEDULED_TIME : int 205 194 250 181 32 45 110 140 61 48 ...
## $ ELAPSED_TIME : int 194 173 245 151 35 49 94 130 47 47 ...
## $ AIR_TIME : int 169 147 230 131 17 29 77 117 30 26 ...
## $ DISTANCE : int 1448 1182 1984 1024 84 109 515 763 118 157 ...
## $ WHEELS_ON : int 404 1136 1641 1418 1323 1508 1725 2020 1853 2324 ...
## $ TAXI_IN : int 4 11 3 8 6 5 5 4 9 9 ...
## $ SCHEDULED_ARRIVAL : int 430 1208 1650 1320 1324 1510 1750 2000 1924 2333 ...
## $ ARRIVAL_TIME : int 408 1147 1644 1426 1329 1513 1730 2024 1902 2333 ...
## $ ARRIVAL_DELAY : int -22 -21 -6 66 5 3 -20 24 -22 0 ...
## $ DIVERTED : int 0 0 0 0 0 0 0 0 0 0 ...
## $ CANCELLED : int 0 0 0 0 0 0 0 0 0 0 ...
## $ CANCELLATION_REASON: chr "" "" "" "" ...
## $ AIR_SYSTEM_DELAY : int NA NA NA 0 NA NA NA 0 NA NA ...
## $ SECURITY_DELAY : int NA NA NA 0 NA NA NA 0 NA NA ...
## $ AIRLINE_DELAY : int NA NA NA 47 NA NA NA 11 NA NA ...
## $ LATE_AIRCRAFT_DELAY: int NA NA NA 19 NA NA NA 13 NA NA ...
## $ WEATHER_DELAY : int NA NA NA 0 NA NA NA 0 NA NA ...Funktsiooni str() rakendamise tulemusel saime teada, et andmestikus on 3880 objekti ja 31 muutujat. Tundub ka, et kõik andmed on õigesti loetud, kuid on palju puuduvaid väärtusi.
Teeme andmestiust parema ja detailsema ülevaate paketi skimr funktsiooni abil.
library(skimr)
skim(flight)| Name | flight |
| Number of rows | 3880 |
| Number of columns | 31 |
| _______________________ | |
| Column type frequency: | |
| character | 5 |
| numeric | 26 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| AIRLINE | 0 | 1 | 2 | 2 | 0 | 14 | 0 |
| TAIL_NUMBER | 0 | 1 | 0 | 6 | 16 | 2450 | 0 |
| ORIGIN_AIRPORT | 0 | 1 | 3 | 5 | 0 | 326 | 0 |
| DESTINATION_AIRPORT | 0 | 1 | 3 | 5 | 0 | 331 | 0 |
| CANCELLATION_REASON | 0 | 1 | 0 | 1 | 3801 | 4 | 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| YEAR | 0 | 1.00 | 2015.00 | 0.00 | 2015 | 2015 | 2015.0 | 2015.00 | 2015 | ▁▁▇▁▁ |
| MONTH | 0 | 1.00 | 6.52 | 3.41 | 1 | 4 | 7.0 | 9.00 | 12 | ▇▆▆▆▇ |
| DAY | 0 | 1.00 | 15.70 | 8.79 | 1 | 8 | 16.0 | 23.00 | 31 | ▇▇▇▇▆ |
| DAY_OF_WEEK | 0 | 1.00 | 3.92 | 1.99 | 1 | 2 | 4.0 | 6.00 | 7 | ▇▃▅▅▇ |
| FLIGHT_NUMBER | 0 | 1.00 | 2179.29 | 1743.87 | 2 | 777 | 1680.5 | 3193.50 | 7432 | ▇▅▂▂▁ |
| SCHEDULED_DEPARTURE | 0 | 1.00 | 1327.99 | 481.90 | 5 | 915 | 1320.0 | 1730.00 | 2359 | ▁▇▇▇▅ |
| DEPARTURE_TIME | 78 | 0.98 | 1332.88 | 495.12 | 1 | 920 | 1326.5 | 1735.00 | 2359 | ▁▇▇▇▅ |
| DEPARTURE_DELAY | 78 | 0.98 | 9.13 | 34.23 | -20 | -5 | -2.0 | 7.00 | 623 | ▇▁▁▁▁ |
| TAXI_OUT | 79 | 0.98 | 16.15 | 9.18 | 1 | 11 | 14.0 | 19.00 | 145 | ▇▁▁▁▁ |
| WHEELS_OFF | 79 | 0.98 | 1355.07 | 496.67 | 1 | 933 | 1340.0 | 1750.00 | 2400 | ▁▇▇▇▅ |
| SCHEDULED_TIME | 0 | 1.00 | 140.53 | 73.26 | 24 | 86 | 123.0 | 170.00 | 640 | ▇▃▁▁▁ |
| ELAPSED_TIME | 92 | 0.98 | 135.67 | 72.21 | 27 | 82 | 118.0 | 166.00 | 635 | ▇▃▁▁▁ |
| AIR_TIME | 92 | 0.98 | 112.14 | 70.29 | 14 | 60 | 95.0 | 143.00 | 613 | ▇▃▁▁▁ |
| DISTANCE | 0 | 1.00 | 810.81 | 591.77 | 41 | 373 | 650.5 | 1050.00 | 4983 | ▇▂▁▁▁ |
| WHEELS_ON | 80 | 0.98 | 1468.93 | 522.59 | 1 | 1058 | 1503.0 | 1912.00 | 2359 | ▁▅▇▇▇ |
| TAXI_IN | 80 | 0.98 | 7.42 | 5.32 | 1 | 4 | 6.0 | 9.00 | 69 | ▇▁▁▁▁ |
| SCHEDULED_ARRIVAL | 0 | 1.00 | 1493.29 | 506.07 | 5 | 1113 | 1516.0 | 1922.25 | 2359 | ▁▅▇▇▇ |
| ARRIVAL_TIME | 80 | 0.98 | 1473.39 | 527.06 | 1 | 1103 | 1507.5 | 1916.00 | 2400 | ▁▅▇▇▇ |
| ARRIVAL_DELAY | 92 | 0.98 | 4.08 | 36.90 | -57 | -13 | -5.0 | 8.00 | 673 | ▇▁▁▁▁ |
| DIVERTED | 0 | 1.00 | 0.00 | 0.06 | 0 | 0 | 0.0 | 0.00 | 1 | ▇▁▁▁▁ |
| CANCELLED | 0 | 1.00 | 0.02 | 0.14 | 0 | 0 | 0.0 | 0.00 | 1 | ▇▁▁▁▁ |
| AIR_SYSTEM_DELAY | 3165 | 0.18 | 14.45 | 27.60 | 0 | 0 | 2.0 | 19.00 | 235 | ▇▁▁▁▁ |
| SECURITY_DELAY | 3165 | 0.18 | 0.13 | 2.23 | 0 | 0 | 0.0 | 0.00 | 56 | ▇▁▁▁▁ |
| AIRLINE_DELAY | 3165 | 0.18 | 17.18 | 36.19 | 0 | 0 | 2.0 | 19.00 | 308 | ▇▁▁▁▁ |
| LATE_AIRCRAFT_DELAY | 3165 | 0.18 | 21.64 | 39.96 | 0 | 0 | 2.0 | 28.00 | 397 | ▇▁▁▁▁ |
| WEATHER_DELAY | 3165 | 0.18 | 3.84 | 22.59 | 0 | 0 | 0.0 | 0.00 | 314 | ▇▁▁▁▁ |
Funktsioon skim() annab teada, et andmestikus on 5 sümbolitüüpi ja 26 arvulist tunnust. Faktortunnuse tabeli põhjal näeme, et andmestikus on puuduvate väärtustega väärtusi 0, tühjasi lahtreid on 16 ORIGIN_AIRPORT muutujal ja 3801 CANCELLATION_REASON muutujal.
Numbriliste väärtuste tabelis näeme, et puuduvaid väärtusi on muutujatel DEPARTURE_TIME, DEPARTURE_DELAY- 78, TAXI_OUT, WHEELS_OFF- 79, ELAPSED_TIME, AIR_TIME- 92, WHEELS_ON, TAXI_IN, ARRIVAL_TIME- 80, ARRIVAL_DELAY- 92, AIR_SYSTEM_DELAY, SECURITY_DELAY, AIRLINE_DELAY, LATE_AIRCRAFT_DELAY, WEATHER_DELAY- 3165.
Duplikaatides me ülevaadet ei teinud kuna neid ei olnud.
Andmestiku puhastamine
Andmestiku ülevaade näitab, et andmestikus puuduvate väärtustega objekte ja duplikaate pole. Kui need on, siis duplikaate ja puuduvate andmetega objekte tuleb eemaldada või puuduvaid väärtusi imputeerida.
Duplikaatide kontroll
kontrollime duplikaatide olemasolu duplicated() abil.
sum(duplicated(flight))## [1] 0Andmestkus duplikaadid puuduvad.
Puuduvate väärtuste kontroll
Puuduvate väärtuste olemsolu kontrollimiseks kasutatakse funktsiooni is.na(). Leiame puuduvate väärtuste koguarvu andmestikus. Puuduvate väärtuste koguarv on r sum(is.na(air))
sum(!complete.cases(flight))## [1] 3165Puuduvate väärtuste koguarv on 3165. Varasemast tabeli ülevaatest teame, et ilmselt on see seotud muutujatega AIR_SYSTEM_DELAY, SECURITY_DELAY, AIRLINE_DELAY, LATE_AIRCRAFT_DELAY ja WEATHER_DELAY.
Toome veel korra välja parema ülevaate jaoks mitu puuduvat väärtust igal tunnusel on funktsiooni apply() abil.
apply(is.na(flight), 2, sum)## YEAR MONTH DAY DAY_OF_WEEK
## 0 0 0 0
## AIRLINE FLIGHT_NUMBER TAIL_NUMBER ORIGIN_AIRPORT
## 0 0 0 0
## DESTINATION_AIRPORT SCHEDULED_DEPARTURE DEPARTURE_TIME DEPARTURE_DELAY
## 0 0 78 78
## TAXI_OUT WHEELS_OFF SCHEDULED_TIME ELAPSED_TIME
## 79 79 0 92
## AIR_TIME DISTANCE WHEELS_ON TAXI_IN
## 92 0 80 80
## SCHEDULED_ARRIVAL ARRIVAL_TIME ARRIVAL_DELAY DIVERTED
## 0 80 92 0
## CANCELLED CANCELLATION_REASON AIR_SYSTEM_DELAY SECURITY_DELAY
## 0 0 3165 3165
## AIRLINE_DELAY LATE_AIRCRAFT_DELAY WEATHER_DELAY
## 3165 3165 3165Puuduvad väärtused on seotud numbriliste tunnustega: DEPARTURE_TIME, DEPARTURE_DELAY- 78, TAXI_OUT, WHEELS_OFF- 79, ELAPSED_TIME, AIR_TIME- 92, WHEELS_ON, TAXI_IN, ARRIVAL_TIME- 80, ARRIVAL_DELAY- 92, AIR_SYSTEM_DELAY, SECURITY_DELAY, AIRLINE_DELAY, LATE_AIRCRAFT_DELAY, WEATHER_DELAY- 3165.
Eemaldame puuduvate väärtustega objekte
Puuduvate väärtustega objektide eemaldamine: Eemaldamiseks kasutame ‘na.omit()’ funktsiooni. Esamlt eemaltame muutjuad AIR_SYSTEM_DELAY, SECURITY_DELAY, AIRLINE_DELAY, LATE_AIRCRAFT_DELAY, WEATHER_DELAY, kuna väärtusi on nendel üle poole puudu.
Samuti eemaldame ka muutuja CANCELLATION_REASON, kuna kuigi puuduvaid väärtusi pole on palju sisuta väärtusi.
Eemaldame ka väärtused CANCELLED ja DIVERTED sisu puudumise tõttu ülevaate põhjal.
flight <- subset(flight, select = -c(AIR_SYSTEM_DELAY, SECURITY_DELAY, LATE_AIRCRAFT_DELAY, WEATHER_DELAY, CANCELLATION_REASON, CANCELLED, DIVERTED))Nüüd eemaldame puuduvad väärtused.
flight <- na.omit(flight)Pärast puuduvate väärtustega objektide eemaldamist kontrollime, et andmestikus poleks rohkem puuduvaid väärtusi:
sum(!complete.cases(flight))## [1] 0Andmestikus nüüd puuduvad väärtused puuduvad.
Eemaldame tunnused madalate varieeruvustega
Tunnused, mille väärtused on peaaegu konstantsed, ei anna meile analüüsimisel palju lisainfot ning nende eemaldamiseks kasutame nearZeroVar() funktsiooni paketist caret:
library(caret)## Loading required package: ggplot2## Loading required package: latticenzv <- nearZeroVar(flight, saveMetrics = TRUE)
flight <- flight[, !nzv$nzv]Pärast eemaldamist teeme uuesti ülevaate andmestikust:
library(skimr)
skim(flight)| Name | flight |
| Number of rows | 715 |
| Number of columns | 23 |
| _______________________ | |
| Column type frequency: | |
| character | 4 |
| numeric | 19 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| AIRLINE | 0 | 1 | 2 | 2 | 0 | 14 | 0 |
| TAIL_NUMBER | 0 | 1 | 5 | 6 | 0 | 644 | 0 |
| ORIGIN_AIRPORT | 0 | 1 | 3 | 5 | 0 | 154 | 0 |
| DESTINATION_AIRPORT | 0 | 1 | 3 | 5 | 0 | 158 | 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| MONTH | 0 | 1 | 6.17 | 3.41 | 1 | 3.0 | 6 | 9.0 | 12 | ▇▅▆▃▆ |
| DAY | 0 | 1 | 15.64 | 8.89 | 1 | 8.0 | 16 | 23.0 | 31 | ▇▇▆▆▆ |
| DAY_OF_WEEK | 0 | 1 | 3.94 | 2.02 | 1 | 2.0 | 4 | 6.0 | 7 | ▇▃▅▅▇ |
| FLIGHT_NUMBER | 0 | 1 | 2147.72 | 1741.50 | 5 | 754.0 | 1585 | 3173.0 | 6539 | ▇▅▂▂▂ |
| SCHEDULED_DEPARTURE | 0 | 1 | 1479.59 | 450.84 | 500 | 1120.5 | 1521 | 1845.0 | 2359 | ▃▆▆▇▅ |
| DEPARTURE_TIME | 0 | 1 | 1533.76 | 486.37 | 7 | 1155.5 | 1607 | 1930.5 | 2359 | ▁▃▇▇▇ |
| DEPARTURE_DELAY | 0 | 1 | 53.56 | 59.44 | -14 | 19.0 | 37 | 71.0 | 623 | ▇▁▁▁▁ |
| TAXI_OUT | 0 | 1 | 21.11 | 15.18 | 4 | 12.0 | 16 | 25.0 | 145 | ▇▁▁▁▁ |
| WHEELS_OFF | 0 | 1 | 1555.88 | 494.19 | 3 | 1211.5 | 1623 | 1947.0 | 2400 | ▁▂▇▇▇ |
| SCHEDULED_TIME | 0 | 1 | 142.44 | 76.79 | 24 | 85.0 | 125 | 173.5 | 544 | ▇▆▂▁▁ |
| ELAPSED_TIME | 0 | 1 | 146.13 | 77.30 | 33 | 89.5 | 129 | 181.0 | 530 | ▇▅▂▁▁ |
| AIR_TIME | 0 | 1 | 116.46 | 74.96 | 14 | 60.5 | 98 | 147.0 | 505 | ▇▅▁▁▁ |
| DISTANCE | 0 | 1 | 833.42 | 628.37 | 41 | 375.0 | 679 | 1069.0 | 4243 | ▇▃▁▁▁ |
| WHEELS_ON | 0 | 1 | 1582.10 | 603.53 | 1 | 1256.5 | 1712 | 2047.0 | 2358 | ▂▁▅▆▇ |
| TAXI_IN | 0 | 1 | 8.56 | 7.88 | 1 | 4.0 | 6 | 10.0 | 69 | ▇▁▁▁▁ |
| SCHEDULED_ARRIVAL | 0 | 1 | 1628.46 | 506.50 | 5 | 1304.0 | 1710 | 2036.5 | 2359 | ▁▂▅▆▇ |
| ARRIVAL_TIME | 0 | 1 | 1577.06 | 616.31 | 2 | 1251.0 | 1718 | 2050.5 | 2359 | ▂▁▅▆▇ |
| ARRIVAL_DELAY | 0 | 1 | 57.25 | 56.82 | 15 | 23.0 | 38 | 68.0 | 673 | ▇▁▁▁▁ |
| AIRLINE_DELAY | 0 | 1 | 17.18 | 36.19 | 0 | 0.0 | 2 | 19.0 | 308 | ▇▁▁▁▁ |
Nüüd on andmestikus alles 19 numbrilist muutujat.
Eemaldame tunnused tugeva korrelatsiooniga
num_tunnused <- Filter(is.numeric, flight)
cor_matrix <- cor(num_tunnused, use = "complete.obs")
library(caret)
high_cor <- findCorrelation(cor_matrix, cutoff = 0.9)
flight <- flight[, -high_cor]
ncol(flight)## [1] 17Teeme uuesti ülevaate allesjäänud väärtustest:
library(skimr)
skim(flight)| Name | flight |
| Number of rows | 715 |
| Number of columns | 17 |
| _______________________ | |
| Column type frequency: | |
| character | 2 |
| numeric | 15 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| AIRLINE | 0 | 1 | 2 | 2 | 0 | 14 | 0 |
| DESTINATION_AIRPORT | 0 | 1 | 3 | 5 | 0 | 158 | 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| MONTH | 0 | 1 | 6.17 | 3.41 | 1 | 3.0 | 6 | 9.0 | 12 | ▇▅▆▃▆ |
| DAY | 0 | 1 | 15.64 | 8.89 | 1 | 8.0 | 16 | 23.0 | 31 | ▇▇▆▆▆ |
| DAY_OF_WEEK | 0 | 1 | 3.94 | 2.02 | 1 | 2.0 | 4 | 6.0 | 7 | ▇▃▅▅▇ |
| FLIGHT_NUMBER | 0 | 1 | 2147.72 | 1741.50 | 5 | 754.0 | 1585 | 3173.0 | 6539 | ▇▅▂▂▂ |
| SCHEDULED_DEPARTURE | 0 | 1 | 1479.59 | 450.84 | 500 | 1120.5 | 1521 | 1845.0 | 2359 | ▃▆▆▇▅ |
| DEPARTURE_TIME | 0 | 1 | 1533.76 | 486.37 | 7 | 1155.5 | 1607 | 1930.5 | 2359 | ▁▃▇▇▇ |
| ELAPSED_TIME | 0 | 1 | 146.13 | 77.30 | 33 | 89.5 | 129 | 181.0 | 530 | ▇▅▂▁▁ |
| AIR_TIME | 0 | 1 | 116.46 | 74.96 | 14 | 60.5 | 98 | 147.0 | 505 | ▇▅▁▁▁ |
| DISTANCE | 0 | 1 | 833.42 | 628.37 | 41 | 375.0 | 679 | 1069.0 | 4243 | ▇▃▁▁▁ |
| WHEELS_ON | 0 | 1 | 1582.10 | 603.53 | 1 | 1256.5 | 1712 | 2047.0 | 2358 | ▂▁▅▆▇ |
| TAXI_IN | 0 | 1 | 8.56 | 7.88 | 1 | 4.0 | 6 | 10.0 | 69 | ▇▁▁▁▁ |
| SCHEDULED_ARRIVAL | 0 | 1 | 1628.46 | 506.50 | 5 | 1304.0 | 1710 | 2036.5 | 2359 | ▁▂▅▆▇ |
| ARRIVAL_TIME | 0 | 1 | 1577.06 | 616.31 | 2 | 1251.0 | 1718 | 2050.5 | 2359 | ▂▁▅▆▇ |
| ARRIVAL_DELAY | 0 | 1 | 57.25 | 56.82 | 15 | 23.0 | 38 | 68.0 | 673 | ▇▁▁▁▁ |
| AIRLINE_DELAY | 0 | 1 | 17.18 | 36.19 | 0 | 0.0 | 2 | 19.0 | 308 | ▇▁▁▁▁ |
Faltortunnuste moodustamine
Asendame edasiseks analüüsiks sümboltüüpi tunnused faktoritega:
flight <- as.data.frame(lapply(flight, function(x) if(is.character(x)) as.factor(x) else x))Teeme uuendatud andmestiku ülevaade
skim(flight)| Name | flight |
| Number of rows | 715 |
| Number of columns | 17 |
| _______________________ | |
| Column type frequency: | |
| factor | 2 |
| numeric | 15 |
| ________________________ | |
| Group variables | None |
Variable type: factor
| skim_variable | n_missing | complete_rate | ordered | n_unique | top_counts |
|---|---|---|---|---|---|
| AIRLINE | 0 | 1 | FALSE | 14 | WN: 158, AA: 83, DL: 83, OO: 76 |
| DESTINATION_AIRPORT | 0 | 1 | FALSE | 158 | ORD: 36, LAX: 35, ATL: 27, DEN: 26 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| MONTH | 0 | 1 | 6.17 | 3.41 | 1 | 3.0 | 6 | 9.0 | 12 | ▇▅▆▃▆ |
| DAY | 0 | 1 | 15.64 | 8.89 | 1 | 8.0 | 16 | 23.0 | 31 | ▇▇▆▆▆ |
| DAY_OF_WEEK | 0 | 1 | 3.94 | 2.02 | 1 | 2.0 | 4 | 6.0 | 7 | ▇▃▅▅▇ |
| FLIGHT_NUMBER | 0 | 1 | 2147.72 | 1741.50 | 5 | 754.0 | 1585 | 3173.0 | 6539 | ▇▅▂▂▂ |
| SCHEDULED_DEPARTURE | 0 | 1 | 1479.59 | 450.84 | 500 | 1120.5 | 1521 | 1845.0 | 2359 | ▃▆▆▇▅ |
| DEPARTURE_TIME | 0 | 1 | 1533.76 | 486.37 | 7 | 1155.5 | 1607 | 1930.5 | 2359 | ▁▃▇▇▇ |
| ELAPSED_TIME | 0 | 1 | 146.13 | 77.30 | 33 | 89.5 | 129 | 181.0 | 530 | ▇▅▂▁▁ |
| AIR_TIME | 0 | 1 | 116.46 | 74.96 | 14 | 60.5 | 98 | 147.0 | 505 | ▇▅▁▁▁ |
| DISTANCE | 0 | 1 | 833.42 | 628.37 | 41 | 375.0 | 679 | 1069.0 | 4243 | ▇▃▁▁▁ |
| WHEELS_ON | 0 | 1 | 1582.10 | 603.53 | 1 | 1256.5 | 1712 | 2047.0 | 2358 | ▂▁▅▆▇ |
| TAXI_IN | 0 | 1 | 8.56 | 7.88 | 1 | 4.0 | 6 | 10.0 | 69 | ▇▁▁▁▁ |
| SCHEDULED_ARRIVAL | 0 | 1 | 1628.46 | 506.50 | 5 | 1304.0 | 1710 | 2036.5 | 2359 | ▁▂▅▆▇ |
| ARRIVAL_TIME | 0 | 1 | 1577.06 | 616.31 | 2 | 1251.0 | 1718 | 2050.5 | 2359 | ▂▁▅▆▇ |
| ARRIVAL_DELAY | 0 | 1 | 57.25 | 56.82 | 15 | 23.0 | 38 | 68.0 | 673 | ▇▁▁▁▁ |
| AIRLINE_DELAY | 0 | 1 | 17.18 | 36.19 | 0 | 0.0 | 2 | 19.0 | 308 | ▇▁▁▁▁ |
Andmestikus on 2 faktor tunnust ja 15 arvulist tunnust
Suure erinevate kategooriate arvuga tunnuste eemaldamine
Andmestikus on olemas kategoriaalsed tunnused suure erinevate kategooriate arvuga: DESTINATION_AIRPORT(158 kategooriat).
Jätame suure erinevate kategooriate arvuga tunnuseid analüüsist välja. Me kasutame katagoriaalseid tunnuseid arvuliste tunnuste rühmitamiseks, kuid juhuk kui kategooriate arv on liiga suur saame tulemusena liiga väikesed rühmad, mille järgi ei ole võimalik üldistusi teha.
flight<- subset(flight, select = -c(DESTINATION_AIRPORT))Ülejäänud faktor-tunnuste puhul vaatame erinevate kategooriate arvu esenemist
as.data.frame(sapply(flight[, sapply(flight, is.factor)], table))## V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 V21
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## NK 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0
## UA 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 1 0 1 0 1 0 0 0 0 1 1 0 0 0 1 0 0
## V22 V23 V24 V25 V26 V27 V28 V29 V30 V31 V32 V33 V34 V35 V36 V37 V38 V39 V40
## AA 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## DL 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 1 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0
## US 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0
## V41 V42 V43 V44 V45 V46 V47 V48 V49 V50 V51 V52 V53 V54 V55 V56 V57 V58 V59
## AA 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0
## B6 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## F9 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0
## V60 V61 V62 V63 V64 V65 V66 V67 V68 V69 V70 V71 V72 V73 V74 V75 V76 V77 V78
## AA 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0
## DL 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 0 1 1 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## V79 V80 V81 V82 V83 V84 V85 V86 V87 V88 V89 V90 V91 V92 V93 V94 V95 V96 V97
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0
## AS 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## DL 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## OO 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0
## V98 V99 V100 V101 V102 V103 V104 V105 V106 V107 V108 V109 V110 V111 V112
## AA 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## EV 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1
## OO 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## US 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0
## V113 V114 V115 V116 V117 V118 V119 V120 V121 V122 V123 V124 V125 V126 V127
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1
## DL 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0
## EV 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0
## UA 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0
## US 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0
## V128 V129 V130 V131 V132 V133 V134 V135 V136 V137 V138 V139 V140 V141 V142
## AA 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## DL 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0
## EV 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
## F9 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## UA 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
## V143 V144 V145 V146 V147 V148 V149 V150 V151 V152 V153 V154 V155 V156 V157
## AA 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## DL 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0
## EV 1 0 0 1 0 0 0 0 0 0 1 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1
## V158 V159 V160 V161 V162 V163 V164 V165 V166 V167 V168 V169 V170 V171 V172
## AA 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## DL 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## EV 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## NK 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0
## V173 V174 V175 V176 V177 V178 V179 V180 V181 V182 V183 V184 V185 V186 V187
## AA 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## OO 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 1 0 1 0 1 0 1 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 0 0 1 0 1 1 0 1
## V188 V189 V190 V191 V192 V193 V194 V195 V196 V197 V198 V199 V200 V201 V202
## AA 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0
## UA 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
## US 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## WN 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0
## V203 V204 V205 V206 V207 V208 V209 V210 V211 V212 V213 V214 V215 V216 V217
## AA 1 0 0 1 0 0 0 0 0 0 1 0 0 1 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0
## US 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## V218 V219 V220 V221 V222 V223 V224 V225 V226 V227 V228 V229 V230 V231 V232
## AA 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## B6 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## US 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 1 0 1 0 0 1 0 0 0 0 0 0
## V233 V234 V235 V236 V237 V238 V239 V240 V241 V242 V243 V244 V245 V246 V247
## AA 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## OO 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 1 1 0 0 0 0 0 0 0 0 1 0 1
## V248 V249 V250 V251 V252 V253 V254 V255 V256 V257 V258 V259 V260 V261 V262
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## EV 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## OO 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0
## UA 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## WN 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1
## V263 V264 V265 V266 V267 V268 V269 V270 V271 V272 V273 V274 V275 V276 V277
## AA 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## EV 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## F9 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## UA 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0
## V278 V279 V280 V281 V282 V283 V284 V285 V286 V287 V288 V289 V290 V291 V292
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0
## EV 0 0 1 0 0 0 0 0 0 0 0 0 1 1 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## UA 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## V293 V294 V295 V296 V297 V298 V299 V300 V301 V302 V303 V304 V305 V306 V307
## AA 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 1 0 1 0 0 1 1 1 1 0
## V308 V309 V310 V311 V312 V313 V314 V315 V316 V317 V318 V319 V320 V321 V322
## AA 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## DL 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0
## EV 0 1 0 0 0 0 0 0 1 0 0 1 0 0 1
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## OO 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0
## V323 V324 V325 V326 V327 V328 V329 V330 V331 V332 V333 V334 V335 V336 V337
## AA 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## DL 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## OO 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0
## US 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0
## V338 V339 V340 V341 V342 V343 V344 V345 V346 V347 V348 V349 V350 V351 V352
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## NK 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1
## UA 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 0 1 0 0 0 1 0 1 0 1 0
## V353 V354 V355 V356 V357 V358 V359 V360 V361 V362 V363 V364 V365 V366 V367
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 1 0 0 0 1 0 0 1 0 1 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## NK 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 1 0 0 1 0 0 0 1 1 0 0
## V368 V369 V370 V371 V372 V373 V374 V375 V376 V377 V378 V379 V380 V381 V382
## AA 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0
## EV 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0
## F9 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1
## NK 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 0 1 0 0 0 1 0 0 0 0 0
## V383 V384 V385 V386 V387 V388 V389 V390 V391 V392 V393 V394 V395 V396 V397
## AA 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## DL 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## US 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 1 0 1 0 0 0 1 0 1 0 1
## V398 V399 V400 V401 V402 V403 V404 V405 V406 V407 V408 V409 V410 V411 V412
## AA 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## UA 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 1 1 0 0 0 1 1 0 0 0 0 0 1
## V413 V414 V415 V416 V417 V418 V419 V420 V421 V422 V423 V424 V425 V426 V427
## AA 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 1 0 1 0 0 0 1 1 0
## EV 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1
## F9 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
## NK 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0
## V428 V429 V430 V431 V432 V433 V434 V435 V436 V437 V438 V439 V440 V441 V442
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## EV 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## NK 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 1 0 1 1 0 0 0 1 0
## UA 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1
## WN 1 0 0 1 1 0 0 0 0 0 0 1 0 0 0
## V443 V444 V445 V446 V447 V448 V449 V450 V451 V452 V453 V454 V455 V456 V457
## AA 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## B6 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## DL 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 1 0 0 0 0 1 0 0 0 0 0 1
## V458 V459 V460 V461 V462 V463 V464 V465 V466 V467 V468 V469 V470 V471 V472
## AA 0 1 0 0 0 0 0 0 0 1 0 1 0 1 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 1 0 1 1 0 1 0 1 0 0 0 0 0 0 0
## EV 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0
## V473 V474 V475 V476 V477 V478 V479 V480 V481 V482 V483 V484 V485 V486 V487
## AA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0
## DL 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0
## EV 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## OO 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## UA 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 1 1 1 0 0 0 0 0 1 0 0 0
## V488 V489 V490 V491 V492 V493 V494 V495 V496 V497 V498 V499 V500 V501 V502
## AA 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## B6 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 1 0 0 0 0 0 0 0 0 1 0 1 1 1
## V503 V504 V505 V506 V507 V508 V509 V510 V511 V512 V513 V514 V515 V516 V517
## AA 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0
## AS 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## DL 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 1 1 0 0 1 0 0 1 0 0
## V518 V519 V520 V521 V522 V523 V524 V525 V526 V527 V528 V529 V530 V531 V532
## AA 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
## AS 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1
## UA 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 1 0 0 0 1 0 1 0
## V533 V534 V535 V536 V537 V538 V539 V540 V541 V542 V543 V544 V545 V546 V547
## AA 0 0 0 0 0 1 0 1 1 0 0 0 1 0 0
## AS 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
## DL 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0
## EV 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
## WN 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## V548 V549 V550 V551 V552 V553 V554 V555 V556 V557 V558 V559 V560 V561 V562
## AA 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## EV 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 1 0 0 1 0 0 0 0 0 1 0 0 1 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## WN 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0
## V563 V564 V565 V566 V567 V568 V569 V570 V571 V572 V573 V574 V575 V576 V577
## AA 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1
## DL 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
## EV 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0
## UA 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## V578 V579 V580 V581 V582 V583 V584 V585 V586 V587 V588 V589 V590 V591 V592
## AA 0 1 1 0 0 0 0 0 0 0 1 0 0 1 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## EV 0 0 0 0 0 1 0 0 0 0 0 0 1 0 1
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0
## UA 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 0 0 0 1 1 1 0 0 0 0 0 0
## V593 V594 V595 V596 V597 V598 V599 V600 V601 V602 V603 V604 V605 V606 V607
## AA 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## DL 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 1 0 0 0 1 0 0 0 0 0 1 0 0 1 0
## UA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1
## V608 V609 V610 V611 V612 V613 V614 V615 V616 V617 V618 V619 V620 V621 V622
## AA 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 1 0 0 1 0 0 0 0 1 0 0 1
## UA 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 0 0 0 0 0 1 0 0 1 1 0
## V623 V624 V625 V626 V627 V628 V629 V630 V631 V632 V633 V634 V635 V636 V637
## AA 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0
## DL 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## OO 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0
## UA 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
## WN 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0
## V638 V639 V640 V641 V642 V643 V644 V645 V646 V647 V648 V649 V650 V651 V652
## AA 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0
## AS 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## OO 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0
## UA 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1
## V653 V654 V655 V656 V657 V658 V659 V660 V661 V662 V663 V664 V665 V666 V667
## AA 0 0 0 1 0 1 1 0 0 0 0 0 0 1 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## OO 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1
## UA 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## V668 V669 V670 V671 V672 V673 V674 V675 V676 V677 V678 V679 V680 V681 V682
## AA 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 0 0 1 0 0 1 0 1 0 0 0 0 0 0
## EV 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 1 0 1 0 0 0 0 0 0 0 0 0 1
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 1 0 0 0 0 0 1 0 0 0 1 0 0 0
## V683 V684 V685 V686 V687 V688 V689 V690 V691 V692 V693 V694 V695 V696 V697
## AA 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0
## AS 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0
## DL 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1
## EV 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0
## UA 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0
## V698 V699 V700 V701 V702 V703 V704 V705 V706 V707 V708 V709 V710 V711 V712
## AA 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0
## AS 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## B6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## DL 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
## EV 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
## F9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## HA 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## MQ 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0
## NK 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## OO 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
## UA 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0
## US 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## VX 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## WN 1 0 0 1 0 0 1 1 0 0 0 0 0 0 0
## V713 V714 V715
## AA 1 0 0
## AS 0 0 1
## B6 0 0 0
## DL 0 0 0
## EV 0 0 0
## F9 0 0 0
## HA 0 0 0
## MQ 0 0 0
## NK 0 0 0
## OO 0 0 0
## UA 0 1 0
## US 0 0 0
## VX 0 0 0
## WN 0 0 0Andmete kirjeldav analüüs
Kirjeldava andmeanalüüsi etapil esitatakse tunnuste väärtuste jaotuste diagrammid, sagedustabelid, аrvulised karakteristikud ning tehakse ka erindite ehk anomaalväärtuste analüüs.
Tunnuste arvkarakteristikud
Andmestiku arvuliste tunnuste kõik valjalikud arvkarakteristikud on võimalik saada paketi FSA (install.packages(“FSA”)) funktsiooni Summarize() abil.
library(FSA)## ## FSA v0.10.0. See citation('FSA') if used in publication.
## ## Run fishR() for related website and fishR('IFAR') for related book.sapply(Filter(is.numeric, flight[,-1]),Summarize)## $DAY
## n mean sd min Q1 median Q3
## 715.000000 15.644755 8.887073 1.000000 8.000000 16.000000 23.000000
## max
## 31.000000
##
## $DAY_OF_WEEK
## n mean sd min Q1 median Q3
## 715.000000 3.938461 2.021347 1.000000 2.000000 4.000000 6.000000
## max
## 7.000000
##
## $FLIGHT_NUMBER
## n mean sd min Q1 median Q3 max
## 715.000 2147.724 1741.501 5.000 754.000 1585.000 3173.000 6539.000
##
## $SCHEDULED_DEPARTURE
## n mean sd min Q1 median Q3 max
## 715.0000 1479.5874 450.8361 500.0000 1120.5000 1521.0000 1845.0000 2359.0000
##
## $DEPARTURE_TIME
## n mean sd min Q1 median Q3 max
## 715.0000 1533.7580 486.3689 7.0000 1155.5000 1607.0000 1930.5000 2359.0000
##
## $ELAPSED_TIME
## n mean sd min Q1 median Q3 max
## 715.00000 146.13147 77.29718 33.00000 89.50000 129.00000 181.00000 530.00000
##
## $AIR_TIME
## n mean sd min Q1 median Q3 max
## 715.00000 116.46294 74.95575 14.00000 60.50000 98.00000 147.00000 505.00000
##
## $DISTANCE
## n mean sd min Q1 median Q3 max
## 715.0000 833.4224 628.3668 41.0000 375.0000 679.0000 1069.0000 4243.0000
##
## $WHEELS_ON
## n mean sd min Q1 median Q3 max
## 715.0000 1582.1021 603.5329 1.0000 1256.5000 1712.0000 2047.0000 2358.0000
##
## $TAXI_IN
## n mean sd min Q1 median Q3
## 715.000000 8.556643 7.884003 1.000000 4.000000 6.000000 10.000000
## max
## 69.000000
##
## $SCHEDULED_ARRIVAL
## n mean sd min Q1 median Q3 max
## 715.0000 1628.4615 506.5015 5.0000 1304.0000 1710.0000 2036.5000 2359.0000
##
## $ARRIVAL_TIME
## n mean sd min Q1 median Q3 max
## 715.0000 1577.0643 616.3104 2.0000 1251.0000 1718.0000 2050.5000 2359.0000
##
## $ARRIVAL_DELAY
## n mean sd min Q1 median Q3 max
## 715.00000 57.24615 56.82299 15.00000 23.00000 38.00000 68.00000 673.00000
##
## $AIRLINE_DELAY
## n mean sd min Q1 median Q3 max
## 715.00000 17.18042 36.19492 0.00000 0.00000 2.00000 19.00000 308.00000
## percZero
## 47.69231Andmestiku mittearvuliste tunnuste arvkarakteristikute asemel esitame nende väärtuste sagedustabelid.
Kuna meil on üks faktor tunnus kontrollime kas see oletakse vigadeta faktoriks.
is.factor(flight$AIRLINE)## [1] TRUEkuna tunnust ei loetud õigesti teisaldame selle faktoriks.ning moodustame sagedustabeli.
flight$AIRLINE <- as.factor(flight$AIRLINE)
table(flight$AIRLINE)##
## AA AS B6 DL EV F9 HA MQ NK OO UA US VX WN
## 83 19 38 83 63 15 6 39 30 76 69 26 10 158Tulemusena saime teada mitu korda iga kategooria esineb.
Tunnuste jaotused
Arvuliste tunnuste jaotused
Esitame arvuliste tunnuste jaotuste histogrammid ja karpdiagrammid. Uurime erindite olemsolu.
par(mfrow = c(2, 2))
tunnused <- Filter(is.numeric, flight[,-1])
for (i in 1:5)
{
hist(tunnused[,i], col="skyblue",ylab="Sagedus",xlab=names(tunnused)[i], main="Sageduste histogramm",cex.main=0.9,cex.axis=0.7,cex.lab=0.8)
par(mar=c(6,6,5,2)+0.1)
boxplot(tunnused[,i], col="skyblue", horizontal=1, xlab=names(tunnused)[i], main="Karp-vurrud diagramm",cex.main=0.9, cex.axis=0.7,cex.lab=0.8)
points(mean(tunnused[,i]), 1, col = "red", pch = 18)
text(mean(tunnused[,i]), 0.95, "mean", col="red", cex=0.5) }
MONTH karpdiagramm on normaalselt jaotunud kuid kerge asümmeetriaga. Sageduse histogramm on asümeetriline, kuna 30 ümbruses on sagedused madalamad ning kõrgemad väärtused 800 ümber on rohkem paremal. DAY_OF_WEEK histogramm on natukene asümmeetriline paremale kuna osa vastustest on madalad. Sageduse histogramm FLIGHT_NUMBER on asümmeetriaga. Madalad sagedused domineerivad, kuid mõned lennud on väga kõrge sagedusega (6000). SCHEDULED_DEPARTURE on peagu sümmeetriline, SCHEDULED_DEPARTURE karpdiagrammon samuti peagu sümmeetriline.
Kategoriaalsete tunnuste jaotused
Esitame kategoriaalsete tunnuste väärtuste sageduste histogrammid jättes suure erinevate kategooriate arvuga tunnuseid välja.
Esmalt kontrollime, et faktortunnust on R poolt ära tuntud ja loetud.
Ktunnused <- flight[, sapply(flight, is.factor), drop=FALSE]
if (ncol(Ktunnused) == 0) {
stop("Andmestikus pole faktor-tunnuseid.")
}Nüüd joonistame histogrammi
par(mar = c(5, 4, 4, 2))
for (i in 1:ncol(Ktunnused)) {
barplot(table(Ktunnused[, i]), col = "skyblue", ylab = "Sagedus",
main = paste("Tunnuse", names(Ktunnused)[i], "\n sageduste histogramm"),
cex.main = 0.9, cex.axis = 0.7, cex.lab = 0.8, las = 2, cex.names = 0.5)
}
Näeme, et palju on andmeid WIN Airline kohta.
Seoste analüüs
Seosed arvuliste tunnuste vahel
Visualiseerime arvuliste tunnuste vahelised sõltuvused ka plot funktsiooni abil:
png("plot.png", width = 1200, height = 800)
par(mar = c(10, 4, 4, 2))
plot(tunnused, pch = 19, col = "blue", cex = 0.75)
knitr::include_graphics("plot.png")hajuvusmaatriksi põhjal näeme võimalike suhteid väärtuste vahel. Hajuvusmaatriksi põhjal valime sihttunnuseks ARRIVAL_DELAY. See on arvuline ja pidev väärtus.
Näeme hajuvusgraafiku põhjal, et ARRIVAL_DELAY-l võib olla seos väärtustega: DEPARTURE_TIME, AIRLINE_DELAY ja ELAPSED_TIME. Arvuliseks tunnuseks edasi analüüsimiseks valisime DEPARTURE_TIME.
Kuna alles on pärast andme puhastust jäänud mittearvuline tunnus AIRLINE (kategooriline) siis teeme ka analüüsi sellest.
Kategoriaalsete tunnuste mõju sihttunnustele
Sihttunnuse Y=“DEP_DELAY” kategoriaalsete tunnustega seose analüüsimiseks esitame sihttunnuse karpdiagrammid kateoriaalse tunnuse rühmadeks. Enne sihttunnuse töötlemist kontrollime veergude arvu:
if (ncol(Ktunnused) == 0) {
stop("Andmestikus pole faktor-tunnuseid.")
}Andmestikus on faktortunnuseid.
par(mfrow = c(1, 1))
par(mar=c(4,4,4,2))
for (i in 1:ncol(Ktunnused)) {
b <- boxplot(flight$ARRIVAL_DELAY ~ Ktunnused[, i], col = "skyblue",
cex.axis = 0.55, cex.lab = 0.7, las = 2,
main = paste("ARRIVAL_DELAY vs", names(Ktunnused)[i], "\n karpdiagramm"),
ylab = "ARRIVAL_DELAY", xlab = "", cex.main = 0.8)
kesk <- tapply(flight$ARRIVAL_DELAY, Ktunnused[, i], mean)
points(1:length(unique(Ktunnused[, i])), kesk, col = "red", pch = 18, cex = 0.5)
}
Erinevus rühmade keskmiste vahel näitab, et faktor-tunnuse mõju on oluline.
Karpdiagrammi pähjal näeme, et lennufirmadel nagu AS ja VX (võib ka lugeda nende hulka mingil määral US) on madalama saabumise hilinemisega mediaan, see tähendab, et pooletel nende lennufirmade lendudel on saabumise hilinemine väiksem kui mediaani väärtusel.
Lennufirmadel nagu HA ja MQ on kõrgem saabumise hilinemine kui mediaan, mis võib viidata sellele, et nende lendudel esineb keskmisest rohkem hilinemisi.
Lennufirmadel nagu EV, OO ja WN on laiemad karbid, mis näitab, et nende saabumise hilinemised on hajutatumad ehk nende lendude seas esineb väikseid hilinemisi kui ka suuri hilinemisi.
Lennufirmadel nagu AA, EV, F9 ja MQ on rohkem erandeid.
HA lennufirma puhul on näha võrreldes teiste lennufirmadega kõrgemat karpi, mis võib viitada tihedamatele hilinemistele võrreldes teiste lennufirmadega.
Esitame faktor tunnuse AIRLINE arvkarakteristiku faktortunnuse rühmades.
library(kableExtra)
tabel <- kable(Summarize(ARRIVAL_DELAY~AIRLINE, data=flight, digits=2))
kable_styling(tabel, bootstrap_options = c("striped", "hover"))| AIRLINE | n | mean | sd | min | Q1 | median | Q3 | max |
|---|---|---|---|---|---|---|---|---|
| AA | 83 | 53.59 | 54.05 | 15 | 23.50 | 33.0 | 61.50 | 370 |
| AS | 19 | 55.68 | 39.73 | 15 | 27.00 | 38.0 | 70.00 | 139 |
| B6 | 38 | 65.29 | 59.99 | 16 | 26.75 | 37.5 | 68.75 | 233 |
| DL | 83 | 46.88 | 44.32 | 15 | 19.50 | 35.0 | 52.00 | 308 |
| EV | 63 | 58.16 | 53.89 | 15 | 24.50 | 39.0 | 64.50 | 299 |
| F9 | 15 | 55.00 | 41.23 | 15 | 29.50 | 42.0 | 63.50 | 172 |
| HA | 6 | 65.67 | 53.91 | 15 | 22.25 | 49.0 | 115.50 | 130 |
| MQ | 39 | 74.54 | 105.07 | 16 | 26.00 | 58.0 | 82.50 | 673 |
| NK | 30 | 77.53 | 74.83 | 16 | 26.50 | 49.0 | 100.75 | 348 |
| OO | 76 | 54.07 | 46.14 | 15 | 25.25 | 39.5 | 73.25 | 314 |
| UA | 69 | 63.65 | 60.90 | 15 | 21.00 | 34.0 | 89.00 | 318 |
| US | 26 | 51.08 | 49.28 | 16 | 19.25 | 28.0 | 57.00 | 178 |
| VX | 10 | 49.80 | 47.22 | 16 | 21.50 | 23.0 | 54.75 | 152 |
| WN | 158 | 54.49 | 50.90 | 15 | 22.00 | 39.0 | 63.50 | 297 |
faktortunnuse arvkarakteristiku tabeli põhjal näeme, et lennufirmadel NK ja MQ on kõrgemad keskmised saabumis hilinemised võrreldes madalama keskmisega F9 ja US lennufirmade puhul. See tähendab, et lennufirma mängib olulist rolli saabumis-hilinemisel.
tabeli põhjal ka näeme, et NK ja MQ lennufirmadel on suurem standardhälve, mis tähendab hilinemised on ettearvamatud. Lennufirmadel AS ja VX on aga madalama standardhälvega, mis tähendab, et antud lennufirmade puhul pole hilinemised nii ettearvamatud.
MQ lennufirma puhul on maksimaalne hilinemine väga suur, mis tähendab, et selle lennufirma puhul võivad mõned lennud väga pikka vahega hilineda.
Valitud tunnuste analüüs
Valisime edasiseks analüüsiks sihttunnuse ARRIVAL_DELAY. Arvulise tunnuse analüüsimiseks valisime DEPARTURE_TIME ning mittearvulise tunnuseks valisime AIRLINE.
Kirjeldava andmeanalüüsi etapil esitatakse tunnuste väärtuste jaotuse diagrammid, sagedustabelid, аrvulised karakteristikud ning tehakse ka erindite analüüs.
Mittearvuliste tunnuste analüüs
Mittearvuline tunnus on AIRLINE on kategooriline tunnus.
Visualiseerime tunnuse kasutades tulpdiagrammi.
par(mar=c(5,4,4,2))
barplot(prop.table(table(flight$AIRLINE))*100, col = "skyblue", cex.names = 0.7, cex.axis = 0.8, main = "Tunnuse AIRLINE suhteliste sageduste tulpdiagramm", ylab = "suht. sagedus, %",ylim=c(0,60))
Esitame ja tunnuse sagedustabeli. Selleks kasutame funktsiooni kable() paketist knitr ja funktsiooni kable_styling() paketist kableExtra.
sagedus <- table(flight$AIRLINE)
suht.sagedus <-round(prop.table(table(flight$AIRLINE))*100,2)
sagedustabel <- cbind(rownames(sagedus),sagedus,suht.sagedus)
sagedustabel <- data.frame(sagedustabel,row.names = NULL)
sagedustabel <-rbind(sagedustabel,c("Total",sum(sagedus),sum(suht.sagedus)))
names(sagedustabel) <- c("Airline","sagedus","suht.sagedus,%")
library(knitr)
library(kableExtra)
kable_styling(kable(sagedustabel,align = c('l','r','r')),bootstrap_options = c("striped","hover"),full_width = 0,position = "left")| Airline | sagedus | suht.sagedus,% |
|---|---|---|
| AA | 83 | 11.61 |
| AS | 19 | 2.66 |
| B6 | 38 | 5.31 |
| DL | 83 | 11.61 |
| EV | 63 | 8.81 |
| F9 | 15 | 2.1 |
| HA | 6 | 0.84 |
| MQ | 39 | 5.45 |
| NK | 30 | 4.2 |
| OO | 76 | 10.63 |
| UA | 69 | 9.65 |
| US | 26 | 3.64 |
| VX | 10 | 1.4 |
| WN | 158 | 22.1 |
| Total | 715 | 100.01 |
Diagrammi põhjal näeme, et WN moodustab omb kaudu 1/4 andmestikkus olevatest lendudest. AA, DL, EV ja UA on sarnase osakaaluga, igaüks on umbes 10% vahel.
Lennufirma OO jääb umbes 5% vahemikku ning ülejäänud lennufirmad 2% vahemikku. Eriti väike esindus on lennufirmal HA.
Arvulise tunnuse alanüüs
Kuna valitud sihttunnus Y=ARRIVAL_DELAY on pidev tunnus, kasutame hist() funktsiooni andmete visualiseerimiseks.
Funktsioon hist() koondab andmed vahemikesse, neid vahemikke edasi kasutatakse sagedustabeli moodustamiseks;
h <- hist(flight$ARRIVAL_DELAY, col = "skyblue",main = "Tunnuse ARRIVAL_DELAY sageduste histogramm", xlab="saabumise hilinemine (minutites)", ylab = "sagedus", ylim = c(0,500))
Tunnuse ARRIVAL_DELAY suhtelise sageduse histogramm ja karpdiagramm.
par(mfrow=c(1,1))
h$counts <- h$counts/sum(h$counts)*100
plot(h,col = "skyblue",main = "Tunnuse ARRIVAL_DELAY \n suhteliste sageduste histogramm", xlab="saabumise hilinemine (minutites)", ylab = "suht.sagedus,%",cex.main=0.8,ylim = c(0,40),xlim = c(0,7000))
b <- boxplot(flight$ARRIVAL_DELAY, col = "skyblue", horizontal = 1, main="Tunnuse ARRIVAL_DELAY karpdiagramm",xlab="lendude saabumise hilinemine eurodes",cex.main=0.8, range = 3)
Tunnuse ARRIVAL_DELAY jaotus on parempoolse asümmeetriaga ning on olemas ka palju erandeid paremal.
Erindite analüüs: Paljud lennud saabuvad pigem õigeaegselt või väiksema hilinemisega, seda võib järeldada võtes arvesse, et lendude hilinemised on pigem koondunud 0 väärtuse lähedusse.
histogrammi paremas osas on ka näha potendsiaalseid erandeid, mis näitavad mitme tuhandete minutite hilinemisi. Need hilinemised erinevad ülejäänud suurem osa andmetest.
sort(b$out)## [1] 218 219 226 232 233 233 235 238 247 261 281 297 299 308 314 318 348 370 673Objektid, mis esinevad eranditena:
which(flight$ARRIVAL_DELAY%in%b$out)## [1] 103 150 177 238 321 339 341 371 379 384 431 474 482 515 588 590 630 668 690Info erandite kohta:
datatable(flight[which(flight$ARRIVAL_DELAY%in%b$out) , ],options=list(scrollX=1,pageLenght=5,searching = FALSE,scroller = TRUE,scrollY=200))Tabeli põhjal saame teada, et pooled suured hilinemised toimusid juuni kuu vältel. Tabelis olevad viimased kolm lendu on ka oma suure hilinemise tõttu erilised.
Tunnuse sagedustabel (eelnevalt väärtused on vaja intervallidesse jaotada vastavalt tunnuse histogrammile, kasutame selleks funktsiooni cut()):
intervallid <-cut(flight$ARRIVAL_DELAY,h$breaks,include.lowest = TRUE,dig.lab = 4)
sagedus <- table(intervallid)
suht.sagedus <-prop.table(table(intervallid))*100
sagedustabel <- cbind(rownames(sagedus),sagedus,round(suht.sagedus,2),round(cumsum(suht.sagedus),2))
sagedustabel <- data.frame(sagedustabel,row.names = NULL)
sagedustabel <-rbind(sagedustabel,c("Total",sum(sagedus),sum(suht.sagedus),""))
names(sagedustabel) <- c("Price_euros","sagedus","suht.sagedus,%","kum.suht.sagedus,%")
library(knitr)
library(kableExtra)
kable_styling(kable(sagedustabel,align = c('l','r','r','r')),bootstrap_options = c("striped","hover"),full_width = 0,position = "left")| Price_euros | sagedus | suht.sagedus,% | kum.suht.sagedus,% |
|---|---|---|---|
| [0,50] | 445 | 62.24 | 62.24 |
| (50,100] | 168 | 23.5 | 85.73 |
| (100,150] | 59 | 8.25 | 93.99 |
| (150,200] | 22 | 3.08 | 97.06 |
| (200,250] | 11 | 1.54 | 98.6 |
| (250,300] | 4 | 0.56 | 99.16 |
| (300,350] | 4 | 0.56 | 99.72 |
| (350,400] | 1 | 0.14 | 99.86 |
| (400,450] | 0 | 0 | 99.86 |
| (450,500] | 0 | 0 | 99.86 |
| (500,550] | 0 | 0 | 99.86 |
| (550,600] | 0 | 0 | 99.86 |
| (600,650] | 0 | 0 | 99.86 |
| (650,700] | 1 | 0.14 | 100 |
| Total | 715 | 100 |
Tunnuse arvkarakteristikud:
library(FSA)
arvkar <- Summarize(flight$ARRIVAL_DELAY, digits = 2)
kable_styling(kable(t(arvkar)))| n | mean | sd | min | Q1 | median | Q3 | max |
|---|---|---|---|---|---|---|---|
| 715 | 57.25 | 56.82 | 15 | 23 | 38 | 68 | 673 |
Järeldus: Keskmine hilinemine on 57.25 minutit ehk keskmiselt hilinevad lennud saabumisel peagu tund. Mediaan on madalam võrreldes keskmisega, mis viitab positiivsele viltusele andmete jaotusel. See tähendab ka, et enamik lende hilineb vähem ning mõned suurte hilinemistega lennud tõstavad keskmist. Standard hälve on 56.82 minutit, mis näitab suurt varieeruvust saabumise hilinemisel. Minimaalne hilinemine on 15 minutit ja maksimaalne hilinemine on 673 minutit ehk umbes 11 tundi.
Sihttunnuse jaotuse vastavus normaaljaotusele
Sihttunnuse jaotuse uurimiseks visuaalselt esitame andmed histogrammi ja thedusgraafikuna.
hist(flight$ARRIVAL_DELAY, probability = TRUE, col = "skyblue",
main = "Tunnuse ARRIVAL_DELAY jaotus", xlab = "ARRIVAL_DELAY")
lines(density(flight$ARRIVAL_DELAY), col = "red", lwd = 2)
normaaljaotuse graafik näitab, et graafik on paremale poole viltu seega ARRIVAL_DELAY ei ole normaalselt jaotunud.
Usaldusintervallide leidmine
Leidke 95% usaldusintervall sihttunnuse keskväärtusele
Selleks, et leida 95% usaldusintervall sihttunnuse keskväärtusele kasutame funktsiooni t.test.
t.test(flight$ARRIVAL_DELAY, conf.level = 0.95)$conf.int## [1] 53.07404 61.41827
## attr(,"conf.level")
## [1] 0.95Antud tulemus annab meile teada, et me võime 95% kkindlusega väita, et meie ARRIVAL_DELAY jääb vahemikku 53.07 minutit ja 61.42 minutit.
Leidke 95% usaldusintervallid sihttunnuse keskväärtusele kategoriaalse tunnuse rühmades
Kõige rohkem mõjuv kategooriline tunnus on AIRLINE. Leiame selle usaldusintervalli iga rühma jaoks
library(dplyr)##
## Attaching package: 'dplyr'## The following object is masked from 'package:kableExtra':
##
## group_rows## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionusaldusintervallid <- flight %>%
group_by(AIRLINE) %>%
summarise(
keskvaartus = mean(ARRIVAL_DELAY),
usaldusint = list(t.test(ARRIVAL_DELAY, conf.level = 0.95)$conf.int)
)
usaldusintervallid## # A tibble: 14 × 3
## AIRLINE keskvaartus usaldusint
## <fct> <dbl> <list>
## 1 AA 53.6 <dbl [2]>
## 2 AS 55.7 <dbl [2]>
## 3 B6 65.3 <dbl [2]>
## 4 DL 46.9 <dbl [2]>
## 5 EV 58.2 <dbl [2]>
## 6 F9 55 <dbl [2]>
## 7 HA 65.7 <dbl [2]>
## 8 MQ 74.5 <dbl [2]>
## 9 NK 77.5 <dbl [2]>
## 10 OO 54.1 <dbl [2]>
## 11 UA 63.7 <dbl [2]>
## 12 US 51.1 <dbl [2]>
## 13 VX 49.8 <dbl [2]>
## 14 WN 54.5 <dbl [2]>Visualiseerige usaldusintervallid Cleveland plot diagrammil
Kõigepeal valmistame ette andmed, et keskväärtused ja usaldusintervallid oleksid paremini kättesaadavad.
usaldusintervallid <- flight %>%
group_by(AIRLINE) %>%
summarise(
keskvaartus = mean(ARRIVAL_DELAY),
alumine = t.test(ARRIVAL_DELAY, conf.level = 0.95)$conf.int[1],
ulemine = t.test(ARRIVAL_DELAY, conf.level = 0.95)$conf.int[2]
)Pärast andmete ettevalmistamist saame joonistada Cleveland plot joonise.
library(ggplot2)
ggplot(usaldusintervallid, aes(x = keskvaartus, y = AIRLINE)) +
geom_point() +
geom_errorbarh(aes(xmin = alumine, xmax = ulemine), height = 0.2) +
labs(title = "Keskväärtuse usaldusintervallid AIRLINE rühmades",
x = "Keskväärtus", y = "AIRLINE")
Valige kategoriaalse tunnuse kaks rühma, milles erinevus on kõige suurem
Kõigepealt valime kaks rühma, mis on suurima erinevuseda. Valime keskväärtuse järgi.
usaldusintervallid <- usaldusintervallid %>%
arrange(desc(keskvaartus))
top2 <- usaldusintervallid[1:2, ]
top2## # A tibble: 2 × 4
## AIRLINE keskvaartus alumine ulemine
## <fct> <dbl> <dbl> <dbl>
## 1 NK 77.5 49.6 105.
## 2 MQ 74.5 40.5 109.Nüüd saame nende rühmade jaoks joonistada neile karpdiagrammid.
top2_data <- flight %>% filter(AIRLINE %in% top2$AIRLINE)
boxplot(ARRIVAL_DELAY ~ AIRLINE, data = top2_data, col = "skyblue",
main = "Karpdiagramm kahe rühma jaoks", ylab = "ARRIVAL_DELAY")
Nüüd arvutame rühmade põhikarakteristikud kaustades summarise funktsiooni.
top2_data %>%
group_by(AIRLINE) %>%
summarise(
keskvaartus = mean(ARRIVAL_DELAY),
mediaan = median(ARRIVAL_DELAY),
standardhalve = sd(ARRIVAL_DELAY),
minimaalne = min(ARRIVAL_DELAY),
maksimaalne = max(ARRIVAL_DELAY)
)## # A tibble: 2 × 6
## AIRLINE keskvaartus mediaan standardhalve minimaalne maksimaalne
## <fct> <dbl> <dbl> <dbl> <int> <int>
## 1 MQ 74.5 58 105. 16 673
## 2 NK 77.5 49 74.8 16 348Sõnastame hüpoteesi hüpotees: Rühmade keskväärtused on võrdsed. Nüüd koostame t-testi.
t.test(ARRIVAL_DELAY ~ AIRLINE, data = top2_data)##
## Welch Two Sample t-test
##
## data: ARRIVAL_DELAY by AIRLINE
## t = -0.13819, df = 66.659, p-value = 0.8905
## alternative hypothesis: true difference in means between group MQ and group NK is not equal to 0
## 95 percent confidence interval:
## -46.25704 40.26730
## sample estimates:
## mean in group MQ mean in group NK
## 74.53846 77.53333Kuna testi käigus saadud p-väärtus on 0.8905, siis puuduvad piisavad tõendid, et väita, et rühmade keskväärtused on erinevad, mis tõttu seatud nullhüpotees jääb kehtima.
///////////////////////////////////////////////////////////////////////////
Dispersioonianalüüs ehk ANOVA
Visualiseerime karpdiagrammi abil
b = boxplot(flight$ARRIVAL_DELAY~flight$AIRLINE, col="skyblue", main="karpdiagramm", xlab="airlines", ylab="arrivaldelay")
points(b$n, tapply(flight$ARRIVAL_DELAY, flight$AIRLINE, mean), col="red", pch=18)
kas keskväärtused on võrdsed? kas airlinest sõltub hilinemisest või
vastupidi? kas keskväärtused erinevad teineteisest?
ANOVA testi rakendamine: summary(aov(ARRIVAL_DELAY$AIRLINES, flight=flight))
Post hoc analüüs: kontrollime, milliste rühmade vahel on oluline erinevus
see arvutab paari kaupa p-value väärtuse.
pairwise.t.test(flight$ARRIVAL_DELAY, flight$AIRLINE, pool.sd=FALSE, p.adj="bonferroni")##
## Pairwise comparisons using t tests with non-pooled SD
##
## data: flight$ARRIVAL_DELAY and flight$AIRLINE
##
## AA AS B6 DL EV F9 HA MQ NK OO UA US VX
## AS 1 - - - - - - - - - - - -
## B6 1 1 - - - - - - - - - - -
## DL 1 1 1 - - - - - - - - - -
## EV 1 1 1 1 - - - - - - - - -
## F9 1 1 1 1 1 - - - - - - - -
## HA 1 1 1 1 1 1 - - - - - - -
## MQ 1 1 1 1 1 1 1 - - - - - -
## NK 1 1 1 1 1 1 1 1 - - - - -
## OO 1 1 1 1 1 1 1 1 1 - - - -
## UA 1 1 1 1 1 1 1 1 1 1 - - -
## US 1 1 1 1 1 1 1 1 1 1 1 - -
## VX 1 1 1 1 1 1 1 1 1 1 1 1 -
## WN 1 1 1 1 1 1 1 1 1 1 1 1 1
##
## P value adjustment method: bonferroniJäreldus: kas erinevad või mitte? kas on seoseid?