Coffee Review Exploratory Data Analysis

We are exploring data on reviews of coffee quality from different suppliers. The data was downloaded from Kaggle - click on the link to visit the Dataset page.

The dataset contains 1338 rows and 53 columns. Here are the first 10 rows.

The first column is an index, columns 2-20 give information on the coffee being assessed and columns 21-36 contain the quality measurements; columns 21-31 give cupping measurements and columns 32-36 give measurements for the green unroasted beans. Columns 37-40 give information on the coffee certification and columns 41-44 are derived from the farm altitude data.

Let’s see summaries of each of the columns.

-- Data Summary ------------------------
                           Values   
Name                       bean_data
Number of rows             1339     
Number of columns          44       
_______________________             
Column type frequency:              
  character                24       
  numeric                  20       
________________________            
Group variables            None     

-- Variable type: character -----------------------------------------------------------------------------------------
# A tibble: 24 x 8
   skim_variable         n_missing complete_rate   min   max empty n_unique whitespace
 * <chr>                     <int>         <dbl> <int> <int> <int>    <int>      <int>
 1 Species                       0         1         7     7     0        2          0
 2 Owner                         7         0.995     3    50     0      315          0
 3 Country.of.Origin             1         0.999     4    28     0       36          0
 4 Farm.Name                   359         0.732     1   101     0      571          0
 5 Lot.Number                 1063         0.206     1    93     0      227          0
 6 Mill                        318         0.763     1   101     0      459          0
 7 ICO.Number                  157         0.883     1    40     0      846          0
 8 Company                     209         0.844     3    73     0      281          0
 9 Altitude                    226         0.831     1    41     0      396          0
10 Region                       59         0.956     3    76     0      356          0
11 Producer                    232         0.827     1   104     0      690          0
12 Bag.Weight                    0         1         1     8     0       56          0
13 In.Country.Partner            0         1         7    88     0       27          0
14 Harvest.Year                 47         0.965     3    24     0       46          0
15 Grading.Date                  0         1        13    20     0      567          0
16 Owner.1                       7         0.995     3    50     0      319          0
17 Variety                     226         0.831     4    21     0       29          0
18 Processing.Method           170         0.873     5    25     0        5          0
19 Color                       218         0.837     4    12     0        4          0
20 Expiration                    0         1        13    20     0      566          0
21 Certification.Body            0         1         7    88     0       26          0
22 Certification.Address         0         1        40    40     0       32          0
23 Certification.Contact         0         1        40    40     0       29          0
24 unit_of_measurement           0         1         1     2     0        2          0

-- Variable type: numeric -------------------------------------------------------------------------------------------
# A tibble: 20 x 10
   skim_variable        n_missing complete_rate      mean        sd    p0     p25     p50     p75      p100
 * <chr>                    <int>         <dbl>     <dbl>     <dbl> <dbl>   <dbl>   <dbl>   <dbl>     <dbl>
 1 X                            0         1      669       387.         0  334.    669    1004.     1338   
 2 Number.of.Bags               0         1      154.      130.         0   14     175     275      1062   
 3 Aroma                        0         1        7.57      0.378      0    7.42    7.58    7.75      8.75
 4 Flavor                       0         1        7.52      0.398      0    7.33    7.58    7.75      8.83
 5 Aftertaste                   0         1        7.40      0.404      0    7.25    7.42    7.58      8.67
 6 Acidity                      0         1        7.54      0.380      0    7.33    7.58    7.75      8.75
 7 Body                         0         1        7.52      0.370      0    7.33    7.5     7.67      8.58
 8 Balance                      0         1        7.52      0.409      0    7.33    7.5     7.75      8.75
 9 Uniformity                   0         1        9.83      0.555      0   10      10      10        10   
10 Clean.Cup                    0         1        9.84      0.764      0   10      10      10        10   
11 Sweetness                    0         1        9.86      0.616      0   10      10      10        10   
12 Cupper.Points                0         1        7.50      0.473      0    7.25    7.5     7.75     10   
13 Total.Cup.Points             0         1       82.1       3.50       0   81.1    82.5    83.7      90.6 
14 Moisture                     0         1        0.0884    0.0483     0    0.09    0.11    0.12      0.28
15 Category.One.Defects         0         1        0.479     2.55       0    0       0       0        63   
16 Quakers                      1         0.999    0.173     0.832      0    0       0       0        11   
17 Category.Two.Defects         0         1        3.56      5.31       0    0       2       4        55   
18 altitude_low_meters        230         0.828 1751.     8669.         1 1100    1311.   1600    190164   
19 altitude_high_meters       230         0.828 1799.     8669.         1 1100    1350    1650    190164   
20 altitude_mean_meters       230         0.828 1775.     8669.         1 1100    1311.   1600    190164   

We can see that there is plenty of missing data in the character variables, while most of the numeric variables are complete. Some altitude data is missing.

Data Cleaning

Some of the coffee description columns are not of interest - ICO.Number and Lot.Number are just identifiers. Owner is lower case transformed Owner.1, so will be preferentially used.

There are several fields for the origin of the coffee, i.e. Owner, Farm.Name, Mill, Company, Producer and also In.Country.Partner. By far the most complete of these data is the Owner and this may be the best field to use, however some Owners list several Producers, which suggests different origins for these beans. Initially, we will just use Owner and not use Farm.Name, Company, Producer or In.Country.Partner.

Note that we have some weird characters present in the Owner and Region field. We can remove these using string replace. One of the samples has a rating of zero for all scores and can be removed as likely erroneous.

Grading.Date will be transformed from a string to a date and sample Bag.Weight will be changed to a numeric, harmonising to kg units.

Missing Data

We will deal with missing data as follows:

• Those rows with missing Owner, we will replace the missing value with the Farm.Name
• The one row with missing Country.Of.Origin is clearly from Colombia as clear from the Owner entry
• Where Region data is missing, we will default to the Country
• There is a small amount of missing data for Harvest.Year, where the rows could be removed
• For Variety, we can’t make deductions about missing values, so it’s likely we will have to remove missing data
• Similarly for Processing.Method, the missing data can’t be assumed so will likely have to be removed
• Color has some values of “None” which is assumed to be missing data, rather than the beans being colourless
• Where altitude data is missing, we could impute the mean
• One missing value for Quakers can be assumed to be 0

Data Correction

The variable Harvest.Year has the year in various different formats and needs to be cleaned up. We will use the following rules for cleaning: we default to a single year value; where two years are given, we will use the second year; when no year is given we use NA. This variable can then be transformed to a numeric.

Some of the altitude data also requires correction as it appears to have been entered incorrectly or misread. We will re-parse the original Altitude data to correct the errors. After re-parsing to numeric values for mean and range of altitude, the number of missing values becomes 244.

Apart from in Hawaii and Puerto Rico, the United States does not produce coffee, so where Country.of.Origin has been given as United States, this appears to be a data error. We will fix this data using Region and Producer information.

Input Variables

Let’s look at distributions of the input variables. We will plot the categorical variables as bar charts, showing the most popular values.

We will plot the numeric variables as histograms.

What we’ve discovered from the input variables:

• Only a very small number of Robusta beans have been graded
• There are many different values for Owner of which most are of a low frequency
• Country of origin is quite widely distributed with 36 countries represented - Mexico has the largest amount of data
• We have 29 different varieties, with Caturra, Bourbon and Typica being most common
• Most of the samples have been processed using the Washed / Wet method and the remainder are mostly Natural / Dry
• Most of the green beans are coloured Green, the proportion with any Blue colour is 19%
• Coffee is mainly grown between 1000 and 2000 m; some is also grown at lower altitudes but very little is grown at higher altitudes
• Samples are generally provided either in bags of <10 kg or 50-80 kg
• Harvest.Year and Grading.Date show a similar pattern, suggesting that the age of the beans at grading is generally similar between samples; we don’t know exactly when the beans were harvested so any calculation of age of beans using assumptions is prone to error
• Some samples appear to be completely dry from their Moisture values, which seems a bit odd as the majority of samples have 0.08-0.14% water content
• Bean defects of Category One are rare, as are Quakers
• Bean defects of Category Two are more common and the distribution appears to be approximately log-normal

Output Variables

We have 10 individual quality scores, plus a summed overall quality. Let’s visualise how these scores are distributed.

From the output variables, we see:

• Aroma, Flavor, Aftertaste, Acidity, Body, Balance and overall Cupper.Points are similarly distributed with mean values around 7.5 and approximate range of 6.5-8.5
• Uniformity, Clean.Cup and Sweetness are generally highly rated with little deviation from scores of 10
• Total.Cup.Points has a distribution with a mean of approximately 83 and general range of 77-87

We are really interested in Total.Cup.Points as our main quality variable, so let’s look at how this output variable is impacted by the different input variables.

Variable Relationships

We can draw boxplots to illustrate relationships of Total.Cup.Points with categorical input variables.

What do these relationships tell us?

• Arabica beans tend to give slightly higher scores than Robusta
• We see more variation in scores with Country.of.Origin and bean Variety - Ethiopian Yirgacheffe appears to be generally the coffee with highest scores
• We see less variation in scores with Processing.Method and Color

We can draw scatter plots to illustrate relationships of Total.Cup.Points with numeric input variables.

There are no clear trends in these relationships. However, the best scores, i.e. >85 Total.Cup.Points, tend to only be seen when the number of defects and Quakers are low. Quality appears to perhaps increase up to around 2000 m altitude, and then decrease for the few coffees at higher altitudes.

We can now take this data forward in an attempt to model Total.Cup.Points.

---

End