1 Introduction

The purpose of this report is to study the performance of Starbucks based on its closing prices from the year 1992 until 2023. In this time period, starbucks has been boycotted on multiple occasions so this will be something to keep note of as this investigation is pursued. Ultimately, given that these boycotts ocurred, I’ll be interested to see what the forecast is for the starbucks stock and if these bocotts will have a lasting effect whether it be slowing the growth of the stock, or even causing it to lose value.

Note the the data downloaded has the value of the stock at the first of the month everymonth since 1960. This means that as I proceed with my project, I do not need to take into account trading days or anything like that.

2 Processing and GGplot

We will begin with a simple overview of our dataset to identify if anything needs to be addressed.


df <- read.csv("/Users/sumayyakheireddine/Documents/FALLUOttawa2023/4307/Group Project/SBUX.csv")

ggplot(data = df, mapping = aes(x=  Date, y = Adj.Close)) +
        geom_point() +
  ylab("Adjusted Close ") + 
  theme(legend.position = "none",
        axis.text.x = element_text(angle = 40, hjust = 1, size = 8))

Here, though the dates are not visible, we can see it’s consistent for quite a while with a slightly upward trend, it dips, and then steadily rises. It remains consistent and then rises again and at some point, more recently, the trend become indistinguishable. We can look into this furhter.

df$Date <- as.Date(df$Date, format = "%Y-%m-%d")

tsib <- df |> mutate(Month = yearmonth(Date)) |>
as_tsibble(index = Month)

tsib2015<- tsib|>
  filter(yearmonth(Date) >= yearmonth("2015-01-01")) |>
   autoplot(Close)+
  labs(title = "Closing Price of SBUX", y = "$US")

tsib2015

This graph is clearer. We can see some sawtooth behavior starting around 2019. Again, we’ll keep this in mind. For the purpose of modelling and forecasting, we’ll use the entire data set. It doesn’t seem to. be consistent dips so I don’t think there’s any seasonality there.

Based on based on Pegel’s (1969) classification, it looks like this data set has an additive trend with multiplicative seasonality.

However, to double check, I’d like to plot a seasonal plot using months to see if there are any less evident trends that are consistent throughout the years.

dcmp <- tsib |>
model(stl = STL(Close))
components(dcmp) |> 
autoplot()

It seems I was right and there is in fact no seasonality in terms of months. Perhaps this is because Starbucks has become relatively inelastic to their customers. However, there is yearly seasonality which makes a lot more sense.

We can use an ACF to confirm if there are in fact any trends worth obeserving.

tsib |> ACF(Close) |> autoplot()

This ACF corroborates the trend we have identified. The autocorrelations for the smaller lags are larger and positive because because observations nearby in time are also nearby in value. The slow decrease in the ACF as the lags increase is due to the trend.

Evidently, this is not white noise.

3 Model 1 : Basic Forecasting Model

Let’s start with a simple fitted forecast.

fit <- tsib |>
  model(
    Mean = MEAN(Close),
    `Naïve` = NAIVE(Close),
    `Seasonal naïve` = SNAIVE(Close)
  )

fc <- fit |>
forecast(h = "3 years") |>
autoplot(tsib) + labs(title = "Closing Price of SBUX", y = "$US")

fc

It looks like the seasonal naive is the best model. However, the data is not seasonal so let’s use naive instead. We can also use the accuracy function to help us pick the best of the models.

accuracy(fit) |>
arrange(.model) |>
select(.model, .type, RMSE, MAE, MAPE, MASE, RMSSE)

As expected, the RMSE of the naive is smallest. Let’s inspect the naive method further.

Check Residuals

tsib |>
  model(NAIVE(Close)) |>
  gg_tsresiduals()

In terms of residuals:The innovation residuals have about a zero mean suggesting the forecast is good.

In terms of the ACF: The lack of correlation suggests the forecast is good.

In terms of the histogram: It’s not necessary that the forecast be entirely normally distributed, and this only slightly skewed to the right. It is not ideal but it’s okay.

There is no data that needs to be removed.

Let’s now do a training set and test set for visualization purposes.

tsibmodel1train <- tsib |> filter(Date <= as.Date("2021-01-01"))

tsibmodel1test <-tsib |> filter(Date > as.Date("2021-01-01"))

fit <- tsibmodel1train |>
  model(
    `Seasonal naïve` = SNAIVE(Close)
  )

fcmodel1good <- fit |>
forecast(h = "2 years") |>
autoplot(tsib) + labs(title = "Closing Price of SBUX", y = "$US") +
autolayer(tsibmodel1test)
Plot variable not specified, automatically selected `.vars = Open`
fcmodel1good

About half of this data looks accruate while the left-portion is entirely outside of our interval of confidence.

4 Model 2: ETS

To decide whether to use SES, Holt, or Damped Holt, let’s find out which one has the lowest RMSE.

Here, I’ll be using the letters M for multiplicative error, A for trend, and N for no seasonality.

For this model, it will be trained on up to 2 years before the end of the data.This is chosen arbitrarily but mostly because there have been extreme fluctuations in the past 2 years that I want to see if the model will be able to predict.

Initially I used M in the seasonality, but I received this error.

Warning: 8 errors (2 unique) encountered for Holt [3] A seasonal ETS model cannot be used for this data.

Warning: 9 errors (2 unique) encountered for Damped [3] A seasonal ETS model cannot be used for this data.

Both of these errors are making me think, there is no seasonality, so I am changing that paramenter into a N.


fcmodel2train <- tsib |> 
  filter(Month <= as.Date("2021-01-01")) |>
  model(
    Holt = ETS(Close ~ error("A") + trend("A") + season("N")),
    Damped = ETS(Close ~ error("A") + trend("Ad") + season("N"))
  ) |>
  forecast(h = 36)
Warning: There was 1 warning in `filter()`.
ℹ In argument: `Month <= as.Date("2021-01-01")`.
Caused by warning:
! Incompatible methods ("<=.vctrs_vctr", "<=.Date") for "<="
  

fcmodel2train |> as_tsibble() 

fcmodel2test <- tsib |> 
  filter(Month > as.Date("2021-01-01")) |>
  as_tsibble()
Warning: There was 1 warning in `filter()`.
ℹ In argument: `Month > as.Date("2021-01-01")`.
Caused by warning:
! Incompatible methods (">.vctrs_vctr", ">.Date") for ">"
autoplot(fcmodel2train) + 
  labs(title = "Closing Price of SBUX", y = "$US") + autolayer(fcmodel2test)
Plot variable not specified, automatically selected `.vars = Open`

Unfortunately, the RMSE within accuracy function is not working. However, it seems like the damped trend is only slightly more accurate in terms of mean and confidence interval. Let’s plot the one with the lower RMSE, the Damped Holt.

fcmodel2train <- tsib |> 
  filter(Month <= as.Date("2021-01-01")) |>
  model(
    Damped = ETS(Close ~ error("A") + trend("Ad") + season("N"))
  ) |>
  forecast(h = 36)
Warning: There was 1 warning in `filter()`.
ℹ In argument: `Month <= as.Date("2021-01-01")`.
Caused by warning:
! Incompatible methods ("<=.vctrs_vctr", "<=.Date") for "<="
  

fcmodel2train |> as_tsibble(key=Month)

fcmodel2test <- tsib |> 
  filter(Month > as.Date("2021-01-01")) |>
  as_tsibble()
Warning: There was 1 warning in `filter()`.
ℹ In argument: `Month > as.Date("2021-01-01")`.
Caused by warning:
! Incompatible methods (">.vctrs_vctr", ">.Date") for ">"
autoplot(fcmodel2train) + 
  labs(title = "Closing Price of SBUX", y = "$US") +autolayer(fcmodel2test)
Plot variable not specified, automatically selected `.vars = Open`

By validating the model using our training and test set, we can see, this model is still not entirely accurate even with the 95% confidence interval. Given the forecast is around the years of Covid, this does make sense.

5 Model 3: ARIMA

This data is not stationary, so the first thing we have to do is make it stationary.

After differencing once, the mean is now around zero but the variance is still not stabilized. So I took the log. This looks better. Let’s take a look at the ACF of this as well.

tsib <- tsib |>
  mutate(closediff = difference(log(Close)))

tsib |> autoplot(closediff)


 tsib |>
   model(ARIMA(closediff)) |> gg_tsresiduals()

The residuals has mean around zero but some data points still do not feel stabilized. The low negative values at 18 in the ACF indicate some sort of complex seasonality. We’ll let the auto ARIMA take care of that later on. The distribution also looks like an almost perfect normal distribution which is a good sign.

Let’s do a KPSS test on this.

sbux_diff |>
features(difference(log(Close)), unitroot_kpss)
Error: object 'sbux_diff' not found

Since the KPSS value is 0.1, our P value is higher than 0.5 and H0 is not rejected meanign that our data is stationary.

Let’s now design the Arima model.

fcmodel3train <- tsib |> 
  filter(Date <= as.Date("2021-01-01")) |>
  model(auto = ARIMA(Close, stepwise = FALSE, approx = FALSE))

fcmodel3train |>
forecast(h=36) |>
autoplot(tsib)

Let’s break down why the auto-model with these parameters was chosen <ARIMA(3,2,3).

The ARIMA(3,2,3) incorporates three key components. Firstly, the Autoregressive (AR) component with an order (p) of 3 captures the influence of the three most recent observations on the current value. This allows the model to figure out the patterns and dependencies in the historical data. Secondly, the Integrated (I) component with an order (d) of 2 represents the differencing operations applied to the time series. With a differencing order of 2, the model aims to achieve stationarity by removing trends and seasonality through consecutive differencing. This is interesting to know because earlier, I thought that one differencing would be enough. Lastly, the Moving Average (MA) component, with an order (q) of 3, accounts for the relationship between the current value and past forecast errors. The inclusion of a moving average helps the model adapt to any residual patterns not captured by the autoregressive component.

The ACF and PACF plots might show significant correlations at lags 1 and 12 due to the autoregressive and seasonal autoregressive terms, with a decay in correlations for higher lags, while the presence of moving average terms could lead to periodic spikes in the ACF.

6 Compare the Models

We already know that the naive model is simply used a benchmark. We expect both ETS and ARIMA to perform better. Let’s compare those two as a means of concluding the report and presenting the best forecast.

Lastly, let’s compare the two models. I’m using data up to halfway through covid since I do not want to remove the event that occured, but I want to partially factor it in.

split_date <- as.Date("2020-12-31")  # Adjust the split date as needed

train <- tsib %>%
  filter(Date <= split_date)

test <- tsib %>%
  filter(Date > split_date)

# Model training
fit <- train |>
  model(
    arima = ARIMA(Close),
    ets = ETS(Close)
  )

# Forecasting
forecast_results <- fit |>
  forecast(h = nrow(test)) |>
  accuracy(tsib) |>
select(-ME, -MPE, -ACF1)

forecast_results

Here we can see that the ETS has lower values for all measures and so in this case, we should use the ETS model for our forecasting.

fit |>
select(ets) |>
forecast(h = "5 years") |>
autoplot(tsib) +
labs(title = "Closing Price of SBUX", y = "$US")

I hope to conclude that starbucks will continue to perform with the trend increasing as rapidly as it did before as both models suggest, especially given their recent stance on the Palestinian conflict. This ETS model suggests a company that has high risk and reward when investing, and usually, Starbucks has been known to be a safe stock. I know that Starbucks is trying to convince investors to keep their stocks in the company but this report may convince investors otherwise.

7 Improvement

One thing I was not able to do during this project because of my lack of understanding of CPI was to adjust for inflation. Next time I undertake a project like this, I would like to factor this in as I think my analysis would be much more accurate and closer to reality.

8 References

“Starbucks Corporation (SBUX) Stock Historical Prices & Data.” Yahoo! Finance, Yahoo!, 13 Dec. 2023, finance.yahoo.com/quote/SBUX/history?p=SBUX.

LS0tCnRpdGxlOiAnQURNIDQzMDc6IEZpbmFsIFJlcG9ydCcKYXV0aG9yOiAiU3VtYXl5YSBLaGVpcmVkZGluZSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgcGRmX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICB3b3JkX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKLS0tCgojIEludHJvZHVjdGlvbgoKVGhlIHB1cnBvc2Ugb2YgdGhpcyByZXBvcnQgaXMgdG8gc3R1ZHkgdGhlIHBlcmZvcm1hbmNlIG9mIFN0YXJidWNrcyBiYXNlZCBvbiBpdHMgY2xvc2luZyBwcmljZXMgZnJvbSB0aGUgeWVhciAxOTkyIHVudGlsIDIwMjMuIEluIHRoaXMgdGltZSBwZXJpb2QsIHN0YXJidWNrcyBoYXMgYmVlbiBib3ljb3R0ZWQgb24gbXVsdGlwbGUgb2NjYXNpb25zIHNvIHRoaXMgd2lsbCBiZSBzb21ldGhpbmcgdG8ga2VlcCBub3RlIG9mIGFzIHRoaXMgaW52ZXN0aWdhdGlvbiBpcyBwdXJzdWVkLiBVbHRpbWF0ZWx5LCBnaXZlbiB0aGF0IHRoZXNlIGJveWNvdHRzIG9jdXJyZWQsIEknbGwgYmUgaW50ZXJlc3RlZCB0byBzZWUgd2hhdCB0aGUgZm9yZWNhc3QgaXMgZm9yIHRoZSBzdGFyYnVja3Mgc3RvY2sgYW5kIGlmIHRoZXNlIGJvY290dHMgd2lsbCBoYXZlIGEgbGFzdGluZyBlZmZlY3Qgd2hldGhlciBpdCBiZSBzbG93aW5nIHRoZSBncm93dGggb2YgdGhlIHN0b2NrLCBvciBldmVuIGNhdXNpbmcgaXQgdG8gbG9zZSB2YWx1ZS4gIAoKTm90ZSB0aGUgdGhlIGRhdGEgZG93bmxvYWRlZCBoYXMgdGhlIHZhbHVlIG9mIHRoZSBzdG9jayBhdCB0aGUgZmlyc3Qgb2YgdGhlIG1vbnRoIGV2ZXJ5bW9udGggc2luY2UgMTk2MC4gVGhpcyBtZWFucyB0aGF0IGFzIEkgcHJvY2VlZCB3aXRoIG15IHByb2plY3QsIEkgZG8gbm90IG5lZWQgdG8gdGFrZSBpbnRvIGFjY291bnQgdHJhZGluZyBkYXlzIG9yIGFueXRoaW5nIGxpa2UgdGhhdC4KCiAKIyBQcm9jZXNzaW5nIGFuZCBHR3Bsb3QKV2Ugd2lsbCBiZWdpbiB3aXRoIGEgc2ltcGxlIG92ZXJ2aWV3IG9mIG91ciBkYXRhc2V0IHRvIGlkZW50aWZ5IGlmIGFueXRoaW5nIG5lZWRzIHRvIGJlIGFkZHJlc3NlZC4KCmBgYHtyfQpsaWJyYXJ5KGdncGxvdDIpCmRmIDwtIHJlYWQuY3N2KCIvVXNlcnMvc3VtYXl5YWtoZWlyZWRkaW5lL0RvY3VtZW50cy9GQUxMVU90dGF3YTIwMjMvNDMwNy9Hcm91cCBQcm9qZWN0L1NCVVguY3N2IikKCmdncGxvdChkYXRhID0gZGYsIG1hcHBpbmcgPSBhZXMoeD0gIERhdGUsIHkgPSBBZGouQ2xvc2UpKSArCiAgICAgICAgZ2VvbV9wb2ludCgpICsKICB5bGFiKCJBZGp1c3RlZCBDbG9zZSAiKSArIAogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwKICAgICAgICBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQwLCBoanVzdCA9IDEsIHNpemUgPSA4KSkKYGBgCkhlcmUsIHRob3VnaCB0aGUgZGF0ZXMgYXJlIG5vdCB2aXNpYmxlLCB3ZSBjYW4gc2VlIGl0J3MgY29uc2lzdGVudCBmb3IgcXVpdGUgYSB3aGlsZSB3aXRoIGEgc2xpZ2h0bHkgdXB3YXJkIHRyZW5kLCBpdCBkaXBzLCBhbmQgdGhlbiBzdGVhZGlseSByaXNlcy4gSXQgcmVtYWlucyBjb25zaXN0ZW50IGFuZCB0aGVuIHJpc2VzIGFnYWluIGFuZCBhdCBzb21lIHBvaW50LCBtb3JlIHJlY2VudGx5LCB0aGUgdHJlbmQgYmVjb21lIGluZGlzdGluZ3Vpc2hhYmxlLiBXZSBjYW4gbG9vayBpbnRvIHRoaXMgZnVyaHRlci4KCgpgYGB7cn0KbGlicmFyeSh0c2liYmxlKQpsaWJyYXJ5KHRpYmJsZSkKZGYkRGF0ZSA8LSBhcy5EYXRlKGRmJERhdGUsIGZvcm1hdCA9ICIlWS0lbS0lZCIpCgp0c2liIDwtIGRmIHw+IG11dGF0ZShNb250aCA9IHllYXJtb250aChEYXRlKSkgfD4KYXNfdHNpYmJsZShpbmRleCA9IE1vbnRoKQoKdHNpYjIwMTU8LSB0c2lifD4KICBmaWx0ZXIoeWVhcm1vbnRoKERhdGUpID49IHllYXJtb250aCgiMjAxNS0wMS0wMSIpKSB8PgogICBhdXRvcGxvdChDbG9zZSkrCiAgbGFicyh0aXRsZSA9ICJDbG9zaW5nIFByaWNlIG9mIFNCVVgiLCB5ID0gIiRVUyIpCgp0c2liMjAxNQpgYGAKClRoaXMgZ3JhcGggaXMgY2xlYXJlci4gV2UgY2FuIHNlZSBzb21lIHNhd3Rvb3RoIGJlaGF2aW9yIHN0YXJ0aW5nIGFyb3VuZCAyMDE5LiBBZ2Fpbiwgd2UnbGwga2VlcCB0aGlzIGluIG1pbmQuIEZvciB0aGUgcHVycG9zZSBvZiBtb2RlbGxpbmcgYW5kIGZvcmVjYXN0aW5nLCB3ZSdsbCB1c2UgdGhlIGVudGlyZSBkYXRhIHNldC4gSXQgZG9lc24ndCBzZWVtIHRvLiBiZSBjb25zaXN0ZW50IGRpcHMgc28gSSBkb24ndCB0aGluayB0aGVyZSdzIGFueSBzZWFzb25hbGl0eSB0aGVyZS4KCkJhc2VkIG9uIGJhc2VkIG9uIFBlZ2VsJ3MgKDE5NjkpIGNsYXNzaWZpY2F0aW9uLCBpdCBsb29rcyBsaWtlIHRoaXMgZGF0YSBzZXQgaGFzIGFuIGFkZGl0aXZlIHRyZW5kIHdpdGggbXVsdGlwbGljYXRpdmUgc2Vhc29uYWxpdHkuCgpIb3dldmVyLCB0byBkb3VibGUgY2hlY2ssIEknZCBsaWtlIHRvIHBsb3QgYSBzZWFzb25hbCBwbG90IHVzaW5nIG1vbnRocyB0byBzZWUgaWYgdGhlcmUgYXJlIGFueSBsZXNzIGV2aWRlbnQgdHJlbmRzIHRoYXQgYXJlIGNvbnNpc3RlbnQgdGhyb3VnaG91dCB0aGUgeWVhcnMuCgpgYGB7cn0KZGNtcCA8LSB0c2liIHw+Cm1vZGVsKHN0bCA9IFNUTChDbG9zZSkpCmNvbXBvbmVudHMoZGNtcCkgfD4gCmF1dG9wbG90KCkKYGBgCkl0IHNlZW1zIEkgd2FzIHJpZ2h0IGFuZCB0aGVyZSBpcyBpbiBmYWN0IG5vIHNlYXNvbmFsaXR5IGluIHRlcm1zIG9mIG1vbnRocy4gUGVyaGFwcyB0aGlzIGlzIGJlY2F1c2UgU3RhcmJ1Y2tzIGhhcyBiZWNvbWUgcmVsYXRpdmVseSBpbmVsYXN0aWMgdG8gdGhlaXIgY3VzdG9tZXJzLiBIb3dldmVyLCB0aGVyZSBpcyB5ZWFybHkgc2Vhc29uYWxpdHkgd2hpY2ggbWFrZXMgYSBsb3QgbW9yZSBzZW5zZS4KCldlIGNhbiB1c2UgYW4gQUNGIHRvIGNvbmZpcm0gaWYgdGhlcmUgYXJlIGluIGZhY3QgYW55IHRyZW5kcyB3b3J0aCBvYmVzZXJ2aW5nLgpgYGB7cn0KdHNpYiB8PiBBQ0YoQ2xvc2UpIHw+IGF1dG9wbG90KCkKYGBgClRoaXMgQUNGIGNvcnJvYm9yYXRlcyB0aGUgdHJlbmQgd2UgaGF2ZSBpZGVudGlmaWVkLiBUaGUgYXV0b2NvcnJlbGF0aW9ucyBmb3IgdGhlIHNtYWxsZXIgbGFncyBhcmUgbGFyZ2VyIGFuZCBwb3NpdGl2ZSBiZWNhdXNlIGJlY2F1c2Ugb2JzZXJ2YXRpb25zIG5lYXJieSBpbiB0aW1lIGFyZSBhbHNvIG5lYXJieSBpbiB2YWx1ZS4gVGhlIHNsb3cgZGVjcmVhc2UgaW4gdGhlIEFDRiBhcyB0aGUgbGFncyBpbmNyZWFzZSBpcyBkdWUgdG8gdGhlIHRyZW5kLgoKRXZpZGVudGx5LCB0aGlzIGlzIG5vdCB3aGl0ZSBub2lzZS4KCiMgTW9kZWwgMSA6IEJhc2ljIEZvcmVjYXN0aW5nIE1vZGVsCgpMZXQncyBzdGFydCB3aXRoIGEgc2ltcGxlIGZpdHRlZCBmb3JlY2FzdC4KYGBge3J9CmZpdCA8LSB0c2liIHw+CiAgbW9kZWwoCiAgICBNZWFuID0gTUVBTihDbG9zZSksCiAgICBgTmHDr3ZlYCA9IE5BSVZFKENsb3NlKSwKICAgIGBTZWFzb25hbCBuYcOvdmVgID0gU05BSVZFKENsb3NlKQogICkKCmZjIDwtIGZpdCB8Pgpmb3JlY2FzdChoID0gIjMgeWVhcnMiKSB8PgphdXRvcGxvdCh0c2liKSArIGxhYnModGl0bGUgPSAiQ2xvc2luZyBQcmljZSBvZiBTQlVYIiwgeSA9ICIkVVMiKQoKZmMKYGBgCgpJdCBsb29rcyBsaWtlIHRoZSBzZWFzb25hbCBuYWl2ZSBpcyB0aGUgYmVzdCBtb2RlbC4gSG93ZXZlciwgdGhlIGRhdGEgaXMgbm90IHNlYXNvbmFsIHNvIGxldCdzIHVzZSBuYWl2ZSBpbnN0ZWFkLiBXZSBjYW4gYWxzbyB1c2UgdGhlIGFjY3VyYWN5IGZ1bmN0aW9uIHRvIGhlbHAgdXMgcGljayB0aGUgYmVzdCBvZiB0aGUgbW9kZWxzLgpgYGB7cn0KYWNjdXJhY3koZml0KSB8PgphcnJhbmdlKC5tb2RlbCkgfD4Kc2VsZWN0KC5tb2RlbCwgLnR5cGUsIFJNU0UsIE1BRSwgTUFQRSwgTUFTRSwgUk1TU0UpCmBgYApBcyBleHBlY3RlZCwgdGhlIFJNU0Ugb2YgdGhlIG5haXZlIGlzIHNtYWxsZXN0LiBMZXQncyBpbnNwZWN0IHRoZSBuYWl2ZSBtZXRob2QgZnVydGhlci4KCkNoZWNrIFJlc2lkdWFscwoKYGBge3J9CnRzaWIgfD4KICBtb2RlbChOQUlWRShDbG9zZSkpIHw+CiAgZ2dfdHNyZXNpZHVhbHMoKQpgYGAKCkluIHRlcm1zIG9mIHJlc2lkdWFsczpUaGUgaW5ub3ZhdGlvbiByZXNpZHVhbHMgaGF2ZSBhYm91dCBhIHplcm8gbWVhbiBzdWdnZXN0aW5nIHRoZSBmb3JlY2FzdCBpcyBnb29kLgoKSW4gdGVybXMgb2YgdGhlIEFDRjogVGhlIGxhY2sgb2YgY29ycmVsYXRpb24gc3VnZ2VzdHMgdGhlIGZvcmVjYXN0IGlzIGdvb2QuIAoKSW4gdGVybXMgb2YgdGhlIGhpc3RvZ3JhbTogSXQncyBub3QgbmVjZXNzYXJ5IHRoYXQgdGhlIGZvcmVjYXN0IGJlIGVudGlyZWx5IG5vcm1hbGx5IGRpc3RyaWJ1dGVkLCBhbmQgdGhpcyBvbmx5IHNsaWdodGx5IHNrZXdlZCB0byB0aGUgcmlnaHQuIEl0IGlzIG5vdCBpZGVhbCBidXQgaXQncyBva2F5LgoKVGhlcmUgaXMgbm8gZGF0YSB0aGF0IG5lZWRzIHRvIGJlIHJlbW92ZWQuCgpMZXQncyBub3cgZG8gYSB0cmFpbmluZyBzZXQgYW5kIHRlc3Qgc2V0IGZvciB2aXN1YWxpemF0aW9uIHB1cnBvc2VzLgoKYGBge3J9CnRzaWJtb2RlbDF0cmFpbiA8LSB0c2liIHw+IGZpbHRlcihEYXRlIDw9IGFzLkRhdGUoIjIwMjEtMDEtMDEiKSkKCnRzaWJtb2RlbDF0ZXN0IDwtdHNpYiB8PiBmaWx0ZXIoRGF0ZSA+IGFzLkRhdGUoIjIwMjEtMDEtMDEiKSkKCmZpdCA8LSB0c2libW9kZWwxdHJhaW4gfD4KICBtb2RlbCgKICAgIGBTZWFzb25hbCBuYcOvdmVgID0gU05BSVZFKENsb3NlKQogICkKCmZjbW9kZWwxZ29vZCA8LSBmaXQgfD4KZm9yZWNhc3QoaCA9ICIyIHllYXJzIikgfD4KYXV0b3Bsb3QodHNpYikgKyBsYWJzKHRpdGxlID0gIkNsb3NpbmcgUHJpY2Ugb2YgU0JVWCIsIHkgPSAiJFVTIikgKwphdXRvbGF5ZXIodHNpYm1vZGVsMXRlc3QpCgpmY21vZGVsMWdvb2QKYGBgCgpBYm91dCBoYWxmIG9mIHRoaXMgZGF0YSBsb29rcyBhY2NydWF0ZSB3aGlsZSB0aGUgbGVmdC1wb3J0aW9uIGlzIGVudGlyZWx5IG91dHNpZGUgb2Ygb3VyIGludGVydmFsIG9mIGNvbmZpZGVuY2UuCgoKIyBNb2RlbCAyOiBFVFMKVG8gZGVjaWRlIHdoZXRoZXIgdG8gdXNlIFNFUywgSG9sdCwgb3IgRGFtcGVkIEhvbHQsIGxldCdzIGZpbmQgb3V0IHdoaWNoIG9uZSBoYXMgdGhlIGxvd2VzdCBSTVNFLgoKSGVyZSwgSSdsbCBiZSB1c2luZyB0aGUgbGV0dGVycyBNIGZvciBtdWx0aXBsaWNhdGl2ZSBlcnJvciwgQSBmb3IgdHJlbmQsIGFuZCBOIGZvciBubyBzZWFzb25hbGl0eS4KCkZvciB0aGlzIG1vZGVsLCBpdCB3aWxsIGJlIHRyYWluZWQgb24gdXAgdG8gMiB5ZWFycyBiZWZvcmUgdGhlIGVuZCBvZiB0aGUgZGF0YS5UaGlzIGlzIGNob3NlbiBhcmJpdHJhcmlseSBidXQgbW9zdGx5IGJlY2F1c2UgdGhlcmUgaGF2ZSBiZWVuIGV4dHJlbWUgZmx1Y3R1YXRpb25zIGluIHRoZSBwYXN0IDIgeWVhcnMgdGhhdCBJIHdhbnQgdG8gc2VlIGlmIHRoZSBtb2RlbCB3aWxsIGJlIGFibGUgdG8gcHJlZGljdC4gCgpJbml0aWFsbHkgSSB1c2VkIE0gaW4gdGhlIHNlYXNvbmFsaXR5LCBidXQgSSByZWNlaXZlZCB0aGlzIGVycm9yLiAKCldhcm5pbmc6IDggZXJyb3JzICgyIHVuaXF1ZSkgZW5jb3VudGVyZWQgZm9yIEhvbHQKWzNdIEEgc2Vhc29uYWwgRVRTIG1vZGVsIGNhbm5vdCBiZSB1c2VkIGZvciB0aGlzIGRhdGEuCgpXYXJuaW5nOiA5IGVycm9ycyAoMiB1bmlxdWUpIGVuY291bnRlcmVkIGZvciBEYW1wZWQKWzNdIEEgc2Vhc29uYWwgRVRTIG1vZGVsIGNhbm5vdCBiZSB1c2VkIGZvciB0aGlzIGRhdGEuCgoKQm90aCBvZiB0aGVzZSBlcnJvcnMgYXJlIG1ha2luZyBtZSB0aGluaywgdGhlcmUgaXMgbm8gc2Vhc29uYWxpdHksIHNvIEkgYW0gY2hhbmdpbmcgdGhhdCBwYXJhbWVudGVyIGludG8gYSBOLgoKYGBge3J9CgpmY21vZGVsMnRyYWluIDwtIHRzaWIgfD4gCiAgZmlsdGVyKE1vbnRoIDw9IGFzLkRhdGUoIjIwMjEtMDEtMDEiKSkgfD4KICBtb2RlbCgKICAgIEhvbHQgPSBFVFMoQ2xvc2UgfiBlcnJvcigiQSIpICsgdHJlbmQoIkEiKSArIHNlYXNvbigiTiIpKSwKICAgIERhbXBlZCA9IEVUUyhDbG9zZSB+IGVycm9yKCJBIikgKyB0cmVuZCgiQWQiKSArIHNlYXNvbigiTiIpKQogICkgfD4KICBmb3JlY2FzdChoID0gMzYpCiAgCgpmY21vZGVsMnRyYWluIHw+IGFzX3RzaWJibGUoKSAKCmZjbW9kZWwydGVzdCA8LSB0c2liIHw+IAogIGZpbHRlcihNb250aCA+IGFzLkRhdGUoIjIwMjEtMDEtMDEiKSkgfD4KICBhc190c2liYmxlKCkKCmF1dG9wbG90KGZjbW9kZWwydHJhaW4pICsgCiAgbGFicyh0aXRsZSA9ICJDbG9zaW5nIFByaWNlIG9mIFNCVVgiLCB5ID0gIiRVUyIpICsgYXV0b2xheWVyKGZjbW9kZWwydGVzdCkKCmBgYApVbmZvcnR1bmF0ZWx5LCB0aGUgUk1TRSB3aXRoaW4gYWNjdXJhY3kgZnVuY3Rpb24gaXMgbm90IHdvcmtpbmcuIEhvd2V2ZXIsIGl0IHNlZW1zIGxpa2UgdGhlIGRhbXBlZCB0cmVuZCBpcyBvbmx5IHNsaWdodGx5IG1vcmUgYWNjdXJhdGUgaW4gdGVybXMgb2YgbWVhbiBhbmQgY29uZmlkZW5jZSBpbnRlcnZhbC4gTGV0J3MgcGxvdCB0aGUgb25lIHdpdGggdGhlIGxvd2VyIFJNU0UsIHRoZSBEYW1wZWQgSG9sdC4KCmBgYHtyfQpmY21vZGVsMnRyYWluIDwtIHRzaWIgfD4gCiAgZmlsdGVyKE1vbnRoIDw9IGFzLkRhdGUoIjIwMjEtMDEtMDEiKSkgfD4KICBtb2RlbCgKICAgIERhbXBlZCA9IEVUUyhDbG9zZSB+IGVycm9yKCJBIikgKyB0cmVuZCgiQWQiKSArIHNlYXNvbigiTiIpKQogICkgfD4KICBmb3JlY2FzdChoID0gMzYpCiAgCgpmY21vZGVsMnRyYWluIHw+IGFzX3RzaWJibGUoa2V5PU1vbnRoKQoKZmNtb2RlbDJ0ZXN0IDwtIHRzaWIgfD4gCiAgZmlsdGVyKE1vbnRoID4gYXMuRGF0ZSgiMjAyMS0wMS0wMSIpKSB8PgogIGFzX3RzaWJibGUoKQoKCmF1dG9wbG90KGZjbW9kZWwydHJhaW4pICsgCiAgbGFicyh0aXRsZSA9ICJDbG9zaW5nIFByaWNlIG9mIFNCVVgiLCB5ID0gIiRVUyIpICthdXRvbGF5ZXIoZmNtb2RlbDJ0ZXN0KQpgYGAKQnkgdmFsaWRhdGluZyB0aGUgbW9kZWwgdXNpbmcgb3VyIHRyYWluaW5nIGFuZCB0ZXN0IHNldCwgd2UgY2FuIHNlZSwgdGhpcyBtb2RlbCBpcyBzdGlsbCBub3QgZW50aXJlbHkgYWNjdXJhdGUgZXZlbiB3aXRoIHRoZSA5NSUgY29uZmlkZW5jZSBpbnRlcnZhbC4gR2l2ZW4gdGhlIGZvcmVjYXN0IGlzIGFyb3VuZCB0aGUgeWVhcnMgb2YgQ292aWQsIHRoaXMgZG9lcyBtYWtlIHNlbnNlLgoKIyBNb2RlbCAzOiBBUklNQQoKVGhpcyBkYXRhIGlzIG5vdCBzdGF0aW9uYXJ5LCBzbyB0aGUgZmlyc3QgdGhpbmcgd2UgaGF2ZSB0byBkbyBpcyBtYWtlIGl0IHN0YXRpb25hcnkuCgpBZnRlciBkaWZmZXJlbmNpbmcgb25jZSwgdGhlIG1lYW4gaXMgbm93IGFyb3VuZCB6ZXJvIGJ1dCB0aGUgdmFyaWFuY2UgaXMgc3RpbGwgbm90IHN0YWJpbGl6ZWQuIFNvIEkgdG9vayB0aGUgbG9nLiBUaGlzIGxvb2tzIGJldHRlci4gTGV0J3MgdGFrZSBhIGxvb2sgYXQgdGhlIEFDRiBvZiB0aGlzIGFzIHdlbGwuCmBgYHtyfQp0c2liIDwtIHRzaWIgfD4KICBtdXRhdGUoY2xvc2VkaWZmID0gZGlmZmVyZW5jZShsb2coQ2xvc2UpKSkKCnRzaWIgfD4gYXV0b3Bsb3QoY2xvc2VkaWZmKQoKIHRzaWIgfD4KICAgbW9kZWwoQVJJTUEoY2xvc2VkaWZmKSkgfD4gZ2dfdHNyZXNpZHVhbHMoKQpgYGAKVGhlIHJlc2lkdWFscyBoYXMgbWVhbiBhcm91bmQgemVybyBidXQgc29tZSBkYXRhIHBvaW50cyBzdGlsbCBkbyBub3QgZmVlbCBzdGFiaWxpemVkLiBUaGUgbG93IG5lZ2F0aXZlIHZhbHVlcyBhdCAxOCBpbiB0aGUgQUNGIGluZGljYXRlIHNvbWUgc29ydCBvZiBjb21wbGV4IHNlYXNvbmFsaXR5LiBXZSdsbCBsZXQgdGhlIGF1dG8gQVJJTUEgdGFrZSBjYXJlIG9mIHRoYXQgbGF0ZXIgb24uIFRoZSBkaXN0cmlidXRpb24gYWxzbyBsb29rcyBsaWtlIGFuIGFsbW9zdCBwZXJmZWN0IG5vcm1hbCBkaXN0cmlidXRpb24gd2hpY2ggaXMgYSBnb29kIHNpZ24uCgoKTGV0J3MgZG8gYSBLUFNTIHRlc3Qgb24gdGhpcy4KYGBge3J9CnNidXhfZGlmZiB8PgpmZWF0dXJlcyhkaWZmZXJlbmNlKGxvZyhDbG9zZSkpLCB1bml0cm9vdF9rcHNzKQpgYGAKU2luY2UgdGhlIEtQU1MgdmFsdWUgaXMgMC4xLCBvdXIgUCB2YWx1ZSBpcyBoaWdoZXIgdGhhbiAwLjUgYW5kIEgwIGlzIG5vdCByZWplY3RlZCBtZWFuaWduIHRoYXQgb3VyIGRhdGEgaXMgc3RhdGlvbmFyeS4gCgoKTGV0J3Mgbm93IGRlc2lnbiB0aGUgQXJpbWEgbW9kZWwuCmBgYHtyfQpmY21vZGVsM3RyYWluIDwtIHRzaWIgfD4gCiAgZmlsdGVyKERhdGUgPD0gYXMuRGF0ZSgiMjAyMS0wMS0wMSIpKSB8PgogIG1vZGVsKGF1dG8gPSBBUklNQShDbG9zZSwgc3RlcHdpc2UgPSBGQUxTRSwgYXBwcm94ID0gRkFMU0UpKQoKZmNtb2RlbDN0cmFpbiB8Pgpmb3JlY2FzdChoPTM2KSB8PgphdXRvcGxvdCh0c2liKQpgYGAKCkxldCdzIGJyZWFrIGRvd24gd2h5IHRoZSBhdXRvLW1vZGVsIHdpdGggdGhlc2UgcGFyYW1ldGVycyB3YXMgY2hvc2VuIDxBUklNQSgzLDIsMykuIAoKVGhlIEFSSU1BKDMsMiwzKSBpbmNvcnBvcmF0ZXMgdGhyZWUga2V5IGNvbXBvbmVudHMuIEZpcnN0bHksIHRoZSBBdXRvcmVncmVzc2l2ZSAoQVIpIGNvbXBvbmVudCB3aXRoIGFuIG9yZGVyIChwKSBvZiAzIGNhcHR1cmVzIHRoZSBpbmZsdWVuY2Ugb2YgdGhlIHRocmVlIG1vc3QgcmVjZW50IG9ic2VydmF0aW9ucyBvbiB0aGUgY3VycmVudCB2YWx1ZS4gVGhpcyBhbGxvd3MgdGhlIG1vZGVsIHRvIGZpZ3VyZSBvdXQgdGhlIHBhdHRlcm5zIGFuZCBkZXBlbmRlbmNpZXMgaW4gdGhlIGhpc3RvcmljYWwgZGF0YS4gU2Vjb25kbHksIHRoZSBJbnRlZ3JhdGVkIChJKSBjb21wb25lbnQgd2l0aCBhbiBvcmRlciAoZCkgb2YgMiByZXByZXNlbnRzIHRoZSBkaWZmZXJlbmNpbmcgb3BlcmF0aW9ucyBhcHBsaWVkIHRvIHRoZSB0aW1lIHNlcmllcy4gV2l0aCBhIGRpZmZlcmVuY2luZyBvcmRlciBvZiAyLCB0aGUgbW9kZWwgYWltcyB0byBhY2hpZXZlIHN0YXRpb25hcml0eSBieSByZW1vdmluZyB0cmVuZHMgYW5kIHNlYXNvbmFsaXR5IHRocm91Z2ggY29uc2VjdXRpdmUgZGlmZmVyZW5jaW5nLiBUaGlzIGlzIGludGVyZXN0aW5nIHRvIGtub3cgYmVjYXVzZSBlYXJsaWVyLCBJIHRob3VnaHQgdGhhdCBvbmUgZGlmZmVyZW5jaW5nIHdvdWxkIGJlIGVub3VnaC4gCkxhc3RseSwgdGhlIE1vdmluZyBBdmVyYWdlIChNQSkgY29tcG9uZW50LCB3aXRoIGFuIG9yZGVyIChxKSBvZiAzLCBhY2NvdW50cyBmb3IgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSBjdXJyZW50IHZhbHVlIGFuZCBwYXN0IGZvcmVjYXN0IGVycm9ycy4gVGhlIGluY2x1c2lvbiBvZiBhIG1vdmluZyBhdmVyYWdlIGhlbHBzIHRoZSBtb2RlbCBhZGFwdCB0byBhbnkgcmVzaWR1YWwgcGF0dGVybnMgbm90IGNhcHR1cmVkIGJ5IHRoZSBhdXRvcmVncmVzc2l2ZSBjb21wb25lbnQuCgoKVGhlIEFDRiBhbmQgUEFDRiBwbG90cyBtaWdodCBzaG93IHNpZ25pZmljYW50IGNvcnJlbGF0aW9ucyBhdCBsYWdzIDEgYW5kIDEyIGR1ZSB0byB0aGUgYXV0b3JlZ3Jlc3NpdmUgYW5kIHNlYXNvbmFsIGF1dG9yZWdyZXNzaXZlIHRlcm1zLCB3aXRoIGEgZGVjYXkgaW4gY29ycmVsYXRpb25zIGZvciBoaWdoZXIgbGFncywgd2hpbGUgdGhlIHByZXNlbmNlIG9mIG1vdmluZyBhdmVyYWdlIHRlcm1zIGNvdWxkIGxlYWQgdG8gcGVyaW9kaWMgc3Bpa2VzIGluIHRoZSBBQ0YuCgoKCiMgQ29tcGFyZSB0aGUgTW9kZWxzCgpXZSBhbHJlYWR5IGtub3cgdGhhdCB0aGUgbmFpdmUgbW9kZWwgaXMgc2ltcGx5IHVzZWQgYSBiZW5jaG1hcmsuIFdlIGV4cGVjdCBib3RoIEVUUyBhbmQgQVJJTUEgdG8gcGVyZm9ybSBiZXR0ZXIuIExldCdzIGNvbXBhcmUgdGhvc2UgdHdvIGFzIGEgbWVhbnMgb2YgY29uY2x1ZGluZyB0aGUgcmVwb3J0IGFuZCBwcmVzZW50aW5nIHRoZSBiZXN0IGZvcmVjYXN0LiAKCgpMYXN0bHksIGxldCdzIGNvbXBhcmUgdGhlIHR3byBtb2RlbHMuIEknbSB1c2luZyBkYXRhIHVwIHRvIGhhbGZ3YXkgdGhyb3VnaCBjb3ZpZCBzaW5jZSBJIGRvIG5vdCB3YW50IHRvIHJlbW92ZSB0aGUgZXZlbnQgdGhhdCBvY2N1cmVkLCBidXQgSSB3YW50IHRvIHBhcnRpYWxseSBmYWN0b3IgaXQgaW4uCmBgYHtyfQpzcGxpdF9kYXRlIDwtIGFzLkRhdGUoIjIwMjAtMTItMzEiKSAKCnRyYWluIDwtIHRzaWIgfD4KICBmaWx0ZXIoRGF0ZSA8PSBzcGxpdF9kYXRlKQoKdGVzdCA8LSB0c2liIHw+CiAgZmlsdGVyKERhdGUgPiBzcGxpdF9kYXRlKQoKZml0IDwtIHRyYWluIHw+CiAgbW9kZWwoCiAgICBhcmltYSA9IEFSSU1BKENsb3NlKSwKICAgIGV0cyA9IEVUUyhDbG9zZSkKICApCgpmb3JlY2FzdF9yZXN1bHRzIDwtIGZpdCB8PgogIGZvcmVjYXN0KGggPSBucm93KHRlc3QpKSB8PgogIGFjY3VyYWN5KHRzaWIpIHw+CnNlbGVjdCgtTUUsIC1NUEUsIC1BQ0YxKQoKZm9yZWNhc3RfcmVzdWx0cwpgYGAKSGVyZSB3ZSBjYW4gc2VlIHRoYXQgdGhlIEVUUyBoYXMgbG93ZXIgdmFsdWVzIGZvciBhbGwgbWVhc3VyZXMgYW5kIHNvIGluIHRoaXMgY2FzZSwgd2Ugc2hvdWxkIHVzZSB0aGUgRVRTIG1vZGVsIGZvciBvdXIgZm9yZWNhc3RpbmcuIAoKYGBge3J9CmZpdCB8PgpzZWxlY3QoZXRzKSB8Pgpmb3JlY2FzdChoID0gIjUgeWVhcnMiKSB8PgphdXRvcGxvdCh0c2liKSArCmxhYnModGl0bGUgPSAiQ2xvc2luZyBQcmljZSBvZiBTQlVYIiwgeSA9ICIkVVMiKQpgYGAKCkkgaG9wZSB0byBjb25jbHVkZSB0aGF0IHN0YXJidWNrcyB3aWxsIGNvbnRpbnVlIHRvIHBlcmZvcm0gd2l0aCB0aGUgdHJlbmQgaW5jcmVhc2luZyBhcyByYXBpZGx5IGFzIGl0IGRpZCBiZWZvcmUgYXMgYm90aCBtb2RlbHMgc3VnZ2VzdCwgZXNwZWNpYWxseSBnaXZlbiB0aGVpciByZWNlbnQgc3RhbmNlIG9uIHRoZSBQYWxlc3RpbmlhbiBjb25mbGljdC4gVGhpcyBFVFMgbW9kZWwgc3VnZ2VzdHMgYSBjb21wYW55IHRoYXQgaGFzIGhpZ2ggcmlzayBhbmQgcmV3YXJkIHdoZW4gaW52ZXN0aW5nLCBhbmQgdXN1YWxseSwgU3RhcmJ1Y2tzIGhhcyBiZWVuIGtub3duIHRvIGJlIGEgc2FmZSBzdG9jay4gSSBrbm93IHRoYXQgU3RhcmJ1Y2tzIGlzIHRyeWluZyB0byBjb252aW5jZSBpbnZlc3RvcnMgdG8ga2VlcCB0aGVpciBzdG9ja3MgaW4gdGhlIGNvbXBhbnkgYnV0IHRoaXMgcmVwb3J0IG1heSBjb252aW5jZSBpbnZlc3RvcnMgb3RoZXJ3aXNlLgoKCiMgSW1wcm92ZW1lbnQKT25lIHRoaW5nIEkgd2FzIG5vdCBhYmxlIHRvIGRvIGR1cmluZyB0aGlzIHByb2plY3QgYmVjYXVzZSBvZiBteSBsYWNrIG9mIHVuZGVyc3RhbmRpbmcgb2YgQ1BJIHdhcyB0byBhZGp1c3QgZm9yIGluZmxhdGlvbi4gTmV4dCB0aW1lIEkgdW5kZXJ0YWtlIGEgcHJvamVjdCBsaWtlIHRoaXMsIEkgd291bGQgbGlrZSB0byBmYWN0b3IgdGhpcyBpbiBhcyBJIHRoaW5rIG15IGFuYWx5c2lzIHdvdWxkIGJlIG11Y2ggbW9yZSBhY2N1cmF0ZSBhbmQgY2xvc2VyIHRvIHJlYWxpdHkuIAoKIyBSZWZlcmVuY2VzCuKAnFN0YXJidWNrcyBDb3Jwb3JhdGlvbiAoU0JVWCkgU3RvY2sgSGlzdG9yaWNhbCBQcmljZXMgJmFtcDsgRGF0YS7igJ0gWWFob28hIEZpbmFuY2UsIFlhaG9vISwgMTMgRGVjLiAyMDIzLCBmaW5hbmNlLnlhaG9vLmNvbS9xdW90ZS9TQlVYL2hpc3Rvcnk/cD1TQlVYLiAK