1 Introduction

In this project we want to find a time series with both trend and seasonality. We used a Seattle weather data set and are going to use the 160 most recent observations. We will look to fit three types of smoothing models. These models are simple exponential model, Holt models, and Holt-Winter’s models. We will also look at visualizations for this.

1.1 Data Description

  • Month- Month of the year (in numbers. Ex: 1-Jan, 2-Feb…)
  • Year- The year it was
  • LowTemp- Lowest temperature
  • HighTemp- Highest temperature
  • WarmestMin- The lowest warm temperature
  • ColdestHigh- The highest cold temperature
  • AveMin-The average minimum temperature
  • AveMax- The average maximum temperature
  • meanTemp- The mean temperature
  • TotPrecip- The total precipitation
  • TotSnow- The total snow
  • Max24hrPrecip- The maximum amount of precipitation in 24 hours

1.2 Research Question

The goal of the analysis is to use time series to make conclusions on the mean temperature in Seattle for the next year based on this data set.

1.3 Data turned into Smoothing Models

We want to look at only the most recent data and narrowed it down to the last 160 observations to be productive. We made a test and training data set then for the project. There is only 12 observations in the testing data set and 148 in the training set. We then wanted top put the data into smoothing models to then test them for accuracy measures later on.

2 Accuracy Measures for training data

We get many accuracy values to see which model is best for our data.

The accuracy measures of various exponential smoothing models based on the training data
ME RMSE MAE MPE MAPE MASE ACF1
SES 0.1088 5.5656 4.7334 -0.4109 9.1980 2.1660 0.5295
Holt Linear -0.0025 5.6668 4.8193 -0.6222 9.3625 2.2053 0.4889
Holt Add. Damped 0.0909 4.9184 3.9638 0.4974 7.7992 1.8138 -0.0326
Holt Exp. Damped -0.4527 4.8314 3.9869 -0.4701 7.7967 1.8244 -0.0588
HW Add. -0.0370 1.9630 1.5911 -0.2092 3.1935 0.7281 0.0465
HW Exp. -0.0228 2.0470 1.6313 -0.1951 3.2656 0.7465 0.1123
HW Add. Damp 0.0340 1.9434 1.5751 -0.0877 3.1666 0.7208 0.0594
HW Exp. Damp 0.0890 1.9721 1.5883 0.0096 3.1904 0.7268 0.0886

We see from the output above that HW Add Damp is the best fit for our data due to how it has the lowest average values of all the values above. For ACF1 it is a relatively low value meaning that residual autocorrelation is not a concern.

2.1 Visualization

We are now visualizing the original time series data and forecast results from different smoothing models

Case study: Comparing various exponential smoothing models.

Case study: Comparing various exponential smoothing models.

We see from the above visual that HW’s linear trend with an additive damped seasonal model is the best fit. This matches up with the accuracy measures we talked about before. It performed well due to how it does not have a wide prediction band.

3 Accuracy Measures for testing data

We saw that the visualization and our training data showed us that HW Add Damp is the appropriate model. To make a real life forecast of our data we need to use the whole data set with all 160 recent observations.

The accuracy measures of various exponential smoothing models based on the testing data
MSE MAPE
SES 249.974419 18.921862
Holt.Add 253.333843 19.054993
Holt.Add.Damp 324.332191 21.565641
Holt.Exp 353.076074 22.650636
HW.Add 6.124606 4.041155
HW.Exp 4.338878 3.463678
HW.Add.Damp 7.190380 4.315925
HW.Exp.Damp 7.956855 4.642423

Looking at the table we see that HW Add Damped is again the best model which is consistent to our conclusions made earlier in the project. HW.Exp has the lowest average values and HW.ADD.Damp has the second lowest. HW Add Damped is the best model that we have looked at due to MSE (585.04) and MAPE (40.44) both being one of the lowest average values out of the models. It also has the lowest average in the training data while HW.Exp has about the third best average for the training data. Which means that the HW.Add.Damped has the best all around accuracy measures for both the training anf testing data.

4 Final Model

Here we use the whole data set and refit it using smoothing parameters.

Estimated values of the smoothing parameters in Holt-Winters linear trend with additive seasonality
x
alpha 0.1555996
beta 0.0001000
gamma 0.0001000

The output above shows a value of 0.156 for alpha. This indicates that the model places moderate weight on our recent observations. For Gamma, 0.0001, since it is close to 0 it shows that seasonality patterns are stable. For Beta, 0.0001, since it is close to 0 it doesn’t have importance to updating the trend based on our recent observations.

5 Conclusion

The Seattle weather data set I used showed that the best model we could use was HW’s linear trend using an additive damped seasonal model. We got this by splitting our data into a training data set and testing data set. We also saw from the accuracy measures and the visuals that the additive damped model from HW was the best model. For our final model we got gamma and beta values close to 0 so that showed seasonality patterns were stable and we do not need to update the trend. Our alpha was 0.156 which showed that the model placed some weight on our recent observations.

This project we made sure to look at the most recent 160 observations due to needing more than 100. We then used a frequency of 12 for the 12 months of the year.

Our performance metrics of MSE and MAPE showed that their values were the lowest. This meant that our predictors are closer to the true values and that our model has the most accurate forecast.

LS0tCnRpdGxlOiAiU2VhdHRsZSBXZWF0aGVyIFRpbWUgU2VyaWVzIgphdXRob3I6ICJSeWFuIExlYm8iCmRhdGU6ICIyMDI0LTEyLTEwIgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgdG9jX2RlcHRoOiA0CiAgICB0b2NfZmxvYXQ6IHllcwogICAgZmlnX3dpZHRoOiA0CiAgICBmaWdfY2FwdGlvbjogeWVzCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcwogICAgdG9jX2NvbGxhcHNlZDogeWVzCiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUKICAgIGNvZGVfZG93bmxvYWQ6IHllcwogICAgc21vb3RoX3Njcm9sbDogeWVzCiAgICB0aGVtZTogbHVtZW4KICB3b3JkX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICAgIHRvY19kZXB0aDogNAogICAgZmlnX2NhcHRpb246IHllcwogICAga2VlcF9tZDogeWVzCiAgcGRmX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICAgIHRvY19kZXB0aDogNAogICAgZmlnX2NhcHRpb246IHllcwogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMKICAgIGZpZ193aWR0aDogMwogICAgZmlnX2hlaWdodDogMwplZGl0b3Jfb3B0aW9uczoKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCnNsd2F5c19hbGxvd19odG1sOiB0cnVlCi0tLQoKYGBgez1odG1sfQoKPHN0eWxlIHR5cGU9InRleHQvY3NzIj4KCi8qIENhc2NhZGluZyBTdHlsZSBTaGVldHMgKENTUykgaXMgYSBzdHlsZXNoZWV0IGxhbmd1YWdlIHVzZWQgdG8gZGVzY3JpYmUgdGhlIHByZXNlbnRhdGlvbiBvZiBhIGRvY3VtZW50IHdyaXR0ZW4gaW4gSFRNTCBvciBYTUwuIGl0IGlzIGEgc2ltcGxlIG1lY2hhbmlzbSBmb3IgYWRkaW5nIHN0eWxlIChlLmcuLCBmb250cywgY29sb3JzLCBzcGFjaW5nKSB0byBXZWIgZG9jdW1lbnRzLiAqLwoKaDEudGl0bGUgeyAgLyogVGl0bGUgLSBmb250IHNwZWNpZmljYXRpb25zIG9mIHRoZSByZXBvcnQgdGl0bGUgKi8KICBmb250LXNpemU6IDI0cHg7CiAgY29sb3I6IERhcmtSZWQ7CiAgdGV4dC1hbGlnbjogY2VudGVyOwogIGZvbnQtZmFtaWx5OiAiR2lsbCBTYW5zIiwgc2Fucy1zZXJpZjsKfQpoNC5hdXRob3IgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGF1dGhvcnMgICovCiAgZm9udC1zaXplOiAyMHB4OwogIGZvbnQtZmFtaWx5OiBzeXN0ZW0tdWk7CiAgY29sb3I6IERhcmtSZWQ7CiAgdGV4dC1hbGlnbjogY2VudGVyOwp9Cmg0LmRhdGUgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIHRoZSBkYXRlICAqLwogIGZvbnQtc2l6ZTogMThweDsKICBmb250LWZhbWlseTogc3lzdGVtLXVpOwogIGNvbG9yOiBEYXJrQmx1ZTsKICB0ZXh0LWFsaWduOiBjZW50ZXI7Cn0KaDEgeyAvKiBIZWFkZXIgMSAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGxldmVsIDEgc2VjdGlvbiB0aXRsZSAgKi8KICAgIGZvbnQtc2l6ZTogMjJweDsKICAgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOwogICAgY29sb3I6IG5hdnk7CiAgICB0ZXh0LWFsaWduOiBjZW50ZXI7Cn0KaDIgeyAvKiBIZWFkZXIgMiAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGxldmVsIDIgc2VjdGlvbiB0aXRsZSAqLwogICAgZm9udC1zaXplOiAyMHB4OwogICAgZm9udC1mYW1pbHk6ICJUaW1lcyBOZXcgUm9tYW4iLCBUaW1lcywgc2VyaWY7CiAgICBjb2xvcjogbmF2eTsKICAgIHRleHQtYWxpZ246IGxlZnQ7Cn0KCmgzIHsgLyogSGVhZGVyIDMgLSBmb250IHNwZWNpZmljYXRpb25zIG9mIGxldmVsIDMgc2VjdGlvbiB0aXRsZSAgKi8KICAgIGZvbnQtc2l6ZTogMThweDsKICAgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOwogICAgY29sb3I6IG5hdnk7CiAgICB0ZXh0LWFsaWduOiBsZWZ0Owp9CgpoNCB7IC8qIEhlYWRlciA0IC0gZm9udCBzcGVjaWZpY2F0aW9ucyBvZiBsZXZlbCA0IHNlY3Rpb24gdGl0bGUgICovCiAgICBmb250LXNpemU6IDE4cHg7CiAgICBmb250LWZhbWlseTogIlRpbWVzIE5ldyBSb21hbiIsIFRpbWVzLCBzZXJpZjsKICAgIGNvbG9yOiBkYXJrcmVkOwogICAgdGV4dC1hbGlnbjogbGVmdDsKfQoKYm9keSB7IGJhY2tncm91bmQtY29sb3I6d2hpdGU7IH0KCi5oaWdobGlnaHRtZSB7IGJhY2tncm91bmQtY29sb3I6eWVsbG93OyB9CgpwIHsgYmFja2dyb3VuZC1jb2xvcjp3aGl0ZTsgfQoKPC9zdHlsZT4KYGBgCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQojIERldGVjdCwgaW5zdGFsbCwgYW5kIGxvYWQgcGFja2FnZXMgaWYgbmVlZGVkLgppZiAoIXJlcXVpcmUoImtuaXRyIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygia25pdHIiKQogICBsaWJyYXJ5KGtuaXRyKQp9CmlmICghcmVxdWlyZSgibGVhZmxldCIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoImxlYWZsZXQiKQogICBsaWJyYXJ5KGxlYWZsZXQpCn0KaWYgKCFyZXF1aXJlKCJFbnZTdGF0cyIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoIkVudlN0YXRzIikKICAgbGlicmFyeShFbnZTdGF0cykKfQppZiAoIXJlcXVpcmUoIk1BU1MiKSkgewogICBpbnN0YWxsLnBhY2thZ2VzKCJNQVNTIikKICAgbGlicmFyeShNQVNTKQp9CmlmICghcmVxdWlyZSgicGh5dG9vbHMiKSkgewogICBpbnN0YWxsLnBhY2thZ2VzKCJwaHl0b29scyIpCiAgIGxpYnJhcnkocGh5dG9vbHMpCn0KaWYgKCFyZXF1aXJlKCJtbGJlbmNoIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygibWxiZW5jaCIpCiAgIGxpYnJhcnkobWxiZW5jaCkKfQppZiAoIXJlcXVpcmUoInBhbmRlciIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoInBhbmRlciIpCiAgIGxpYnJhcnkocGFuZGVyKQp9CmlmICghcmVxdWlyZSgiSVN3UiIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoIklTd1IiKQogICBsaWJyYXJ5KElTd1IpCn0KaWYgKCFyZXF1aXJlKCJnZ3Bsb3QyIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpCiAgIGxpYnJhcnkoZ2dwbG90MikKfQppZiAoIXJlcXVpcmUoImZvcmVjYXN0IikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygiZm9yZWNhc3QiKQogICBsaWJyYXJ5KGZvcmVjYXN0KQp9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gRkFMU0UsICAKICAgICAgICAgICAgICAgICAgIHdhcm5pbmcgPSBGQUxTRSwgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICBtZXNzYWdlID0gRkFMU0UsICAKICAgICAgICAgICAgICAgICAgIHJlc3VsdHMgPSBUUlVFLCAgCiAgICAgICAgICAgICAgICAgICBjb21tZW50ID0gRkFMU0UgICAKICAgICAgICAgICAgICAgICAgICAgICkgICAKYGBgCgpgYGB7cn0Kc193ZWF0aCA8LSByZWFkLmNzdigiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL1J5YW5MZWJvL1NUQS0zMjEvcmVmcy9oZWFkcy9tYWluL3NlYXR0bGVfd2VhdGhlci5jc3YiLCBoZWFkZXIgPSBUUlVFKQoKYGBgCgojIEludHJvZHVjdGlvbgpJbiB0aGlzIHByb2plY3Qgd2Ugd2FudCB0byBmaW5kIGEgdGltZSBzZXJpZXMgd2l0aCBib3RoIHRyZW5kIGFuZCBzZWFzb25hbGl0eS4gV2UgdXNlZCBhIFNlYXR0bGUgd2VhdGhlciBkYXRhIHNldCBhbmQgYXJlIGdvaW5nIHRvIHVzZSB0aGUgMTYwIG1vc3QgcmVjZW50IG9ic2VydmF0aW9ucy4gV2Ugd2lsbCBsb29rIHRvIGZpdCB0aHJlZSB0eXBlcyBvZiBzbW9vdGhpbmcgbW9kZWxzLiBUaGVzZSBtb2RlbHMgYXJlIHNpbXBsZSBleHBvbmVudGlhbCBtb2RlbCwgSG9sdCBtb2RlbHMsIGFuZCAgSG9sdC1XaW50ZXLigJlzIG1vZGVscy4gV2Ugd2lsbCBhbHNvIGxvb2sgYXQgdmlzdWFsaXphdGlvbnMgZm9yIHRoaXMuCgoKCgojIyBEYXRhIERlc2NyaXB0aW9uCgoqIE1vbnRoLSBNb250aCBvZiB0aGUgeWVhciAoaW4gbnVtYmVycy4gRXg6IDEtSmFuLCAyLUZlYi4uLikKKiBZZWFyLSBUaGUgeWVhciBpdCB3YXMKKiBMb3dUZW1wLSBMb3dlc3QgdGVtcGVyYXR1cmUgCiogSGlnaFRlbXAtIEhpZ2hlc3QgdGVtcGVyYXR1cmUgCiogV2FybWVzdE1pbi0gVGhlIGxvd2VzdCB3YXJtIHRlbXBlcmF0dXJlIAoqIENvbGRlc3RIaWdoLSBUaGUgaGlnaGVzdCBjb2xkIHRlbXBlcmF0dXJlIAoqIEF2ZU1pbi1UaGUgYXZlcmFnZSBtaW5pbXVtIHRlbXBlcmF0dXJlCiogQXZlTWF4LSBUaGUgYXZlcmFnZSBtYXhpbXVtIHRlbXBlcmF0dXJlCiogbWVhblRlbXAtIFRoZSBtZWFuIHRlbXBlcmF0dXJlCiogVG90UHJlY2lwLSBUaGUgdG90YWwgcHJlY2lwaXRhdGlvbgoqIFRvdFNub3ctIFRoZSB0b3RhbCBzbm93CiogTWF4MjRoclByZWNpcC0gVGhlIG1heGltdW0gYW1vdW50IG9mIHByZWNpcGl0YXRpb24gaW4gMjQgaG91cnMKCiMjIFJlc2VhcmNoIFF1ZXN0aW9uCgpUaGUgZ29hbCBvZiB0aGUgYW5hbHlzaXMgaXMgdG8gdXNlIHRpbWUgc2VyaWVzIHRvIG1ha2UgY29uY2x1c2lvbnMgb24gdGhlIG1lYW4gdGVtcGVyYXR1cmUgaW4gU2VhdHRsZSBmb3IgdGhlIG5leHQgeWVhciBiYXNlZCBvbiB0aGlzIGRhdGEgc2V0LgoKIyMgRGF0YSB0dXJuZWQgaW50byBTbW9vdGhpbmcgTW9kZWxzCgpgYGB7cn0Kc3ViX3dlYXRoIDwtIHRhaWwoc193ZWF0aCwgMTYwKQp0cmFpbi53ZWF0aCA9IHN1Yl93ZWF0aCRtZWFuVGVtcFsxOjE0OF0KdGVzdC5zZWEgPSBzdWJfd2VhdGgkbWVhblRlbXBbMTQ5OjE2MF0KCnNlYS53ZWF0aCA9IHRzKHRyYWluLndlYXRoLCBzdGFydD1jKDIwMDAsIDEpLCBmcmVxdWVuY3k9MTIpCgpmaXQxID0gc2VzKHNlYS53ZWF0aCwgaD0xMikKZml0MiA9IGhvbHQoc2VhLndlYXRoLCBpbml0aWFsPSJvcHRpbWFsIiwgaD0xMikKZml0MyA9IGhvbHQoc2VhLndlYXRoLCBkYW1wZWQ9VFJVRSwgaD0xMikKZml0NCA9IGhvbHQoc2VhLndlYXRoLCBleHBvbmVudGlhbD1UUlVFLCBkYW1wZWQ9VFJVRSwgaD0xMikKZml0NSA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9ImFkZGl0aXZlIikKZml0NiA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9Im11bHRpcGxpY2F0aXZlIikKZml0NyA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9ImFkZGl0aXZlIiwgZGFtcGVkPVRSVUUpCmZpdDggPSBodyhzZWEud2VhdGgsIGg9MTIsIHNlYXNvbmFsPSJtdWx0aXBsaWNhdGl2ZSIsIGRhbXBlZD1UUlVFKQoKCgpgYGAKCldlIHdhbnQgdG8gbG9vayBhdCBvbmx5IHRoZSBtb3N0IHJlY2VudCBkYXRhIGFuZCBuYXJyb3dlZCBpdCBkb3duIHRvIHRoZSBsYXN0IDE2MCBvYnNlcnZhdGlvbnMgdG8gYmUgcHJvZHVjdGl2ZS4gV2UgbWFkZSBhIHRlc3QgYW5kIHRyYWluaW5nIGRhdGEgc2V0IHRoZW4gZm9yIHRoZSBwcm9qZWN0LiBUaGVyZSBpcyBvbmx5IDEyIG9ic2VydmF0aW9ucyBpbiB0aGUgdGVzdGluZyBkYXRhIHNldCBhbmQgMTQ4IGluIHRoZSB0cmFpbmluZyBzZXQuIFdlIHRoZW4gd2FudGVkIHRvcCBwdXQgdGhlIGRhdGEgaW50byBzbW9vdGhpbmcgbW9kZWxzIHRvIHRoZW4gdGVzdCB0aGVtIGZvciBhY2N1cmFjeSBtZWFzdXJlcyBsYXRlciBvbi4gCgoKCiMgQWNjdXJhY3kgTWVhc3VyZXMgZm9yIHRyYWluaW5nIGRhdGEKCldlIGdldCBtYW55IGFjY3VyYWN5IHZhbHVlcyB0byBzZWUgd2hpY2ggbW9kZWwgaXMgYmVzdCBmb3Igb3VyIGRhdGEuIAoKYGBge3J9CmFjY3VyYWN5LnRhYmxlID0gcm91bmQocmJpbmQoYWNjdXJhY3koZml0MSksIGFjY3VyYWN5KGZpdDIpLCBhY2N1cmFjeShmaXQzKSwgYWNjdXJhY3koZml0NCksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWNjdXJhY3koZml0NSksIGFjY3VyYWN5KGZpdDYpLCBhY2N1cmFjeShmaXQ3KSwgYWNjdXJhY3koZml0OCkpLDQpCnJvdy5uYW1lcyhhY2N1cmFjeS50YWJsZSk9YygiU0VTIiwiSG9sdCBMaW5lYXIiLCJIb2x0IEFkZC4gRGFtcGVkIiwgIkhvbHQgRXhwLiBEYW1wZWQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIkhXIEFkZC4iLCJIVyBFeHAuIiwiSFcgQWRkLiBEYW1wIiwgIkhXIEV4cC4gRGFtcCIpCmthYmxlKGFjY3VyYWN5LnRhYmxlLCBjYXB0aW9uID0gIlRoZSBhY2N1cmFjeSBtZWFzdXJlcyBvZiB2YXJpb3VzIGV4cG9uZW50aWFsIHNtb290aGluZyBtb2RlbHMgCiAgICAgIGJhc2VkIG9uIHRoZSB0cmFpbmluZyBkYXRhIikKCmBgYAoKV2Ugc2VlIGZyb20gdGhlIG91dHB1dCBhYm92ZSB0aGF0IEhXIEFkZCBEYW1wIGlzIHRoZSBiZXN0IGZpdCBmb3Igb3VyIGRhdGEgZHVlIHRvIGhvdyBpdCBoYXMgdGhlIGxvd2VzdCBhdmVyYWdlIHZhbHVlcyBvZiBhbGwgdGhlIHZhbHVlcyBhYm92ZS4gRm9yIEFDRjEgaXQgaXMgYSByZWxhdGl2ZWx5IGxvdyB2YWx1ZSBtZWFuaW5nIHRoYXQgcmVzaWR1YWwgYXV0b2NvcnJlbGF0aW9uIGlzIG5vdCBhIGNvbmNlcm4uIAoKCiMjIFZpc3VhbGl6YXRpb24KCldlIGFyZSBub3cgdmlzdWFsaXppbmcgdGhlIG9yaWdpbmFsIHRpbWUgc2VyaWVzIGRhdGEgYW5kIGZvcmVjYXN0IHJlc3VsdHMgZnJvbSBkaWZmZXJlbnQgc21vb3RoaW5nIG1vZGVscwoKYGBge3IgZmlnLmFsaWduPSdjZW50ZXInLGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTYuNSwgZmlnLmNhcD0iQ2FzZSBzdHVkeTogQ29tcGFyaW5nIHZhcmlvdXMgZXhwb25lbnRpYWwgc21vb3RoaW5nIG1vZGVscy4ifQpwYXIobWZyb3c9YygyLDEpLCBtYXI9YygzLDQsMywxKSkKCnByZWQuaWQgPSAxNDk6MTYwCnBsb3QoMToxNDgsIHRyYWluLndlYXRoLCBsd2Q9Mix0eXBlPSJvIiwgeWxhYj0iTWVhblRlbXAiLCB4bGFiPSIiLCAKICAgICB4bGltPWMoMSwxNzApLCB5bGltPWMoMzAsIDkwKSwgY2V4PTAuMywKICAgICBtYWluPSJOb24tc2Vhc29uYWwgU21vb3RoaW5nIE1vZGVscyIpCmxpbmVzKHByZWQuaWQsIGZpdDEkbWVhbiwgY29sPSJyZWQiKQpsaW5lcyhwcmVkLmlkLCBmaXQyJG1lYW4sIGNvbD0iYmx1ZSIpCmxpbmVzKHByZWQuaWQsIGZpdDMkbWVhbiwgY29sPSJwdXJwbGUiKQpsaW5lcyhwcmVkLmlkLCBmaXQ0JG1lYW4sIGNvbD0ibmF2eSIpCiMjCnBvaW50cyhwcmVkLmlkLCBmaXQxJG1lYW4sIHBjaD0xNiwgY29sPSJyZWQiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQyJG1lYW4sIHBjaD0xNywgY29sPSJibHVlIiwgY2V4ID0gMC41KQpwb2ludHMocHJlZC5pZCwgZml0MyRtZWFuLCBwY2g9MTksIGNvbD0icHVycGxlIiwgY2V4ID0gMC41KQpwb2ludHMocHJlZC5pZCwgZml0NCRtZWFuLCBwY2g9MjEsIGNvbD0ibmF2eSIsIGNleCA9IDAuNSkKI3BvaW50cyhmaXQwLCBjb2w9ImJsYWNrIiwgcGNoPTEpCmxlZ2VuZCgiYm90dG9tcmlnaHQiLCBsdHk9MSwgY29sPWMoInJlZCIsImJsdWUiLCJwdXJwbGUiLCAibmF2eSIpLHBjaD1jKDE2LDE3LDE5LDIxKSwKICAgYygiU0VTIiwiSG9sdCBMaW5lYXIiLCJIb2x0IExpbmVhciBEYW1wZWQiLCAiSG9sdCBNdWx0aXBsaWNhdGl2ZSBEYW1wZWQiKSwgCiAgIGNleCA9IDAuNywgYnR5PSJuIikKCnBsb3QoMToxNDgsIHRyYWluLndlYXRoLCBsd2Q9Mix0eXBlPSJvIiwgeWxhYj0iTWVhblRlbXAiLCB4bGFiPSIiLCAKICAgICB4bGltPWMoMSwxNzApLCB5bGltPWMoMzAsIDkwKSwgY2V4PTAuMywKICAgICBtYWluPSJIb2x0LVdpbnRlcmQgVHJlbmQgYW5kIFNlYXNvbmFsIFNtb290aGluZyBNb2RlbHMiKQpsaW5lcyhwcmVkLmlkLCBmaXQ1JG1lYW4sIGNvbD0icmVkIikKbGluZXMocHJlZC5pZCwgZml0NiRtZWFuLCBjb2w9ImJsdWUiKQpsaW5lcyhwcmVkLmlkLCBmaXQ3JG1lYW4sIGNvbD0icHVycGxlIikKbGluZXMocHJlZC5pZCwgZml0OCRtZWFuLCBjb2w9Im5hdnkiKQoKcG9pbnRzKHByZWQuaWQsIGZpdDUkbWVhbiwgcGNoPTE2LCBjb2w9InJlZCIsIGNleCA9IDAuNSkKcG9pbnRzKHByZWQuaWQsIGZpdDYkbWVhbiwgcGNoPTE3LCBjb2w9ImJsdWUiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQ3JG1lYW4sIHBjaD0xOSwgY29sPSJwdXJwbGUiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQ4JG1lYW4sIHBjaD0yMSwgY29sPSJuYXZ5IiwgY2V4ID0gMC41KQojIyMKbGVnZW5kKCJib3R0b21yaWdodCIsIGx0eT0xLCBjb2w9YygicmVkIiwiYmx1ZSIsInB1cnBsZSIsICJuYXZ5IikscGNoPWMoMTYsMTcsMTksMjEpLAogICBjKCJIVyBBZGRpdGl2ZSIsIkhXIE11bHRpcGxpY2F0aXZlIiwiSFcgQWRkaXRpdmUgRGFtcGVkIiwgIkhXIE11bHRpcGxpY2F0aXZlIERhbXBlZCIpLCAKICAgY2V4ID0gMC43LCBidHk9Im4iKQoKYGBgCgpXZSBzZWUgZnJvbSB0aGUgYWJvdmUgdmlzdWFsIHRoYXQgSFfigJlzIGxpbmVhciB0cmVuZCB3aXRoIGFuIGFkZGl0aXZlIGRhbXBlZCBzZWFzb25hbCBtb2RlbCBpcyB0aGUgYmVzdCBmaXQuIFRoaXMgbWF0Y2hlcyB1cCB3aXRoIHRoZSBhY2N1cmFjeSBtZWFzdXJlcyB3ZSB0YWxrZWQgYWJvdXQgYmVmb3JlLiBJdCBwZXJmb3JtZWQgd2VsbCBkdWUgdG8gaG93IGl0IGRvZXMgbm90IGhhdmUgYSB3aWRlIHByZWRpY3Rpb24gYmFuZC4gCgojIEFjY3VyYWN5IE1lYXN1cmVzIGZvciB0ZXN0aW5nIGRhdGEKCldlIHNhdyB0aGF0IHRoZSB2aXN1YWxpemF0aW9uIGFuZCBvdXIgdHJhaW5pbmcgZGF0YSBzaG93ZWQgdXMgdGhhdCBIVyBBZGQgRGFtcCBpcyB0aGUgYXBwcm9wcmlhdGUgbW9kZWwuIFRvIG1ha2UgYSByZWFsIGxpZmUgZm9yZWNhc3Qgb2Ygb3VyIGRhdGEgd2UgbmVlZCB0byB1c2UgdGhlIHdob2xlIGRhdGEgc2V0IHdpdGggYWxsIDE2MCByZWNlbnQgb2JzZXJ2YXRpb25zLiAKCmBgYHtyfQoKYWNjLmZ1biA9IGZ1bmN0aW9uKHRlc3QuZGF0YSwgbW9kLm9iail7CiAgUEU9MTAwKih0ZXN0LmRhdGEtbW9kLm9iaiRtZWFuKS9tb2Qub2JqJG1lYW4KICBNQVBFID0gbWVhbihhYnMoUEUpKQogICMjIwogIEU9dGVzdC5kYXRhLW1vZC5vYmokbWVhbgogIE1TRT1tZWFuKEVeMikKICAjIyMKICBhY2N1cmFjeS5tZXRyaWM9YyhNU0U9TVNFLCBNQVBFPU1BUEUpCiAgYWNjdXJhY3kubWV0cmljCiAgCn0KCgpgYGAKCmBgYHtyfQpwcmVkLmFjY3VyYWN5ID0gcmJpbmQoU0VTID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQxKSwKICAgICAgICAgICAgICAgICAgICAgIEhvbHQuQWRkID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQyKSwKICAgICAgICAgICAgICAgICAgICAgIEhvbHQuQWRkLkRhbXAgPWFjYy5mdW4odGVzdC5kYXRhPXRlc3Quc2VhLCBtb2Qub2JqPWZpdDMpLAogICAgICAgICAgICAgICAgICAgICAgSG9sdC5FeHAgPWFjYy5mdW4odGVzdC5kYXRhPXRlc3Quc2VhLCBtb2Qub2JqPWZpdDQpLAogICAgICAgICAgICAgICAgICAgICAgSFcuQWRkID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQ1KSwKICAgICAgICAgICAgICAgICAgICAgIEhXLkV4cCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0NiksCiAgICAgICAgICAgICAgICAgICAgICBIVy5BZGQuRGFtcCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0NyksCiAgICAgICAgICAgICAgICAgICAgICBIVy5FeHAuRGFtcCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0OCkpCmthYmxlKHByZWQuYWNjdXJhY3ksIGNhcHRpb249IlRoZSBhY2N1cmFjeSBtZWFzdXJlcyBvZiB2YXJpb3VzIGV4cG9uZW50aWFsIHNtb290aGluZyBtb2RlbHMgCiAgICAgIGJhc2VkIG9uIHRoZSB0ZXN0aW5nIGRhdGEiKQoKYGBgCgpMb29raW5nIGF0IHRoZSB0YWJsZSB3ZSBzZWUgdGhhdCBIVyBBZGQgRGFtcGVkIGlzIGFnYWluIHRoZSBiZXN0IG1vZGVsIHdoaWNoIGlzIGNvbnNpc3RlbnQgdG8gb3VyIGNvbmNsdXNpb25zIG1hZGUgZWFybGllciBpbiB0aGUgcHJvamVjdC4gSFcuRXhwIGhhcyB0aGUgbG93ZXN0IGF2ZXJhZ2UgdmFsdWVzIGFuZCBIVy5BREQuRGFtcCBoYXMgdGhlIHNlY29uZCBsb3dlc3QuIEhXIEFkZCBEYW1wZWQgaXMgdGhlIGJlc3QgbW9kZWwgdGhhdCB3ZSBoYXZlIGxvb2tlZCBhdCBkdWUgdG8gTVNFICg1ODUuMDQpIGFuZCBNQVBFICg0MC40NCkgYm90aCBiZWluZyBvbmUgb2YgdGhlIGxvd2VzdCBhdmVyYWdlIHZhbHVlcyBvdXQgb2YgdGhlIG1vZGVscy4gSXQgYWxzbyBoYXMgdGhlIGxvd2VzdCBhdmVyYWdlIGluIHRoZSB0cmFpbmluZyBkYXRhIHdoaWxlIEhXLkV4cCBoYXMgYWJvdXQgdGhlIHRoaXJkIGJlc3QgYXZlcmFnZSBmb3IgdGhlIHRyYWluaW5nIGRhdGEuIFdoaWNoIG1lYW5zIHRoYXQgdGhlICBIVy5BZGQuRGFtcGVkIGhhcyB0aGUgYmVzdCBhbGwgYXJvdW5kIGFjY3VyYWN5IG1lYXN1cmVzIGZvciBib3RoIHRoZSB0cmFpbmluZyBhbmYgdGVzdGluZyBkYXRhLgoKIyBGaW5hbCBNb2RlbAoKSGVyZSB3ZSB1c2UgdGhlIHdob2xlIGRhdGEgc2V0IGFuZCByZWZpdCBpdCB1c2luZyBzbW9vdGhpbmcgcGFyYW1ldGVycy4KCmBgYHtyfQpzX3dlYXRoIDwtIHJlYWQuY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vUnlhbkxlYm8vU1RBLTMyMS9yZWZzL2hlYWRzL21haW4vc2VhdHRsZV93ZWF0aGVyLmNzdiIsIGhlYWRlciA9IFRSVUUpCnNlYXR0bGV3ZWF0aD10cyhzX3dlYXRoJG1lYW5UZW1wWzE6MTYwXSwgc3RhcnQ9MjAwMCwgZnJlcXVlbmN5ID0gMTIpCmZpbmFsLm1vZGVsID0gaHcoc2VhdHRsZXdlYXRoLGg9MTIsIHNlYXNvbmFsPSJhZGRpdGl2ZSIpIApzbW9vdGhpbmcucGFyYW1ldGVyID0gZmluYWwubW9kZWwkbW9kZWwkcGFyWzE6M10Ka2FibGUoc21vb3RoaW5nLnBhcmFtZXRlciwgY2FwdGlvbj0iRXN0aW1hdGVkIHZhbHVlcyBvZiB0aGUgc21vb3RoaW5nIHBhcmFtZXRlcnMgaW4KICAgICAgSG9sdC1XaW50ZXJzIGxpbmVhciB0cmVuZCB3aXRoIGFkZGl0aXZlIHNlYXNvbmFsaXR5IikKCmBgYAoKVGhlIG91dHB1dCBhYm92ZSBzaG93cyBhIHZhbHVlIG9mIDAuMTU2IGZvciBhbHBoYS4gVGhpcyBpbmRpY2F0ZXMgdGhhdCB0aGUgbW9kZWwgcGxhY2VzIG1vZGVyYXRlIHdlaWdodCBvbiBvdXIgcmVjZW50IG9ic2VydmF0aW9ucy4gIEZvciBHYW1tYSwgMC4wMDAxLCBzaW5jZSBpdCBpcyBjbG9zZSB0byAwIGl0IHNob3dzIHRoYXQgc2Vhc29uYWxpdHkgcGF0dGVybnMgYXJlIHN0YWJsZS4gRm9yIEJldGEsIDAuMDAwMSwgc2luY2UgaXQgaXMgY2xvc2UgdG8gMCBpdCBkb2Vzbid0IGhhdmUgaW1wb3J0YW5jZSB0byB1cGRhdGluZyB0aGUgdHJlbmQgYmFzZWQgb24gb3VyIHJlY2VudCBvYnNlcnZhdGlvbnMuCgoKIyBDb25jbHVzaW9uCgpUaGUgU2VhdHRsZSB3ZWF0aGVyIGRhdGEgc2V0IEkgdXNlZCBzaG93ZWQgdGhhdCB0aGUgYmVzdCBtb2RlbCB3ZSBjb3VsZCB1c2Ugd2FzIEhXJ3MgbGluZWFyIHRyZW5kIHVzaW5nIGFuIGFkZGl0aXZlIGRhbXBlZCBzZWFzb25hbCBtb2RlbC4gV2UgZ290IHRoaXMgYnkgc3BsaXR0aW5nIG91ciBkYXRhIGludG8gYSB0cmFpbmluZyBkYXRhIHNldCBhbmQgdGVzdGluZyBkYXRhIHNldC4gV2UgYWxzbyBzYXcgZnJvbSB0aGUgYWNjdXJhY3kgbWVhc3VyZXMgYW5kIHRoZSB2aXN1YWxzIHRoYXQgdGhlIGFkZGl0aXZlIGRhbXBlZCBtb2RlbCBmcm9tIEhXIHdhcyB0aGUgYmVzdCBtb2RlbC4gRm9yIG91ciBmaW5hbCBtb2RlbCB3ZSBnb3QgZ2FtbWEgYW5kIGJldGEgdmFsdWVzIGNsb3NlIHRvIDAgc28gdGhhdCBzaG93ZWQgc2Vhc29uYWxpdHkgcGF0dGVybnMgd2VyZSBzdGFibGUgYW5kIHdlIGRvIG5vdCBuZWVkIHRvIHVwZGF0ZSB0aGUgdHJlbmQuIE91ciBhbHBoYSB3YXMgMC4xNTYgd2hpY2ggc2hvd2VkIHRoYXQgdGhlIG1vZGVsIHBsYWNlZCBzb21lIHdlaWdodCBvbiBvdXIgcmVjZW50IG9ic2VydmF0aW9ucy4KClRoaXMgcHJvamVjdCB3ZSBtYWRlIHN1cmUgdG8gbG9vayBhdCB0aGUgbW9zdCByZWNlbnQgMTYwIG9ic2VydmF0aW9ucyBkdWUgdG8gbmVlZGluZyBtb3JlIHRoYW4gMTAwLiBXZSB0aGVuIHVzZWQgYSBmcmVxdWVuY3kgb2YgMTIgZm9yIHRoZSAxMiBtb250aHMgb2YgdGhlIHllYXIuCgpPdXIgcGVyZm9ybWFuY2UgbWV0cmljcyBvZiBNU0UgYW5kIE1BUEUgc2hvd2VkIHRoYXQgdGhlaXIgdmFsdWVzIHdlcmUgdGhlIGxvd2VzdC4gVGhpcyBtZWFudCB0aGF0IG91ciBwcmVkaWN0b3JzIGFyZSBjbG9zZXIgdG8gdGhlIHRydWUgdmFsdWVzIGFuZCB0aGF0IG91ciBtb2RlbCBoYXMgdGhlIG1vc3QgYWNjdXJhdGUgZm9yZWNhc3QuICAgICAgICAKCgoKCgoK