Introduction
In this project we want to find a time series with both trend and
seasonality. We used a Seattle weather data set and are going to use the
160 most recent observations. We will look to fit three types of
smoothing models. These models are simple exponential model, Holt
models, and Holt-Winter’s models. We will also look at visualizations
for this.
Data Description
- Month- Month of the year (in numbers. Ex: 1-Jan, 2-Feb…)
- Year- The year it was
- LowTemp- Lowest temperature
- HighTemp- Highest temperature
- WarmestMin- The lowest warm temperature
- ColdestHigh- The highest cold temperature
- AveMin-The average minimum temperature
- AveMax- The average maximum temperature
- meanTemp- The mean temperature
- TotPrecip- The total precipitation
- TotSnow- The total snow
- Max24hrPrecip- The maximum amount of precipitation in 24 hours
Research
Question
The goal of the analysis is to use time series to make conclusions on
the mean temperature in Seattle for the next year based on this data
set.
Data turned into
Smoothing Models
We want to look at only the most recent data and narrowed it down to
the last 160 observations to be productive. We made a test and training
data set then for the project. There is only 12 observations in the
testing data set and 148 in the training set. We then wanted top put the
data into smoothing models to then test them for accuracy measures later
on.
Accuracy Measures for
training data
We get many accuracy values to see which model is best for our
data.
The accuracy measures of various exponential smoothing models
based on the training data
| SES |
0.1088 |
5.5656 |
4.7334 |
-0.4109 |
9.1980 |
2.1660 |
0.5295 |
| Holt Linear |
-0.0025 |
5.6668 |
4.8193 |
-0.6222 |
9.3625 |
2.2053 |
0.4889 |
| Holt Add. Damped |
0.0909 |
4.9184 |
3.9638 |
0.4974 |
7.7992 |
1.8138 |
-0.0326 |
| Holt Exp. Damped |
-0.4527 |
4.8314 |
3.9869 |
-0.4701 |
7.7967 |
1.8244 |
-0.0588 |
| HW Add. |
-0.0370 |
1.9630 |
1.5911 |
-0.2092 |
3.1935 |
0.7281 |
0.0465 |
| HW Exp. |
-0.0228 |
2.0470 |
1.6313 |
-0.1951 |
3.2656 |
0.7465 |
0.1123 |
| HW Add. Damp |
0.0340 |
1.9434 |
1.5751 |
-0.0877 |
3.1666 |
0.7208 |
0.0594 |
| HW Exp. Damp |
0.0890 |
1.9721 |
1.5883 |
0.0096 |
3.1904 |
0.7268 |
0.0886 |
We see from the output above that HW Add Damp is the best fit for our
data due to how it has the lowest average values of all the values
above. For ACF1 it is a relatively low value meaning that residual
autocorrelation is not a concern.
Visualization
We are now visualizing the original time series data and forecast
results from different smoothing models
We see from the above visual that HW’s linear trend with an additive
damped seasonal model is the best fit. This matches up with the accuracy
measures we talked about before. It performed well due to how it does
not have a wide prediction band.
Accuracy Measures for
testing data
We saw that the visualization and our training data showed us that HW
Add Damp is the appropriate model. To make a real life forecast of our
data we need to use the whole data set with all 160 recent
observations.
The accuracy measures of various exponential smoothing models
based on the testing data
| SES |
249.974419 |
18.921862 |
| Holt.Add |
253.333843 |
19.054993 |
| Holt.Add.Damp |
324.332191 |
21.565641 |
| Holt.Exp |
353.076074 |
22.650636 |
| HW.Add |
6.124606 |
4.041155 |
| HW.Exp |
4.338878 |
3.463678 |
| HW.Add.Damp |
7.190380 |
4.315925 |
| HW.Exp.Damp |
7.956855 |
4.642423 |
Looking at the table we see that HW Add Damped is again the best
model which is consistent to our conclusions made earlier in the
project. HW.Exp has the lowest average values and HW.ADD.Damp has the
second lowest. HW Add Damped is the best model that we have looked at
due to MSE (585.04) and MAPE (40.44) both being one of the lowest
average values out of the models. It also has the lowest average in the
training data while HW.Exp has about the third best average for the
training data. Which means that the HW.Add.Damped has the best all
around accuracy measures for both the training anf testing data.
Final Model
Here we use the whole data set and refit it using smoothing
parameters.
Estimated values of the smoothing parameters in Holt-Winters
linear trend with additive seasonality
| alpha |
0.1555996 |
| beta |
0.0001000 |
| gamma |
0.0001000 |
The output above shows a value of 0.156 for alpha. This indicates
that the model places moderate weight on our recent observations. For
Gamma, 0.0001, since it is close to 0 it shows that seasonality patterns
are stable. For Beta, 0.0001, since it is close to 0 it doesn’t have
importance to updating the trend based on our recent observations.
Conclusion
The Seattle weather data set I used showed that the best model we
could use was HW’s linear trend using an additive damped seasonal model.
We got this by splitting our data into a training data set and testing
data set. We also saw from the accuracy measures and the visuals that
the additive damped model from HW was the best model. For our final
model we got gamma and beta values close to 0 so that showed seasonality
patterns were stable and we do not need to update the trend. Our alpha
was 0.156 which showed that the model placed some weight on our recent
observations.
This project we made sure to look at the most recent 160 observations
due to needing more than 100. We then used a frequency of 12 for the 12
months of the year.
Our performance metrics of MSE and MAPE showed that their values were
the lowest. This meant that our predictors are closer to the true values
and that our model has the most accurate forecast.
LS0tCnRpdGxlOiAiU2VhdHRsZSBXZWF0aGVyIFRpbWUgU2VyaWVzIgphdXRob3I6ICJSeWFuIExlYm8iCmRhdGU6ICIyMDI0LTEyLTEwIgpvdXRwdXQ6IAogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgdG9jX2RlcHRoOiA0CiAgICB0b2NfZmxvYXQ6IHllcwogICAgZmlnX3dpZHRoOiA0CiAgICBmaWdfY2FwdGlvbjogeWVzCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcwogICAgdG9jX2NvbGxhcHNlZDogeWVzCiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUKICAgIGNvZGVfZG93bmxvYWQ6IHllcwogICAgc21vb3RoX3Njcm9sbDogeWVzCiAgICB0aGVtZTogbHVtZW4KICB3b3JkX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICAgIHRvY19kZXB0aDogNAogICAgZmlnX2NhcHRpb246IHllcwogICAga2VlcF9tZDogeWVzCiAgcGRmX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICAgIHRvY19kZXB0aDogNAogICAgZmlnX2NhcHRpb246IHllcwogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMKICAgIGZpZ193aWR0aDogMwogICAgZmlnX2hlaWdodDogMwplZGl0b3Jfb3B0aW9uczoKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCnNsd2F5c19hbGxvd19odG1sOiB0cnVlCi0tLQoKYGBgez1odG1sfQoKPHN0eWxlIHR5cGU9InRleHQvY3NzIj4KCi8qIENhc2NhZGluZyBTdHlsZSBTaGVldHMgKENTUykgaXMgYSBzdHlsZXNoZWV0IGxhbmd1YWdlIHVzZWQgdG8gZGVzY3JpYmUgdGhlIHByZXNlbnRhdGlvbiBvZiBhIGRvY3VtZW50IHdyaXR0ZW4gaW4gSFRNTCBvciBYTUwuIGl0IGlzIGEgc2ltcGxlIG1lY2hhbmlzbSBmb3IgYWRkaW5nIHN0eWxlIChlLmcuLCBmb250cywgY29sb3JzLCBzcGFjaW5nKSB0byBXZWIgZG9jdW1lbnRzLiAqLwoKaDEudGl0bGUgeyAgLyogVGl0bGUgLSBmb250IHNwZWNpZmljYXRpb25zIG9mIHRoZSByZXBvcnQgdGl0bGUgKi8KICBmb250LXNpemU6IDI0cHg7CiAgY29sb3I6IERhcmtSZWQ7CiAgdGV4dC1hbGlnbjogY2VudGVyOwogIGZvbnQtZmFtaWx5OiAiR2lsbCBTYW5zIiwgc2Fucy1zZXJpZjsKfQpoNC5hdXRob3IgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGF1dGhvcnMgICovCiAgZm9udC1zaXplOiAyMHB4OwogIGZvbnQtZmFtaWx5OiBzeXN0ZW0tdWk7CiAgY29sb3I6IERhcmtSZWQ7CiAgdGV4dC1hbGlnbjogY2VudGVyOwp9Cmg0LmRhdGUgeyAvKiBIZWFkZXIgNCAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIHRoZSBkYXRlICAqLwogIGZvbnQtc2l6ZTogMThweDsKICBmb250LWZhbWlseTogc3lzdGVtLXVpOwogIGNvbG9yOiBEYXJrQmx1ZTsKICB0ZXh0LWFsaWduOiBjZW50ZXI7Cn0KaDEgeyAvKiBIZWFkZXIgMSAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGxldmVsIDEgc2VjdGlvbiB0aXRsZSAgKi8KICAgIGZvbnQtc2l6ZTogMjJweDsKICAgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOwogICAgY29sb3I6IG5hdnk7CiAgICB0ZXh0LWFsaWduOiBjZW50ZXI7Cn0KaDIgeyAvKiBIZWFkZXIgMiAtIGZvbnQgc3BlY2lmaWNhdGlvbnMgZm9yIGxldmVsIDIgc2VjdGlvbiB0aXRsZSAqLwogICAgZm9udC1zaXplOiAyMHB4OwogICAgZm9udC1mYW1pbHk6ICJUaW1lcyBOZXcgUm9tYW4iLCBUaW1lcywgc2VyaWY7CiAgICBjb2xvcjogbmF2eTsKICAgIHRleHQtYWxpZ246IGxlZnQ7Cn0KCmgzIHsgLyogSGVhZGVyIDMgLSBmb250IHNwZWNpZmljYXRpb25zIG9mIGxldmVsIDMgc2VjdGlvbiB0aXRsZSAgKi8KICAgIGZvbnQtc2l6ZTogMThweDsKICAgIGZvbnQtZmFtaWx5OiAiVGltZXMgTmV3IFJvbWFuIiwgVGltZXMsIHNlcmlmOwogICAgY29sb3I6IG5hdnk7CiAgICB0ZXh0LWFsaWduOiBsZWZ0Owp9CgpoNCB7IC8qIEhlYWRlciA0IC0gZm9udCBzcGVjaWZpY2F0aW9ucyBvZiBsZXZlbCA0IHNlY3Rpb24gdGl0bGUgICovCiAgICBmb250LXNpemU6IDE4cHg7CiAgICBmb250LWZhbWlseTogIlRpbWVzIE5ldyBSb21hbiIsIFRpbWVzLCBzZXJpZjsKICAgIGNvbG9yOiBkYXJrcmVkOwogICAgdGV4dC1hbGlnbjogbGVmdDsKfQoKYm9keSB7IGJhY2tncm91bmQtY29sb3I6d2hpdGU7IH0KCi5oaWdobGlnaHRtZSB7IGJhY2tncm91bmQtY29sb3I6eWVsbG93OyB9CgpwIHsgYmFja2dyb3VuZC1jb2xvcjp3aGl0ZTsgfQoKPC9zdHlsZT4KYGBgCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQojIERldGVjdCwgaW5zdGFsbCwgYW5kIGxvYWQgcGFja2FnZXMgaWYgbmVlZGVkLgppZiAoIXJlcXVpcmUoImtuaXRyIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygia25pdHIiKQogICBsaWJyYXJ5KGtuaXRyKQp9CmlmICghcmVxdWlyZSgibGVhZmxldCIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoImxlYWZsZXQiKQogICBsaWJyYXJ5KGxlYWZsZXQpCn0KaWYgKCFyZXF1aXJlKCJFbnZTdGF0cyIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoIkVudlN0YXRzIikKICAgbGlicmFyeShFbnZTdGF0cykKfQppZiAoIXJlcXVpcmUoIk1BU1MiKSkgewogICBpbnN0YWxsLnBhY2thZ2VzKCJNQVNTIikKICAgbGlicmFyeShNQVNTKQp9CmlmICghcmVxdWlyZSgicGh5dG9vbHMiKSkgewogICBpbnN0YWxsLnBhY2thZ2VzKCJwaHl0b29scyIpCiAgIGxpYnJhcnkocGh5dG9vbHMpCn0KaWYgKCFyZXF1aXJlKCJtbGJlbmNoIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygibWxiZW5jaCIpCiAgIGxpYnJhcnkobWxiZW5jaCkKfQppZiAoIXJlcXVpcmUoInBhbmRlciIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoInBhbmRlciIpCiAgIGxpYnJhcnkocGFuZGVyKQp9CmlmICghcmVxdWlyZSgiSVN3UiIpKSB7CiAgIGluc3RhbGwucGFja2FnZXMoIklTd1IiKQogICBsaWJyYXJ5KElTd1IpCn0KaWYgKCFyZXF1aXJlKCJnZ3Bsb3QyIikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygiZ2dwbG90MiIpCiAgIGxpYnJhcnkoZ2dwbG90MikKfQppZiAoIXJlcXVpcmUoImZvcmVjYXN0IikpIHsKICAgaW5zdGFsbC5wYWNrYWdlcygiZm9yZWNhc3QiKQogICBsaWJyYXJ5KGZvcmVjYXN0KQp9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gRkFMU0UsICAKICAgICAgICAgICAgICAgICAgIHdhcm5pbmcgPSBGQUxTRSwgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICBtZXNzYWdlID0gRkFMU0UsICAKICAgICAgICAgICAgICAgICAgIHJlc3VsdHMgPSBUUlVFLCAgCiAgICAgICAgICAgICAgICAgICBjb21tZW50ID0gRkFMU0UgICAKICAgICAgICAgICAgICAgICAgICAgICkgICAKYGBgCgpgYGB7cn0Kc193ZWF0aCA8LSByZWFkLmNzdigiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL1J5YW5MZWJvL1NUQS0zMjEvcmVmcy9oZWFkcy9tYWluL3NlYXR0bGVfd2VhdGhlci5jc3YiLCBoZWFkZXIgPSBUUlVFKQoKYGBgCgojIEludHJvZHVjdGlvbgpJbiB0aGlzIHByb2plY3Qgd2Ugd2FudCB0byBmaW5kIGEgdGltZSBzZXJpZXMgd2l0aCBib3RoIHRyZW5kIGFuZCBzZWFzb25hbGl0eS4gV2UgdXNlZCBhIFNlYXR0bGUgd2VhdGhlciBkYXRhIHNldCBhbmQgYXJlIGdvaW5nIHRvIHVzZSB0aGUgMTYwIG1vc3QgcmVjZW50IG9ic2VydmF0aW9ucy4gV2Ugd2lsbCBsb29rIHRvIGZpdCB0aHJlZSB0eXBlcyBvZiBzbW9vdGhpbmcgbW9kZWxzLiBUaGVzZSBtb2RlbHMgYXJlIHNpbXBsZSBleHBvbmVudGlhbCBtb2RlbCwgSG9sdCBtb2RlbHMsIGFuZCAgSG9sdC1XaW50ZXLigJlzIG1vZGVscy4gV2Ugd2lsbCBhbHNvIGxvb2sgYXQgdmlzdWFsaXphdGlvbnMgZm9yIHRoaXMuCgoKCgojIyBEYXRhIERlc2NyaXB0aW9uCgoqIE1vbnRoLSBNb250aCBvZiB0aGUgeWVhciAoaW4gbnVtYmVycy4gRXg6IDEtSmFuLCAyLUZlYi4uLikKKiBZZWFyLSBUaGUgeWVhciBpdCB3YXMKKiBMb3dUZW1wLSBMb3dlc3QgdGVtcGVyYXR1cmUgCiogSGlnaFRlbXAtIEhpZ2hlc3QgdGVtcGVyYXR1cmUgCiogV2FybWVzdE1pbi0gVGhlIGxvd2VzdCB3YXJtIHRlbXBlcmF0dXJlIAoqIENvbGRlc3RIaWdoLSBUaGUgaGlnaGVzdCBjb2xkIHRlbXBlcmF0dXJlIAoqIEF2ZU1pbi1UaGUgYXZlcmFnZSBtaW5pbXVtIHRlbXBlcmF0dXJlCiogQXZlTWF4LSBUaGUgYXZlcmFnZSBtYXhpbXVtIHRlbXBlcmF0dXJlCiogbWVhblRlbXAtIFRoZSBtZWFuIHRlbXBlcmF0dXJlCiogVG90UHJlY2lwLSBUaGUgdG90YWwgcHJlY2lwaXRhdGlvbgoqIFRvdFNub3ctIFRoZSB0b3RhbCBzbm93CiogTWF4MjRoclByZWNpcC0gVGhlIG1heGltdW0gYW1vdW50IG9mIHByZWNpcGl0YXRpb24gaW4gMjQgaG91cnMKCiMjIFJlc2VhcmNoIFF1ZXN0aW9uCgpUaGUgZ29hbCBvZiB0aGUgYW5hbHlzaXMgaXMgdG8gdXNlIHRpbWUgc2VyaWVzIHRvIG1ha2UgY29uY2x1c2lvbnMgb24gdGhlIG1lYW4gdGVtcGVyYXR1cmUgaW4gU2VhdHRsZSBmb3IgdGhlIG5leHQgeWVhciBiYXNlZCBvbiB0aGlzIGRhdGEgc2V0LgoKIyMgRGF0YSB0dXJuZWQgaW50byBTbW9vdGhpbmcgTW9kZWxzCgpgYGB7cn0Kc3ViX3dlYXRoIDwtIHRhaWwoc193ZWF0aCwgMTYwKQp0cmFpbi53ZWF0aCA9IHN1Yl93ZWF0aCRtZWFuVGVtcFsxOjE0OF0KdGVzdC5zZWEgPSBzdWJfd2VhdGgkbWVhblRlbXBbMTQ5OjE2MF0KCnNlYS53ZWF0aCA9IHRzKHRyYWluLndlYXRoLCBzdGFydD1jKDIwMDAsIDEpLCBmcmVxdWVuY3k9MTIpCgpmaXQxID0gc2VzKHNlYS53ZWF0aCwgaD0xMikKZml0MiA9IGhvbHQoc2VhLndlYXRoLCBpbml0aWFsPSJvcHRpbWFsIiwgaD0xMikKZml0MyA9IGhvbHQoc2VhLndlYXRoLCBkYW1wZWQ9VFJVRSwgaD0xMikKZml0NCA9IGhvbHQoc2VhLndlYXRoLCBleHBvbmVudGlhbD1UUlVFLCBkYW1wZWQ9VFJVRSwgaD0xMikKZml0NSA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9ImFkZGl0aXZlIikKZml0NiA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9Im11bHRpcGxpY2F0aXZlIikKZml0NyA9IGh3KHNlYS53ZWF0aCwgaD0xMiwgc2Vhc29uYWw9ImFkZGl0aXZlIiwgZGFtcGVkPVRSVUUpCmZpdDggPSBodyhzZWEud2VhdGgsIGg9MTIsIHNlYXNvbmFsPSJtdWx0aXBsaWNhdGl2ZSIsIGRhbXBlZD1UUlVFKQoKCgpgYGAKCldlIHdhbnQgdG8gbG9vayBhdCBvbmx5IHRoZSBtb3N0IHJlY2VudCBkYXRhIGFuZCBuYXJyb3dlZCBpdCBkb3duIHRvIHRoZSBsYXN0IDE2MCBvYnNlcnZhdGlvbnMgdG8gYmUgcHJvZHVjdGl2ZS4gV2UgbWFkZSBhIHRlc3QgYW5kIHRyYWluaW5nIGRhdGEgc2V0IHRoZW4gZm9yIHRoZSBwcm9qZWN0LiBUaGVyZSBpcyBvbmx5IDEyIG9ic2VydmF0aW9ucyBpbiB0aGUgdGVzdGluZyBkYXRhIHNldCBhbmQgMTQ4IGluIHRoZSB0cmFpbmluZyBzZXQuIFdlIHRoZW4gd2FudGVkIHRvcCBwdXQgdGhlIGRhdGEgaW50byBzbW9vdGhpbmcgbW9kZWxzIHRvIHRoZW4gdGVzdCB0aGVtIGZvciBhY2N1cmFjeSBtZWFzdXJlcyBsYXRlciBvbi4gCgoKCiMgQWNjdXJhY3kgTWVhc3VyZXMgZm9yIHRyYWluaW5nIGRhdGEKCldlIGdldCBtYW55IGFjY3VyYWN5IHZhbHVlcyB0byBzZWUgd2hpY2ggbW9kZWwgaXMgYmVzdCBmb3Igb3VyIGRhdGEuIAoKYGBge3J9CmFjY3VyYWN5LnRhYmxlID0gcm91bmQocmJpbmQoYWNjdXJhY3koZml0MSksIGFjY3VyYWN5KGZpdDIpLCBhY2N1cmFjeShmaXQzKSwgYWNjdXJhY3koZml0NCksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWNjdXJhY3koZml0NSksIGFjY3VyYWN5KGZpdDYpLCBhY2N1cmFjeShmaXQ3KSwgYWNjdXJhY3koZml0OCkpLDQpCnJvdy5uYW1lcyhhY2N1cmFjeS50YWJsZSk9YygiU0VTIiwiSG9sdCBMaW5lYXIiLCJIb2x0IEFkZC4gRGFtcGVkIiwgIkhvbHQgRXhwLiBEYW1wZWQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIkhXIEFkZC4iLCJIVyBFeHAuIiwiSFcgQWRkLiBEYW1wIiwgIkhXIEV4cC4gRGFtcCIpCmthYmxlKGFjY3VyYWN5LnRhYmxlLCBjYXB0aW9uID0gIlRoZSBhY2N1cmFjeSBtZWFzdXJlcyBvZiB2YXJpb3VzIGV4cG9uZW50aWFsIHNtb290aGluZyBtb2RlbHMgCiAgICAgIGJhc2VkIG9uIHRoZSB0cmFpbmluZyBkYXRhIikKCmBgYAoKV2Ugc2VlIGZyb20gdGhlIG91dHB1dCBhYm92ZSB0aGF0IEhXIEFkZCBEYW1wIGlzIHRoZSBiZXN0IGZpdCBmb3Igb3VyIGRhdGEgZHVlIHRvIGhvdyBpdCBoYXMgdGhlIGxvd2VzdCBhdmVyYWdlIHZhbHVlcyBvZiBhbGwgdGhlIHZhbHVlcyBhYm92ZS4gRm9yIEFDRjEgaXQgaXMgYSByZWxhdGl2ZWx5IGxvdyB2YWx1ZSBtZWFuaW5nIHRoYXQgcmVzaWR1YWwgYXV0b2NvcnJlbGF0aW9uIGlzIG5vdCBhIGNvbmNlcm4uIAoKCiMjIFZpc3VhbGl6YXRpb24KCldlIGFyZSBub3cgdmlzdWFsaXppbmcgdGhlIG9yaWdpbmFsIHRpbWUgc2VyaWVzIGRhdGEgYW5kIGZvcmVjYXN0IHJlc3VsdHMgZnJvbSBkaWZmZXJlbnQgc21vb3RoaW5nIG1vZGVscwoKYGBge3IgZmlnLmFsaWduPSdjZW50ZXInLGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTYuNSwgZmlnLmNhcD0iQ2FzZSBzdHVkeTogQ29tcGFyaW5nIHZhcmlvdXMgZXhwb25lbnRpYWwgc21vb3RoaW5nIG1vZGVscy4ifQpwYXIobWZyb3c9YygyLDEpLCBtYXI9YygzLDQsMywxKSkKCnByZWQuaWQgPSAxNDk6MTYwCnBsb3QoMToxNDgsIHRyYWluLndlYXRoLCBsd2Q9Mix0eXBlPSJvIiwgeWxhYj0iTWVhblRlbXAiLCB4bGFiPSIiLCAKICAgICB4bGltPWMoMSwxNzApLCB5bGltPWMoMzAsIDkwKSwgY2V4PTAuMywKICAgICBtYWluPSJOb24tc2Vhc29uYWwgU21vb3RoaW5nIE1vZGVscyIpCmxpbmVzKHByZWQuaWQsIGZpdDEkbWVhbiwgY29sPSJyZWQiKQpsaW5lcyhwcmVkLmlkLCBmaXQyJG1lYW4sIGNvbD0iYmx1ZSIpCmxpbmVzKHByZWQuaWQsIGZpdDMkbWVhbiwgY29sPSJwdXJwbGUiKQpsaW5lcyhwcmVkLmlkLCBmaXQ0JG1lYW4sIGNvbD0ibmF2eSIpCiMjCnBvaW50cyhwcmVkLmlkLCBmaXQxJG1lYW4sIHBjaD0xNiwgY29sPSJyZWQiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQyJG1lYW4sIHBjaD0xNywgY29sPSJibHVlIiwgY2V4ID0gMC41KQpwb2ludHMocHJlZC5pZCwgZml0MyRtZWFuLCBwY2g9MTksIGNvbD0icHVycGxlIiwgY2V4ID0gMC41KQpwb2ludHMocHJlZC5pZCwgZml0NCRtZWFuLCBwY2g9MjEsIGNvbD0ibmF2eSIsIGNleCA9IDAuNSkKI3BvaW50cyhmaXQwLCBjb2w9ImJsYWNrIiwgcGNoPTEpCmxlZ2VuZCgiYm90dG9tcmlnaHQiLCBsdHk9MSwgY29sPWMoInJlZCIsImJsdWUiLCJwdXJwbGUiLCAibmF2eSIpLHBjaD1jKDE2LDE3LDE5LDIxKSwKICAgYygiU0VTIiwiSG9sdCBMaW5lYXIiLCJIb2x0IExpbmVhciBEYW1wZWQiLCAiSG9sdCBNdWx0aXBsaWNhdGl2ZSBEYW1wZWQiKSwgCiAgIGNleCA9IDAuNywgYnR5PSJuIikKCnBsb3QoMToxNDgsIHRyYWluLndlYXRoLCBsd2Q9Mix0eXBlPSJvIiwgeWxhYj0iTWVhblRlbXAiLCB4bGFiPSIiLCAKICAgICB4bGltPWMoMSwxNzApLCB5bGltPWMoMzAsIDkwKSwgY2V4PTAuMywKICAgICBtYWluPSJIb2x0LVdpbnRlcmQgVHJlbmQgYW5kIFNlYXNvbmFsIFNtb290aGluZyBNb2RlbHMiKQpsaW5lcyhwcmVkLmlkLCBmaXQ1JG1lYW4sIGNvbD0icmVkIikKbGluZXMocHJlZC5pZCwgZml0NiRtZWFuLCBjb2w9ImJsdWUiKQpsaW5lcyhwcmVkLmlkLCBmaXQ3JG1lYW4sIGNvbD0icHVycGxlIikKbGluZXMocHJlZC5pZCwgZml0OCRtZWFuLCBjb2w9Im5hdnkiKQoKcG9pbnRzKHByZWQuaWQsIGZpdDUkbWVhbiwgcGNoPTE2LCBjb2w9InJlZCIsIGNleCA9IDAuNSkKcG9pbnRzKHByZWQuaWQsIGZpdDYkbWVhbiwgcGNoPTE3LCBjb2w9ImJsdWUiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQ3JG1lYW4sIHBjaD0xOSwgY29sPSJwdXJwbGUiLCBjZXggPSAwLjUpCnBvaW50cyhwcmVkLmlkLCBmaXQ4JG1lYW4sIHBjaD0yMSwgY29sPSJuYXZ5IiwgY2V4ID0gMC41KQojIyMKbGVnZW5kKCJib3R0b21yaWdodCIsIGx0eT0xLCBjb2w9YygicmVkIiwiYmx1ZSIsInB1cnBsZSIsICJuYXZ5IikscGNoPWMoMTYsMTcsMTksMjEpLAogICBjKCJIVyBBZGRpdGl2ZSIsIkhXIE11bHRpcGxpY2F0aXZlIiwiSFcgQWRkaXRpdmUgRGFtcGVkIiwgIkhXIE11bHRpcGxpY2F0aXZlIERhbXBlZCIpLCAKICAgY2V4ID0gMC43LCBidHk9Im4iKQoKYGBgCgpXZSBzZWUgZnJvbSB0aGUgYWJvdmUgdmlzdWFsIHRoYXQgSFfigJlzIGxpbmVhciB0cmVuZCB3aXRoIGFuIGFkZGl0aXZlIGRhbXBlZCBzZWFzb25hbCBtb2RlbCBpcyB0aGUgYmVzdCBmaXQuIFRoaXMgbWF0Y2hlcyB1cCB3aXRoIHRoZSBhY2N1cmFjeSBtZWFzdXJlcyB3ZSB0YWxrZWQgYWJvdXQgYmVmb3JlLiBJdCBwZXJmb3JtZWQgd2VsbCBkdWUgdG8gaG93IGl0IGRvZXMgbm90IGhhdmUgYSB3aWRlIHByZWRpY3Rpb24gYmFuZC4gCgojIEFjY3VyYWN5IE1lYXN1cmVzIGZvciB0ZXN0aW5nIGRhdGEKCldlIHNhdyB0aGF0IHRoZSB2aXN1YWxpemF0aW9uIGFuZCBvdXIgdHJhaW5pbmcgZGF0YSBzaG93ZWQgdXMgdGhhdCBIVyBBZGQgRGFtcCBpcyB0aGUgYXBwcm9wcmlhdGUgbW9kZWwuIFRvIG1ha2UgYSByZWFsIGxpZmUgZm9yZWNhc3Qgb2Ygb3VyIGRhdGEgd2UgbmVlZCB0byB1c2UgdGhlIHdob2xlIGRhdGEgc2V0IHdpdGggYWxsIDE2MCByZWNlbnQgb2JzZXJ2YXRpb25zLiAKCmBgYHtyfQoKYWNjLmZ1biA9IGZ1bmN0aW9uKHRlc3QuZGF0YSwgbW9kLm9iail7CiAgUEU9MTAwKih0ZXN0LmRhdGEtbW9kLm9iaiRtZWFuKS9tb2Qub2JqJG1lYW4KICBNQVBFID0gbWVhbihhYnMoUEUpKQogICMjIwogIEU9dGVzdC5kYXRhLW1vZC5vYmokbWVhbgogIE1TRT1tZWFuKEVeMikKICAjIyMKICBhY2N1cmFjeS5tZXRyaWM9YyhNU0U9TVNFLCBNQVBFPU1BUEUpCiAgYWNjdXJhY3kubWV0cmljCiAgCn0KCgpgYGAKCmBgYHtyfQpwcmVkLmFjY3VyYWN5ID0gcmJpbmQoU0VTID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQxKSwKICAgICAgICAgICAgICAgICAgICAgIEhvbHQuQWRkID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQyKSwKICAgICAgICAgICAgICAgICAgICAgIEhvbHQuQWRkLkRhbXAgPWFjYy5mdW4odGVzdC5kYXRhPXRlc3Quc2VhLCBtb2Qub2JqPWZpdDMpLAogICAgICAgICAgICAgICAgICAgICAgSG9sdC5FeHAgPWFjYy5mdW4odGVzdC5kYXRhPXRlc3Quc2VhLCBtb2Qub2JqPWZpdDQpLAogICAgICAgICAgICAgICAgICAgICAgSFcuQWRkID1hY2MuZnVuKHRlc3QuZGF0YT10ZXN0LnNlYSwgbW9kLm9iaj1maXQ1KSwKICAgICAgICAgICAgICAgICAgICAgIEhXLkV4cCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0NiksCiAgICAgICAgICAgICAgICAgICAgICBIVy5BZGQuRGFtcCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0NyksCiAgICAgICAgICAgICAgICAgICAgICBIVy5FeHAuRGFtcCA9YWNjLmZ1bih0ZXN0LmRhdGE9dGVzdC5zZWEsIG1vZC5vYmo9Zml0OCkpCmthYmxlKHByZWQuYWNjdXJhY3ksIGNhcHRpb249IlRoZSBhY2N1cmFjeSBtZWFzdXJlcyBvZiB2YXJpb3VzIGV4cG9uZW50aWFsIHNtb290aGluZyBtb2RlbHMgCiAgICAgIGJhc2VkIG9uIHRoZSB0ZXN0aW5nIGRhdGEiKQoKYGBgCgpMb29raW5nIGF0IHRoZSB0YWJsZSB3ZSBzZWUgdGhhdCBIVyBBZGQgRGFtcGVkIGlzIGFnYWluIHRoZSBiZXN0IG1vZGVsIHdoaWNoIGlzIGNvbnNpc3RlbnQgdG8gb3VyIGNvbmNsdXNpb25zIG1hZGUgZWFybGllciBpbiB0aGUgcHJvamVjdC4gSFcuRXhwIGhhcyB0aGUgbG93ZXN0IGF2ZXJhZ2UgdmFsdWVzIGFuZCBIVy5BREQuRGFtcCBoYXMgdGhlIHNlY29uZCBsb3dlc3QuIEhXIEFkZCBEYW1wZWQgaXMgdGhlIGJlc3QgbW9kZWwgdGhhdCB3ZSBoYXZlIGxvb2tlZCBhdCBkdWUgdG8gTVNFICg1ODUuMDQpIGFuZCBNQVBFICg0MC40NCkgYm90aCBiZWluZyBvbmUgb2YgdGhlIGxvd2VzdCBhdmVyYWdlIHZhbHVlcyBvdXQgb2YgdGhlIG1vZGVscy4gSXQgYWxzbyBoYXMgdGhlIGxvd2VzdCBhdmVyYWdlIGluIHRoZSB0cmFpbmluZyBkYXRhIHdoaWxlIEhXLkV4cCBoYXMgYWJvdXQgdGhlIHRoaXJkIGJlc3QgYXZlcmFnZSBmb3IgdGhlIHRyYWluaW5nIGRhdGEuIFdoaWNoIG1lYW5zIHRoYXQgdGhlICBIVy5BZGQuRGFtcGVkIGhhcyB0aGUgYmVzdCBhbGwgYXJvdW5kIGFjY3VyYWN5IG1lYXN1cmVzIGZvciBib3RoIHRoZSB0cmFpbmluZyBhbmYgdGVzdGluZyBkYXRhLgoKIyBGaW5hbCBNb2RlbAoKSGVyZSB3ZSB1c2UgdGhlIHdob2xlIGRhdGEgc2V0IGFuZCByZWZpdCBpdCB1c2luZyBzbW9vdGhpbmcgcGFyYW1ldGVycy4KCmBgYHtyfQpzX3dlYXRoIDwtIHJlYWQuY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vUnlhbkxlYm8vU1RBLTMyMS9yZWZzL2hlYWRzL21haW4vc2VhdHRsZV93ZWF0aGVyLmNzdiIsIGhlYWRlciA9IFRSVUUpCnNlYXR0bGV3ZWF0aD10cyhzX3dlYXRoJG1lYW5UZW1wWzE6MTYwXSwgc3RhcnQ9MjAwMCwgZnJlcXVlbmN5ID0gMTIpCmZpbmFsLm1vZGVsID0gaHcoc2VhdHRsZXdlYXRoLGg9MTIsIHNlYXNvbmFsPSJhZGRpdGl2ZSIpIApzbW9vdGhpbmcucGFyYW1ldGVyID0gZmluYWwubW9kZWwkbW9kZWwkcGFyWzE6M10Ka2FibGUoc21vb3RoaW5nLnBhcmFtZXRlciwgY2FwdGlvbj0iRXN0aW1hdGVkIHZhbHVlcyBvZiB0aGUgc21vb3RoaW5nIHBhcmFtZXRlcnMgaW4KICAgICAgSG9sdC1XaW50ZXJzIGxpbmVhciB0cmVuZCB3aXRoIGFkZGl0aXZlIHNlYXNvbmFsaXR5IikKCmBgYAoKVGhlIG91dHB1dCBhYm92ZSBzaG93cyBhIHZhbHVlIG9mIDAuMTU2IGZvciBhbHBoYS4gVGhpcyBpbmRpY2F0ZXMgdGhhdCB0aGUgbW9kZWwgcGxhY2VzIG1vZGVyYXRlIHdlaWdodCBvbiBvdXIgcmVjZW50IG9ic2VydmF0aW9ucy4gIEZvciBHYW1tYSwgMC4wMDAxLCBzaW5jZSBpdCBpcyBjbG9zZSB0byAwIGl0IHNob3dzIHRoYXQgc2Vhc29uYWxpdHkgcGF0dGVybnMgYXJlIHN0YWJsZS4gRm9yIEJldGEsIDAuMDAwMSwgc2luY2UgaXQgaXMgY2xvc2UgdG8gMCBpdCBkb2Vzbid0IGhhdmUgaW1wb3J0YW5jZSB0byB1cGRhdGluZyB0aGUgdHJlbmQgYmFzZWQgb24gb3VyIHJlY2VudCBvYnNlcnZhdGlvbnMuCgoKIyBDb25jbHVzaW9uCgpUaGUgU2VhdHRsZSB3ZWF0aGVyIGRhdGEgc2V0IEkgdXNlZCBzaG93ZWQgdGhhdCB0aGUgYmVzdCBtb2RlbCB3ZSBjb3VsZCB1c2Ugd2FzIEhXJ3MgbGluZWFyIHRyZW5kIHVzaW5nIGFuIGFkZGl0aXZlIGRhbXBlZCBzZWFzb25hbCBtb2RlbC4gV2UgZ290IHRoaXMgYnkgc3BsaXR0aW5nIG91ciBkYXRhIGludG8gYSB0cmFpbmluZyBkYXRhIHNldCBhbmQgdGVzdGluZyBkYXRhIHNldC4gV2UgYWxzbyBzYXcgZnJvbSB0aGUgYWNjdXJhY3kgbWVhc3VyZXMgYW5kIHRoZSB2aXN1YWxzIHRoYXQgdGhlIGFkZGl0aXZlIGRhbXBlZCBtb2RlbCBmcm9tIEhXIHdhcyB0aGUgYmVzdCBtb2RlbC4gRm9yIG91ciBmaW5hbCBtb2RlbCB3ZSBnb3QgZ2FtbWEgYW5kIGJldGEgdmFsdWVzIGNsb3NlIHRvIDAgc28gdGhhdCBzaG93ZWQgc2Vhc29uYWxpdHkgcGF0dGVybnMgd2VyZSBzdGFibGUgYW5kIHdlIGRvIG5vdCBuZWVkIHRvIHVwZGF0ZSB0aGUgdHJlbmQuIE91ciBhbHBoYSB3YXMgMC4xNTYgd2hpY2ggc2hvd2VkIHRoYXQgdGhlIG1vZGVsIHBsYWNlZCBzb21lIHdlaWdodCBvbiBvdXIgcmVjZW50IG9ic2VydmF0aW9ucy4KClRoaXMgcHJvamVjdCB3ZSBtYWRlIHN1cmUgdG8gbG9vayBhdCB0aGUgbW9zdCByZWNlbnQgMTYwIG9ic2VydmF0aW9ucyBkdWUgdG8gbmVlZGluZyBtb3JlIHRoYW4gMTAwLiBXZSB0aGVuIHVzZWQgYSBmcmVxdWVuY3kgb2YgMTIgZm9yIHRoZSAxMiBtb250aHMgb2YgdGhlIHllYXIuCgpPdXIgcGVyZm9ybWFuY2UgbWV0cmljcyBvZiBNU0UgYW5kIE1BUEUgc2hvd2VkIHRoYXQgdGhlaXIgdmFsdWVzIHdlcmUgdGhlIGxvd2VzdC4gVGhpcyBtZWFudCB0aGF0IG91ciBwcmVkaWN0b3JzIGFyZSBjbG9zZXIgdG8gdGhlIHRydWUgdmFsdWVzIGFuZCB0aGF0IG91ciBtb2RlbCBoYXMgdGhlIG1vc3QgYWNjdXJhdGUgZm9yZWNhc3QuICAgICAgICAKCgoKCgoK