Sawyer Benson’s Master Thesis
December 19, 2021


Current Progress on OLS Models & Methodology

Hey Yuki,

I wanted to send you an update on my progress with the Hedonic Pricing Model (i.e. HPM or OLS) portion of my thesis and figured this would be the best way to do it since I have so many methods and lines of code to share with you and notate on.

I have a section at the end (6) with questions and next steps, but the main thing I’m asking for here is your approval (or suggestions) about the current process. If you are satisfied with the way I analyzed the Corona subset, and the method used for the specific beta coefficient comparisons across the corona subset (see results in section 5.3 for example of final statement), I will begin to complete the rest of the subsects in a similar way.

Note:


Thanks!

Sawyer



Outline of This Document

  1. Update on data set dimensions
  2. Base OLS model using data including outliers
  3. Outlier-removal process
  4. Base OLS model using data excluding outliers
  5. Subset analysis
    • Test significance of corona dummy variable
    • Compare specific beta coefficient (e.g. number of bedrooms)
  6. Questions for you and next steps


1. Update on Dataset

After cleaning the original data set of 5000 observations, I realized just how many observations are lost in the process of the first-stage cleaning (e.g. removing all #N/A) and how unbalanced the cleaned set was in terms of post and pre-corona observations.

So, I decided to do another, larger data pull with the following results:

  • Location: Louisiana, USA
  • Observations: 15,000
    • Pre-corona: 10,000
    • Post-corona: 5000
  • Date Range: March 23, 2010 - December 15, 2021


2. Base OLS: Including Outliers

This model is just to see where we are in terms of data consistency and relevance.

attach(data)
summary(lm_raw)

Call:
lm(formula = sold_price ~ . - mls_number - school_high - school_junior - 
    school_middle - roof_type_shingle_bi - sold_date - total_area - 
    corona_crisis - post_corona_bi, data = data)

Residuals:
    Min      1Q  Median      3Q     Max 
-661307  -37161    -573   33353 1746936 

Coefficients:
                                       Estimate Std. Error t value Pr(>|t|)    
(Intercept)                           1.176e+05  6.404e+04   1.837 0.066268 .  
property_typeDUP                      4.597e+03  4.584e+04   0.100 0.920117    
property_typeOTH                      1.204e+05  4.598e+04   2.617 0.008867 ** 
property_typePAT                      1.245e+04  1.635e+04   0.762 0.446193    
property_typeSGL                      5.470e+04  5.654e+03   9.675  < 2e-16 ***
property_typeTNH                      2.772e+04  7.616e+03   3.639 0.000274 ***
air_conditioningnot_central          -2.581e+04  2.599e+03  -9.931  < 2e-16 ***
appartment_bi                        -1.226e+04  1.993e+04  -0.615 0.538653    
patio_bi                              6.882e+03  1.489e+03   4.623 3.82e-06 ***
school_general                        2.611e+04  2.779e+03   9.397  < 2e-16 ***
photo_count                           1.473e+03  8.050e+01  18.299  < 2e-16 ***
pool_bi                               2.035e+04  2.365e+03   8.605  < 2e-16 ***
rear_yard_access_bi                  -4.922e+02  4.627e+03  -0.106 0.915277    
roof_typeother                       -8.992e+02  2.498e+03  -0.360 0.718919    
roof_typeshingle                      2.820e+04  2.820e+03   9.998  < 2e-16 ***
roof_typeslate                        1.236e+04  1.085e+04   1.139 0.254770    
gas_typeNatural                      -8.737e+04  5.617e+04  -1.555 0.119877    
gas_typeNone                         -9.020e+04  5.618e+04  -1.606 0.108370    
gas_typePropane                      -2.815e+04  5.878e+04  -0.479 0.631982    
out_building_livable_bi               4.749e+04  1.197e+04   3.969 7.25e-05 ***
out_building_not_livable_bi          -7.553e+03  1.571e+03  -4.807 1.55e-06 ***
living_area                           8.170e+01  1.736e+00  47.051  < 2e-16 ***
land_acres                           -3.619e-01  4.141e+00  -0.087 0.930370    
appliances_included_bi                2.187e+04  1.978e+03  11.060  < 2e-16 ***
garage_bi                             1.108e+04  1.477e+03   7.500 6.74e-14 ***
conditionExcellent                    8.670e+04  8.505e+03  10.194  < 2e-16 ***
conditionFair                        -4.604e+04  1.211e+04  -3.803 0.000144 ***
conditionNew                          8.495e+04  1.211e+04   7.013 2.44e-12 ***
conditionOther                        2.756e+04  8.981e+03   3.069 0.002152 ** 
conditionPoor                        -5.053e+04  1.542e+04  -3.278 0.001049 ** 
conditionVrgd                         3.133e+04  8.680e+03   3.609 0.000308 ***
energy_efficient_bi                   1.384e+04  1.647e+03   8.402  < 2e-16 ***
exterior_typemetal                    7.434e+03  4.497e+03   1.653 0.098357 .  
exterior_typeother                    1.260e+04  2.113e+03   5.963 2.53e-09 ***
exterior_typevinyl                    3.459e+03  2.417e+03   1.431 0.152507    
exterior_typewood                    -2.405e+02  3.470e+03  -0.069 0.944740    
exterior_featurescourtyard            6.573e+03  1.331e+04   0.494 0.621506    
exterior_featuresfence               -9.261e+04  8.013e+03 -11.559  < 2e-16 ***
exterior_featuresnone                -8.187e+04  8.007e+03 -10.225  < 2e-16 ***
exterior_featuresporch               -8.265e+04  8.171e+03 -10.116  < 2e-16 ***
exterior_featurestennis_court        -7.168e+04  3.316e+04  -2.161 0.030686 *  
fire_place_bi                         8.144e+03  1.544e+03   5.273 1.36e-07 ***
foundation_typeraised                -1.661e+04  2.454e+03  -6.768 1.35e-11 ***
foundation_typeslab                   1.697e+03  2.182e+03   0.778 0.436555    
beds_total                           -7.684e+03  1.349e+03  -5.698 1.24e-08 ***
bath_full                             2.830e+04  1.503e+03  18.829  < 2e-16 ***
bath_half                             2.430e+04  1.734e+03  14.012  < 2e-16 ***
age                                   6.546e+02  5.991e+01  10.926  < 2e-16 ***
days_on_market                       -7.690e+01  7.085e+00 -10.853  < 2e-16 ***
sewer_typeother                      -3.691e+03  1.568e+03  -2.354 0.018604 *  
sewer_typeseptic                     -6.044e+03  2.414e+03  -2.504 0.012286 *  
spa_locationNone                     -8.745e+04  2.327e+04  -3.758 0.000172 ***
spa_locationOutside                   7.885e+03  3.035e+04   0.260 0.794999    
stories                              -2.618e+03  2.340e+03  -1.119 0.263341    
property_styleArts and Crafts         8.724e+04  3.055e+04   2.856 0.004297 ** 
property_styleCamelback               1.736e+05  2.837e+04   6.120 9.58e-10 ***
property_styleCamp                    2.627e+04  9.572e+03   2.745 0.006066 ** 
property_styleCape Cod               -9.783e+04  4.586e+04  -2.133 0.032914 *  
property_styleColonial                1.981e+04  1.478e+04   1.340 0.180138    
property_styleContemporary           -5.681e+03  4.659e+03  -1.219 0.222700    
property_styleCottage                 8.189e+03  5.495e+03   1.490 0.136166    
property_styleCraftsman               9.607e+04  3.259e+04   2.948 0.003206 ** 
property_styleCreole Cottage          1.038e+05  2.258e+04   4.595 4.37e-06 ***
property_styleCreole/French Colonial -7.754e+04  4.629e+04  -1.675 0.093901 .  
property_styleFarm                    3.365e+04  4.662e+04   0.722 0.470416    
property_styleFrench Country          5.213e+03  4.978e+03   1.047 0.295029    
property_styleFrench Provincial       1.350e+05  2.325e+04   5.807 6.48e-09 ***
property_styleGeorgian                2.751e+05  4.618e+04   5.956 2.64e-09 ***
property_styleLog Home                1.295e+05  7.922e+04   1.634 0.102191    
property_styleMid Century Modern      2.856e+04  3.987e+04   0.716 0.473809    
property_styleMobile Home            -1.003e+05  5.431e+03 -18.475  < 2e-16 ***
property_styleModular                -7.862e+04  1.596e+04  -4.927 8.45e-07 ***
property_styleNot Specific           -5.837e+04  4.225e+03 -13.816  < 2e-16 ***
property_styleOther                   1.672e+04  6.285e+03   2.661 0.007804 ** 
property_styleParty Wall Double       8.516e+03  1.068e+04   0.797 0.425404    
property_styleRaised Basement         1.442e+05  3.272e+04   4.408 1.05e-05 ***
property_styleRanch                  -4.359e+04  4.097e+03 -10.639  < 2e-16 ***
property_styleShot Gun                4.879e+03  1.285e+04   0.380 0.704116    
property_styleSplit Level            -4.373e+04  4.041e+04  -1.082 0.279165    
property_styleTownhouse              -3.516e+04  8.713e+03  -4.036 5.47e-05 ***
property_styleTraditional            -1.610e+04  3.309e+03  -4.864 1.16e-06 ***
property_styleTudor                   2.319e+05  5.587e+04   4.150 3.34e-05 ***
property_styleVictorian               9.847e+04  1.375e+04   7.159 8.51e-13 ***
city_limit_bi                         3.560e+04  4.433e+03   8.031 1.04e-15 ***
subdivision_bi                       -3.279e+03  2.783e+03  -1.178 0.238689    
tennis_court_bi                      -8.952e+04  3.054e+04  -2.931 0.003379 ** 
termite_contractUnknown              -2.548e+04  6.732e+03  -3.785 0.000155 ***
termite_contractYes                   2.949e+04  4.851e+03   6.080 1.23e-09 ***
water_typeOther                      -2.110e+03  1.805e+04  -0.117 0.906958    
water_typePublic                      1.103e+04  1.776e+04   0.621 0.534341    
water_typeWell                        1.712e+04  1.904e+04   0.899 0.368461    
waterfront_bi                         3.789e+04  2.583e+03  14.671  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 78740 on 14517 degrees of freedom
Multiple R-squared:  0.6486,    Adjusted R-squared:  0.6464 
F-statistic: 294.5 on 91 and 14517 DF,  p-value: < 2.2e-16
par(mfrow = c(2,2))
plot(lm_raw)


3. Outlier-Removal Process

All variables subject to outliers are tested and outliers are removed.

All observations more than 1.5 standard deviations from the mean are considered outliers, except for in the case of ‘age,’ which is restricted to 3.5 standard deviations.


3.1 Summary of Outlier Removal
  1. Removals: 3,186
    • NAs: 390
    • Outliers: 2,796
  2. Final count: 11,813
    • Pre-Corona: 8184
    • Post-Corona: 3692
table(data_no_outliers$corona_crisis)


Note: Names of variables are on the left, data with outliers is on the left, and outliers removed are on the right.

Note: the folloiwng variables’ outliers were delt with in Excel

  • beds_total
  • bath_full
  • bath_half



4. Base OLS: No Outliers

We now can see the results of removing outliers on our most general model.

These differences are particulary clear when comparing the two groupings of four diagnostic plots at the end of this regression to see the reduction in both extreme residual values and high-leverage points.

summary(lm_clean)

Call:
lm(formula = sold_price ~ . - mls_number - school_high - school_junior - 
    school_middle - roof_type_shingle_bi - sold_date - total_area - 
    corona_crisis - post_corona_bi, data = data_no_outliers)

Residuals:
    Min      1Q  Median      3Q     Max 
-277056  -28678    -371   28506  277632 

Coefficients:
                                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)                       1.384e+05  5.433e+04   2.548 0.010862 *  
property_typeDUP                 -1.742e+04  2.853e+04  -0.611 0.541401    
property_typeOTH                  1.060e+05  2.865e+04   3.699 0.000218 ***
property_typePAT                  3.921e+03  1.070e+04   0.367 0.713903    
property_typeSGL                  2.976e+04  3.803e+03   7.824 5.57e-15 ***
property_typeTNH                  2.022e+04  4.995e+03   4.048 5.19e-05 ***
air_conditioningnot_central      -2.816e+04  1.807e+03 -15.583  < 2e-16 ***
appartment_bi                    -6.655e+03  2.108e+04  -0.316 0.752215    
patio_bi                          8.603e+03  1.032e+03   8.340  < 2e-16 ***
school_general                    2.626e+04  1.980e+03  13.261  < 2e-16 ***
photo_count                       1.004e+03  6.061e+01  16.556  < 2e-16 ***
pool_bi                           1.142e+04  1.777e+03   6.426 1.36e-10 ***
rear_yard_access_bi              -1.209e+04  3.930e+03  -3.078 0.002090 ** 
roof_typeother                    2.178e+03  1.811e+03   1.203 0.229086    
roof_typeshingle                  3.245e+04  2.057e+03  15.775  < 2e-16 ***
roof_typeslate                    2.030e+03  8.264e+03   0.246 0.806012    
gas_typeNatural                  -1.477e+05  4.951e+04  -2.984 0.002849 ** 
gas_typeNone                     -1.458e+05  4.951e+04  -2.945 0.003239 ** 
gas_typePropane                  -8.344e+04  5.361e+04  -1.557 0.119589    
out_building_livable_bi           2.139e+04  1.259e+04   1.699 0.089299 .  
out_building_not_livable_bi      -8.125e+03  1.114e+03  -7.293 3.24e-13 ***
living_area                       6.133e+01  1.576e+00  38.917  < 2e-16 ***
land_acres                        1.366e+03  1.337e+03   1.021 0.307039    
appliances_included_bi            2.271e+04  1.360e+03  16.695  < 2e-16 ***
garage_bi                         1.417e+04  1.021e+03  13.886  < 2e-16 ***
conditionExcellent                6.011e+04  6.572e+03   9.146  < 2e-16 ***
conditionFair                    -3.016e+04  9.250e+03  -3.261 0.001114 ** 
conditionOther                    1.905e+04  6.752e+03   2.821 0.004795 ** 
conditionPoor                    -3.701e+04  1.157e+04  -3.200 0.001378 ** 
conditionVrgd                     2.836e+04  6.571e+03   4.315 1.61e-05 ***
energy_efficient_bi               1.440e+04  1.159e+03  12.423  < 2e-16 ***
exterior_typemetal               -2.640e+03  3.190e+03  -0.828 0.407914    
exterior_typeother                5.009e+03  1.472e+03   3.403 0.000670 ***
exterior_typevinyl               -9.798e+02  1.673e+03  -0.586 0.558148    
exterior_typewood                -1.111e+04  2.419e+03  -4.595 4.38e-06 ***
exterior_featurescourtyard       -8.801e+03  1.235e+04  -0.712 0.476230    
exterior_featuresfence           -3.169e+04  6.614e+03  -4.791 1.68e-06 ***
exterior_featuresnone            -2.234e+04  6.600e+03  -3.385 0.000713 ***
exterior_featuresporch           -2.366e+04  6.719e+03  -3.522 0.000430 ***
exterior_featurestennis_court    -2.078e+04  2.537e+04  -0.819 0.412714    
fire_place_bi                     1.422e+04  1.079e+03  13.173  < 2e-16 ***
foundation_typeraised            -2.128e+04  1.710e+03 -12.444  < 2e-16 ***
foundation_typeslab               4.839e+03  1.511e+03   3.203 0.001365 ** 
beds_total                        3.961e+03  9.945e+02   3.983 6.85e-05 ***
bath_full                         1.614e+04  1.073e+03  15.033  < 2e-16 ***
bath_half                         1.258e+04  1.285e+03   9.789  < 2e-16 ***
age                               5.138e+02  1.070e+02   4.803 1.58e-06 ***
days_on_market                   -7.547e+01  7.698e+00  -9.804  < 2e-16 ***
sewer_typeother                  -1.871e+03  1.126e+03  -1.661 0.096745 .  
sewer_typeseptic                 -3.047e+03  1.840e+03  -1.656 0.097779 .  
spa_locationNone                 -2.680e+04  1.695e+04  -1.581 0.113912    
spa_locationOutside               7.579e+04  2.988e+04   2.536 0.011213 *  
stories                          -1.576e+04  1.999e+03  -7.883 3.48e-15 ***
property_styleCamelback           1.402e+05  4.933e+04   2.843 0.004474 ** 
property_styleCamp                2.315e+04  7.236e+03   3.199 0.001383 ** 
property_styleCape Cod           -3.307e+04  2.864e+04  -1.155 0.248199    
property_styleColonial           -2.189e+04  1.216e+04  -1.800 0.071825 .  
property_styleContemporary       -1.165e+04  3.236e+03  -3.600 0.000319 ***
property_styleCottage             6.121e+03  3.961e+03   1.545 0.122290    
property_styleCreole Cottage      3.259e+04  2.846e+04   1.145 0.252138    
property_styleFrench Country      3.262e+03  3.506e+03   0.930 0.352182    
property_styleFrench Provincial   8.103e+03  2.856e+04   0.284 0.776589    
property_styleGeorgian            6.654e+04  4.931e+04   1.349 0.177275    
property_styleMid Century Modern  7.290e+04  3.537e+04   2.061 0.039349 *  
property_styleMobile Home        -8.901e+04  3.837e+03 -23.198  < 2e-16 ***
property_styleModular            -6.680e+04  1.159e+04  -5.763 8.46e-09 ***
property_styleNot Specific       -4.082e+04  2.963e+03 -13.773  < 2e-16 ***
property_styleOther              -1.233e+04  4.639e+03  -2.658 0.007882 ** 
property_styleParty Wall Double  -1.958e+02  6.919e+03  -0.028 0.977423    
property_styleRaised Basement    -1.243e+05  4.930e+04  -2.522 0.011682 *  
property_styleRanch              -3.768e+04  2.841e+03 -13.263  < 2e-16 ***
property_styleShot Gun            1.007e+04  1.009e+04   0.998 0.318300    
property_styleSplit Level        -4.310e+04  2.990e+04  -1.441 0.149522    
property_styleTownhouse          -2.185e+04  5.752e+03  -3.799 0.000146 ***
property_styleTraditional        -1.587e+04  2.334e+03  -6.799 1.11e-11 ***
property_styleVictorian          -1.282e+03  1.378e+04  -0.093 0.925900    
city_limit_bi                     1.002e+04  3.451e+03   2.905 0.003679 ** 
subdivision_bi                    1.021e+04  2.038e+03   5.007 5.61e-07 ***
tennis_court_bi                  -3.960e+04  2.060e+04  -1.922 0.054617 .  
termite_contractUnknown          -2.399e+04  5.128e+03  -4.679 2.91e-06 ***
termite_contractYes              -7.041e+02  4.198e+03  -0.168 0.866787    
water_typeOther                  -4.291e+03  1.338e+04  -0.321 0.748394    
water_typePublic                  1.056e+03  1.319e+04   0.080 0.936233    
water_typeWell                    3.811e+03  1.442e+04   0.264 0.791590    
waterfront_bi                     3.275e+04  1.955e+03  16.748  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 48920 on 11728 degrees of freedom
Multiple R-squared:  0.6772,    Adjusted R-squared:  0.6749 
F-statistic: 292.9 on 84 and 11728 DF,  p-value: < 2.2e-16


4.1 OLS Review: Excluding Outliers
par(mfrow = c(2,2))
plot(lm_clean)
not plotting observations with leverage one:
  11243, 11307, 11322, 11352


4.2 OLS Review: Excluding Outliers
par(mfrow = c(2,2))
plot(lm_raw)
not plotting observations with leverage one:
  12398


5. Subset Analysis

Here I test the following:

  1. Is pre and post-corona data significantly different?
    • I test this by testing the significance of a ‘corona’ dummy variable (1 for post-corona and 0 for pre-corona)
  2. Are there significant differences in the way key variables (e.g. premium for bedrooms or βbeds_total) are impacted by the Corona crisis?
    • I follow a method suggested by UCLA’s statistics department which you can read through here if you wish


5.1 Dummy-Variable Regression: Corona = 1

We see that belonging to the ‘post-corona’ subset is associated on average with a market price premium of +$26,750, ceteris parabus. This finding is statistically significant with p-value < 0.00 under the most controlled model. In other words, one could also say:

A house with similar physical features sold after the first mandated COVID_19 lockdown will, on average, sell for $26,750 more than it would have before this event.

summary(lm_corona_test)

Call:
lm(formula = sold_price ~ post_corona_bi + . - mls_number - school_high - 
    school_junior - school_middle - roof_type_shingle_bi - sold_date - 
    total_area - corona_crisis, data = data_no_outliers)

Residuals:
    Min      1Q  Median      3Q     Max 
-282122  -28410    -127   28419  285225 

Coefficients:
                                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)                       1.146e+05  5.365e+04   2.136 0.032715 *  
post_corona_bi                    2.675e+04  1.522e+03  17.575  < 2e-16 ***
property_typeDUP                 -1.892e+04  2.816e+04  -0.672 0.501674    
property_typeOTH                  1.029e+05  2.828e+04   3.639 0.000275 ***
property_typePAT                  3.402e+03  1.056e+04   0.322 0.747305    
property_typeSGL                  2.942e+04  3.754e+03   7.837 5.02e-15 ***
property_typeTNH                  1.949e+04  4.931e+03   3.952 7.78e-05 ***
air_conditioningnot_central      -2.834e+04  1.784e+03 -15.885  < 2e-16 ***
appartment_bi                    -7.143e+03  2.081e+04  -0.343 0.731396    
patio_bi                          8.529e+03  1.018e+03   8.376  < 2e-16 ***
school_general                    9.670e+03  2.171e+03   4.455 8.49e-06 ***
photo_count                       8.839e+02  6.022e+01  14.679  < 2e-16 ***
pool_bi                           1.175e+04  1.754e+03   6.702 2.15e-11 ***
rear_yard_access_bi              -1.064e+04  3.880e+03  -2.743 0.006091 ** 
roof_typeother                    3.407e+03  1.789e+03   1.905 0.056828 .  
roof_typeshingle                  3.202e+04  2.031e+03  15.768  < 2e-16 ***
roof_typeslate                    1.650e+03  8.158e+03   0.202 0.839677    
gas_typeNatural                  -1.440e+05  4.887e+04  -2.947 0.003210 ** 
gas_typeNone                     -1.436e+05  4.888e+04  -2.938 0.003306 ** 
gas_typePropane                  -8.036e+04  5.292e+04  -1.519 0.128869    
out_building_livable_bi           9.217e+03  1.244e+04   0.741 0.458934    
out_building_not_livable_bi      -8.061e+03  1.100e+03  -7.329 2.47e-13 ***
living_area                       6.178e+01  1.556e+00  39.714  < 2e-16 ***
land_acres                        8.981e+02  1.320e+03   0.680 0.496389    
appliances_included_bi            2.280e+04  1.343e+03  16.977  < 2e-16 ***
garage_bi                         1.430e+04  1.008e+03  14.193  < 2e-16 ***
conditionExcellent                6.016e+04  6.488e+03   9.272  < 2e-16 ***
conditionFair                    -3.058e+04  9.131e+03  -3.349 0.000813 ***
conditionOther                    2.066e+04  6.665e+03   3.099 0.001946 ** 
conditionPoor                    -3.731e+04  1.142e+04  -3.268 0.001085 ** 
conditionVrgd                     2.969e+04  6.487e+03   4.576 4.78e-06 ***
energy_efficient_bi               1.464e+04  1.144e+03  12.796  < 2e-16 ***
exterior_typemetal               -5.789e+02  3.151e+03  -0.184 0.854243    
exterior_typeother                6.929e+03  1.457e+03   4.755 2.01e-06 ***
exterior_typevinyl                7.649e+02  1.655e+03   0.462 0.643895    
exterior_typewood                -1.010e+04  2.388e+03  -4.231 2.35e-05 ***
exterior_featurescourtyard       -5.153e+03  1.220e+04  -0.422 0.672680    
exterior_featuresfence           -2.848e+04  6.531e+03  -4.361 1.31e-05 ***
exterior_featuresnone            -1.939e+04  6.517e+03  -2.975 0.002937 ** 
exterior_featuresporch           -2.021e+04  6.635e+03  -3.045 0.002330 ** 
exterior_featurestennis_court    -1.613e+04  2.505e+04  -0.644 0.519694    
fire_place_bi                     1.506e+04  1.067e+03  14.122  < 2e-16 ***
foundation_typeraised            -2.036e+04  1.689e+03 -12.056  < 2e-16 ***
foundation_typeslab               5.394e+03  1.492e+03   3.616 0.000301 ***
beds_total                        3.729e+03  9.818e+02   3.799 0.000146 ***
bath_full                         1.631e+04  1.060e+03  15.394  < 2e-16 ***
bath_half                         1.258e+04  1.269e+03   9.918  < 2e-16 ***
age                               4.695e+02  1.056e+02   4.445 8.86e-06 ***
days_on_market                   -5.026e+01  7.733e+00  -6.499 8.41e-11 ***
sewer_typeother                  -1.160e+03  1.113e+03  -1.043 0.297015    
sewer_typeseptic                 -2.555e+03  1.817e+03  -1.406 0.159621    
spa_locationNone                 -2.644e+04  1.673e+04  -1.580 0.114098    
spa_locationOutside               7.640e+04  2.950e+04   2.590 0.009610 ** 
stories                          -1.654e+04  1.974e+03  -8.377  < 2e-16 ***
property_styleCamelback           1.331e+05  4.869e+04   2.734 0.006260 ** 
property_styleCamp                2.132e+04  7.144e+03   2.984 0.002853 ** 
property_styleCape Cod           -2.997e+04  2.827e+04  -1.060 0.289183    
property_styleColonial           -2.391e+04  1.200e+04  -1.992 0.046413 *  
property_styleContemporary       -1.345e+04  3.196e+03  -4.209 2.58e-05 ***
property_styleCottage             5.569e+03  3.910e+03   1.424 0.154374    
property_styleCreole Cottage      2.804e+04  2.809e+04   0.998 0.318148    
property_styleFrench Country      3.100e+03  3.460e+03   0.896 0.370371    
property_styleFrench Provincial   4.712e+03  2.819e+04   0.167 0.867240    
property_styleGeorgian            6.082e+04  4.868e+04   1.249 0.211530    
property_styleMid Century Modern  6.977e+04  3.492e+04   1.998 0.045742 *  
property_styleMobile Home        -8.925e+04  3.788e+03 -23.564  < 2e-16 ***
property_styleModular            -6.851e+04  1.144e+04  -5.988 2.19e-09 ***
property_styleNot Specific       -4.194e+04  2.926e+03 -14.334  < 2e-16 ***
property_styleOther              -1.300e+04  4.579e+03  -2.839 0.004527 ** 
property_styleParty Wall Double  -2.545e+03  6.831e+03  -0.373 0.709437    
property_styleRaised Basement    -1.092e+05  4.867e+04  -2.244 0.024833 *  
property_styleRanch              -3.796e+04  2.804e+03 -13.536  < 2e-16 ***
property_styleShot Gun            9.938e+03  9.963e+03   0.997 0.318564    
property_styleSplit Level        -4.244e+04  2.952e+04  -1.438 0.150537    
property_styleTownhouse          -2.246e+04  5.678e+03  -3.956 7.66e-05 ***
property_styleTraditional        -1.593e+04  2.304e+03  -6.914 4.95e-12 ***
property_styleVictorian          -9.880e+02  1.361e+04  -0.073 0.942113    
city_limit_bi                     9.052e+03  3.407e+03   2.657 0.007893 ** 
subdivision_bi                    1.019e+04  2.012e+03   5.065 4.14e-07 ***
tennis_court_bi                  -4.177e+04  2.034e+04  -2.054 0.039978 *  
termite_contractUnknown          -9.997e+03  5.124e+03  -1.951 0.051077 .  
termite_contractYes              -2.259e+02  4.144e+03  -0.055 0.956515    
water_typeOther                  -4.896e+03  1.320e+04  -0.371 0.710818    
water_typePublic                 -2.841e+03  1.303e+04  -0.218 0.827328    
water_typeWell                   -6.071e+01  1.424e+04  -0.004 0.996598    
waterfront_bi                     3.316e+04  1.930e+03  17.180  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 48290 on 11727 degrees of freedom
Multiple R-squared:  0.6855,    Adjusted R-squared:  0.6832 
F-statistic: 300.7 on 85 and 11727 DF,  p-value: < 2.2e-16
par(mfrow = c(2,2))
plot(lm_corona_test)
not plotting observations with leverage one:
  11243, 11307, 11322, 11352


5.2 Testing Differences of Specific Variables Post and Pre-Corona

This UCLA method is implimented with the following steps:

  1. Run a reduced regression model with only:
    • subset dummy variable (e.g. corona dummy)
    • variable of interest (e.g. beds_total)
    • interaction term: (e.g. corona_dummy*beds_total)

Note: The results of the significance level for the interaction term’s beta coefficient tests the null hypothesis: H0: βpre = βpost.


5.3 Example: Number of Bedrooms

As you can see, the interaction term (i.e. beds_total*post_corona_bi) is statistically significant at the 0.00 level, suggesting we reject the null hypothesis that βpre = βpost.

This result can also be interpreted as:

The average premium for each additional bedroom significantly increased after the outbreak of COVID-19 in March 23, 2020, resulting in an average premium increase of $8,583 per room, ceteris parabus.

My (the author’s) hypothesis is that this result is likely driven by an increased demand for an additional bedroom functioning as a ‘home office’ as many workers were legally compelled to work from home, with many of them never returning to full-time, on-site working schedules.

summary(lm_beds_corona_test)

Call:
lm(formula = sold_price ~ post_corona_bi + beds_total + beds_total * 
    post_corona_bi, data = data_no_outliers)

Residuals:
    Min      1Q  Median      3Q     Max 
-293588  -49022    1404   44578  335004 

Coefficients:
                          Estimate Std. Error t value Pr(>|t|)    
(Intercept)                 -12004       4274  -2.809 0.004982 ** 
post_corona_bi               21178       7297   2.902 0.003711 ** 
beds_total                   53500       1366  39.151  < 2e-16 ***
post_corona_bi:beds_total     8583       2316   3.706 0.000212 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 74840 on 11809 degrees of freedom
Multiple R-squared:  0.2392,    Adjusted R-squared:  0.239 
F-statistic:  1238 on 3 and 11809 DF,  p-value: < 2.2e-16



6. Questions for You and Next Steps


6.1 Questions
  1. Are you in agreement with the current method of comparing beta coefficients across models?
    • If you are, then I will continue by repeating the process across several key variables and with the other subsets we’ve already discussed (e.g. top vs bottom 25th percentile of size, price, etc.)
  2. Do you have any suggesions for specific variable transformations in these OLS models (e.g. log(price) )?
    • I will make many transformation for the ML pridition models, but wanted to get your opinion here.
  3. Do you have any other extentions or models which could make this analysis ritcher, more robust?
    • Because this ‘core’ portion of the analysis went so well, I’m interested in pushing the limit of academic technicallity for the sake of the paper and the grading process down the road.


6.2 Next Steps
  1. I am about 60% of the way through coding the ML models and will likely make another RMarkdown file like this one for that process as well
  2. I am currently collecting specific papers for a literature review and also pulling other ‘soft’ figures about the crisis itself in Louisiana for narrative and context.
  3. Once I have completed the ML model and have combined the finding into one, succinct line of analysis, I hope to have a review meeting with you to descuss improvements.




Final Statement

As always, thanks for taking the time to read through this and for your guidence!


All the best,
Sawyer







End of Document

LS0tCnRpdGxlOiAiSGVkb25pYyBQcmljaW5nIE1vZGVscyIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQKICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQKY29kZV9mb2xkaW5nOiBoaWRlCkF1dGhvcjogU2F3eWVyIEJlbnNvbgotLS0KCiMjIyMjIFNhd3llciBCZW5zb24ncyBNYXN0ZXIgVGhlc2lzIAojIyMjIyMgRGVjZW1iZXIgMTksIDIwMjEKCjxicj4KCiMjIyMgQ3VycmVudCBQcm9ncmVzcyBvbiBPTFMgTW9kZWxzICYgTWV0aG9kb2xvZ3kgCgpIZXkgWXVraSwgCgpJIHdhbnRlZCB0byBzZW5kIHlvdSBhbiB1cGRhdGUgb24gbXkgcHJvZ3Jlc3Mgd2l0aCB0aGUgSGVkb25pYyBQcmljaW5nIE1vZGVsIChpLmUuIEhQTSBvciBPTFMpIHBvcnRpb24gb2YgbXkgdGhlc2lzIGFuZCBmaWd1cmVkIHRoaXMgd291bGQgYmUgdGhlIGJlc3Qgd2F5IHRvIGRvIGl0IHNpbmNlIEkgaGF2ZSBzbyBtYW55IG1ldGhvZHMgYW5kIGxpbmVzIG9mIGNvZGUgdG8gc2hhcmUgd2l0aCB5b3UgYW5kIG5vdGF0ZSBvbi4KCkkgaGF2ZSBhIHNlY3Rpb24gYXQgdGhlIGVuZCAoNikgd2l0aCBxdWVzdGlvbnMgYW5kIG5leHQgc3RlcHMsIGJ1dCB0aGUgbWFpbiB0aGluZyBJJ20gYXNraW5nIGZvciBoZXJlIGlzIHlvdXIgYXBwcm92YWwgKG9yIHN1Z2dlc3Rpb25zKSBhYm91dCB0aGUgY3VycmVudCBwcm9jZXNzLiBJZiB5b3UgYXJlIHNhdGlzZmllZCB3aXRoIHRoZSB3YXkgSSBhbmFseXplZCB0aGUgQ29yb25hIHN1YnNldCwgYW5kIHRoZSBtZXRob2QgdXNlZCBmb3IgdGhlIHNwZWNpZmljIGJldGEgY29lZmZpY2llbnQgY29tcGFyaXNvbnMgYWNyb3NzIHRoZSBjb3JvbmEgc3Vic2V0ICooc2VlIHJlc3VsdHMgaW4gc2VjdGlvbiA1LjMgZm9yIGV4YW1wbGUgb2YgZmluYWwgc3RhdGVtZW50KSosIEkgd2lsbCBiZWdpbiB0byBjb21wbGV0ZSB0aGUgcmVzdCBvZiB0aGUgc3Vic2VjdHMgaW4gYSBzaW1pbGFyIHdheS4gCgoKKipOb3RlOioqIAoKKiBUaGUgcmVncmVzc2lvbnMgbWFrZSB0aGUgZG9jdW1lbnQgYXBwZWFyIGJpZ2dlciB0aGFuIGl0IGFjdHVhbGx5IGlzLiBUaGlzIHNob3VsZCB0YWtlIG9ubHkgYSBmZXcgbWludXRlcyB0byByZXZpZXcuCiogRmVlbCBmcmVlIHRvIGxlYXZlIGNvbW1lbnRzIHVzaW5nIHRoZSBjb21tZW50IHRvb2wgaGVyZSBvbiBSUHVicy4KCgo8YnI+CgoqKlRoYW5rcyEqKgoKKipTYXd5ZXIqKgoKPGJyPjxicj4KCiMjIyMgT3V0bGluZSBvZiBUaGlzIERvY3VtZW50CjEuICBVcGRhdGUgb24gZGF0YSBzZXQgZGltZW5zaW9ucwoyLiAgQmFzZSBPTFMgbW9kZWwgdXNpbmcgZGF0YSAqaW5jbHVkaW5nKiBvdXRsaWVycwozLiAgT3V0bGllci1yZW1vdmFsIHByb2Nlc3MKNC4gIEJhc2UgT0xTIG1vZGVsIHVzaW5nIGRhdGEgKmV4Y2x1ZGluZyogb3V0bGllcnMKNS4gIFN1YnNldCBhbmFseXNpcwogICAgICArIFRlc3Qgc2lnbmlmaWNhbmNlIG9mIGNvcm9uYSBkdW1teSB2YXJpYWJsZQogICAgICArIENvbXBhcmUgc3BlY2lmaWMgYmV0YSBjb2VmZmljaWVudCAoZS5nLiBudW1iZXIgb2YgYmVkcm9vbXMpCjYuICBRdWVzdGlvbnMgZm9yIHlvdSBhbmQgbmV4dCBzdGVwcwoKPGJyPgoKIyMjIyMgKioxLiBVcGRhdGUgb24gRGF0YXNldCoqCkFmdGVyIGNsZWFuaW5nIHRoZSBvcmlnaW5hbCBkYXRhIHNldCBvZiA1MDAwIG9ic2VydmF0aW9ucywgSSByZWFsaXplZCBqdXN0IGhvdyBtYW55IG9ic2VydmF0aW9ucyBhcmUgbG9zdCBpbiB0aGUgcHJvY2VzcyBvZiB0aGUgZmlyc3Qtc3RhZ2UgY2xlYW5pbmcgKGUuZy4gcmVtb3ZpbmcgYWxsICNOL0EpIGFuZCBob3cgdW5iYWxhbmNlZCB0aGUgY2xlYW5lZCBzZXQgd2FzIGluIHRlcm1zIG9mIHBvc3QgYW5kIHByZS1jb3JvbmEgb2JzZXJ2YXRpb25zLiAKClNvLCBJIGRlY2lkZWQgdG8gZG8gYW5vdGhlciwgbGFyZ2VyIGRhdGEgcHVsbCB3aXRoIHRoZSBmb2xsb3dpbmcgcmVzdWx0czoKCiogTG9jYXRpb246IExvdWlzaWFuYSwgVVNBCiogT2JzZXJ2YXRpb25zOiAxNSwwMDAKICAgKyBQcmUtY29yb25hOiAxMCwwMDAKICAgKyBQb3N0LWNvcm9uYTogNTAwMAoqIERhdGUgUmFuZ2U6IE1hcmNoIDIzLCAyMDEwIC0gRGVjZW1iZXIgMTUsIDIwMjEKCjxicj4KCiMjIyMjICoqMi4gQmFzZSBPTFM6IEluY2x1ZGluZyBPdXRsaWVycyoqCgpUaGlzIG1vZGVsIGlzIGp1c3QgdG8gc2VlIHdoZXJlIHdlIGFyZSBpbiB0ZXJtcyBvZiBkYXRhIGNvbnNpc3RlbmN5IGFuZCByZWxldmFuY2UuCgpgYGB7ciwgcmVzdWx0cyA9ICdoaWRlJywgY29sbGFwc2U9RkFMU0V9CgojIEltcG9ydCBhbmQgYXR0YWNoIGRhdGEgc2V0IApsaWJyYXJ5KHJlYWR4bCkKZGF0YSA8LSByZWFkX2V4Y2VsKCJEYXRhL0RhdGFfTUxTX0ZpbmFsXzE3LjEyLjIxLnhsc3giKQphdHRhY2goZGF0YSkKCiMgUmVhZGluZyBpbiBQYWNrYWdlcwpsaWJyYXJ5KGdncGxvdDIpICMgR3JhcGhzCmxpYnJhcnkoZ3JpZEV4dHJhKSAjIE9yZ2FuaXplIGdyYXBocwpsaWJyYXJ5KGRwbHlyKSAjIERhdGEgd3JhbmdsaW5nCmxpYnJhcnkodGlkeXIpICMgRGF0YSB3cmFuZ2xpbmcKCiNEYXRhZnJhbWVzCmRhdGEgPC0gZHJvcF9uYShkYXRhKSAjIERyb3AgTmEgVmFsdWVzCmRhdGFfbnVtZXJpYyA8LSBkcGx5cjo6c2VsZWN0X2lmKGRhdGEsIGlzLm51bWVyaWMpICMgTmV3IGRhdGEgc2V0IHdpdGggb25seSAKYGBgCgpgYGB7cn0KI0luc3RhbGxlZCB0byBsaW1pdCB0aGUgbGVuZ3RoIG9mIHJlZ3Jlc3Npb24gb3V0cHV0CiMgc2F2ZSB0aGUgYnVpbHQtaW4gb3V0cHV0IGhvb2sKaG9va19vdXRwdXQgPC0ga25pdHI6OmtuaXRfaG9va3MkZ2V0KCJvdXRwdXQiKQoKIyBzZXQgYSBuZXcgb3V0cHV0IGhvb2sgdG8gdHJ1bmNhdGUgdGV4dCBvdXRwdXQKa25pdHI6OmtuaXRfaG9va3Mkc2V0KG91dHB1dCA9IGZ1bmN0aW9uKHgsIG9wdGlvbnMpIHsKICBpZiAoIWlzLm51bGwobiA8LSBvcHRpb25zJG91dC5saW5lcykpIHsKICAgIHggPC0geGZ1bjo6c3BsaXRfbGluZXMoeCkKICAgIGlmIChsZW5ndGgoeCkgPiBuKSB7CiAgICAgICMgdHJ1bmNhdGUgdGhlIG91dHB1dAogICAgICB4IDwtIGMoaGVhZCh4LCBuKSwgIi4uLi5cbiIpCiAgICB9CiAgICB4IDwtIHBhc3RlKHgsIGNvbGxhcHNlID0gIlxuIikKICB9CiAgaG9va19vdXRwdXQoeCwgb3B0aW9ucykKfSkKYGBgIAoKCmBgYHtyLCBhdHRyLm91dHB1dD0nc3R5bGU9Im1heC1oZWlnaHQ6IDM1MHB4OyInfQoKbG1fcmF3IDwtIGxtKHNvbGRfcHJpY2UgfiAuIC1tbHNfbnVtYmVyIAogICAgICAgICAgICAgICAtIHNjaG9vbF9oaWdoIC0gc2Nob29sX2p1bmlvciAKICAgICAgICAgICAgICAgLSBzY2hvb2xfbWlkZGxlIC0gcm9vZl90eXBlX3NoaW5nbGVfYmkKICAgICAgICAgICAgICAgLSBzb2xkX2RhdGUgLSB0b3RhbF9hcmVhIC1jb3JvbmFfY3Jpc2lzCiAgICAgICAgICAgICAgIC0gcG9zdF9jb3JvbmFfYmksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBkYXRhKQpzdW1tYXJ5KGxtX3JhdykKYGBgCgpgYGB7cn0KcGFyKG1mcm93ID0gYygyLDIpKQpwbG90KGxtX3JhdykKYGBgCgo8YnI+CgojIyMjIyAqKjMuIE91dGxpZXItUmVtb3ZhbCBQcm9jZXNzKioKQWxsIHZhcmlhYmxlcyBzdWJqZWN0IHRvIG91dGxpZXJzIGFyZSB0ZXN0ZWQgYW5kIG91dGxpZXJzIGFyZSByZW1vdmVkLgoKQWxsIG9ic2VydmF0aW9ucyBtb3JlIHRoYW4gMS41IHN0YW5kYXJkIGRldmlhdGlvbnMgZnJvbSB0aGUgbWVhbiBhcmUgY29uc2lkZXJlZCBvdXRsaWVycywgZXhjZXB0IGZvciBpbiB0aGUgY2FzZSBvZiAnYWdlLCcgd2hpY2ggaXMgcmVzdHJpY3RlZCB0byAzLjUgc3RhbmRhcmQgZGV2aWF0aW9ucy4gIAoKPGJyPgoKIyMjIyMgKiozLjEgU3VtbWFyeSBvZiBPdXRsaWVyIFJlbW92YWwqKgoKCjEuIFJlbW92YWxzOiAqKjMsMTg2KioKICAgKyBOQXM6IDM5MAogICArIE91dGxpZXJzOiAyLDc5NgoyLiBGaW5hbCBjb3VudDogKioxMSw4MTMqKgogICArIFByZS1Db3JvbmE6IDgxODQKICAgKyBQb3N0LUNvcm9uYTogMzY5MgoKCgpgYGB7ciByZXN1bHRzPSdoaWRlJ30KCmRhdGFfY291bnQgPC0gbnJvdyhkYXRhKQpkYXRhX25vX291dGxpZXJzX2NvdW50IDwtIG5yb3coZGF0YV9ub19vdXRsaWVycykKCmRhdGFfY291bnQKZGF0YV9ub19vdXRsaWVyc19jb3VudAoKI051bWJlciBvZiBvYnNlcnZhdGlvbiByZW1vdmVkCmRhdGFfY291bnQgLSBkYXRhX25vX291dGxpZXJzX2NvdW50CnRhYmxlKGRhdGFfbm9fb3V0bGllcnMkY29yb25hX2NyaXNpcykKCmBgYAoKPGJyPgoKKipOb3RlOioqIE5hbWVzIG9mIHZhcmlhYmxlcyBhcmUgb24gdGhlIGxlZnQsIGRhdGEgd2l0aCBvdXRsaWVycyBpcyBvbiB0aGUgbGVmdCwgYW5kIG91dGxpZXJzIHJlbW92ZWQgYXJlIG9uIHRoZSByaWdodC4KYGBge3IgY29sbGFwc2U9RkFMU0UsIGF0dHIub3V0cHV0PSdzdHlsZT0ibWF4LWhlaWdodDogMzUwcHg7Iid9CiMgc29sZF9wcmljZQp4IDwtIGRhdGEkc29sZF9wcmljZQoKUTEgPC0gcXVhbnRpbGUoeCwgcHJvYnM9LjI1KQpRMyA8LSBxdWFudGlsZSh4LCBwcm9icz0uNzUpCmlxciA9IFEzLVExCnVwcGVyX2xpbWl0ID0gUTMgKyAoaXFyKjEuNSkKbG93ZXJfbGltaXQgPSBRMSAtIChpcXIqMS41KQpkYXRhX25vX291dGxpZXJzIDwtIHN1YnNldChkYXRhLCAoeCA8PSB1cHBlcl9saW1pdCkgJiAoeCA+PSBsb3dlcl9saW1pdCkpCgojIHBob3RvX2NvdW50CnggPC0gZGF0YV9ub19vdXRsaWVycyRwaG90b19jb3VudAoKUTEgPC0gcXVhbnRpbGUoeCwgcHJvYnM9LjI1KQpRMyA8LSBxdWFudGlsZSh4LCBwcm9icz0uNzUpCmlxciA9IFEzLVExCnVwcGVyX2xpbWl0ID0gUTMgKyAoaXFyKjEuNSkKbG93ZXJfbGltaXQgPSBRMSAtIChpcXIqMS41KQpkYXRhX25vX291dGxpZXJzIDwtIHN1YnNldChkYXRhX25vX291dGxpZXJzLCAoeCA8PSB1cHBlcl9saW1pdCkgJiAoeCA+PSBsb3dlcl9saW1pdCkpCgoKcGFyKG1mcm93PWMoMiwgMikpCmJveHBsb3QoZGF0YSRzb2xkX3ByaWNlLCB4bGFiID0gIldpdGggT3V0bGllcnMiLCB5bGFiID0gInNvbGRfcHJpY2UiKQpib3hwbG90KGRhdGFfbm9fb3V0bGllcnMkc29sZF9wcmljZSwgeGxhYiA9ICJXaXRob3V0IE91dGxpZXJzIikKYm94cGxvdChkYXRhJHBob3RvX2NvdW50LCB4bGFiID0gIldpdGggT3V0bGllcnMiLCB5bGFiID0gInBob3RvX2NvdW50IikKYm94cGxvdChkYXRhX25vX291dGxpZXJzJHBob3RvX2NvdW50LCB4bGFiID0iV2l0aG91dCBPdXRsaWVycyIpCgpgYGAKCgpgYGB7cn0KIyBsaXZpbmdfYXJlYQp4IDwtIGRhdGFfbm9fb3V0bGllcnMkbGl2aW5nX2FyZWEKClExIDwtIHF1YW50aWxlKHgsIHByb2JzPS4yNSkKUTMgPC0gcXVhbnRpbGUoeCwgcHJvYnM9Ljc1KQppcXIgPSBRMy1RMQp1cHBlcl9saW1pdCA9IFEzICsgKGlxcioxLjUpCmxvd2VyX2xpbWl0ID0gUTEgLSAoaXFyKjEuNSkKZGF0YV9ub19vdXRsaWVycyA8LSBzdWJzZXQoZGF0YV9ub19vdXRsaWVycywgKHggPD0gdXBwZXJfbGltaXQpICYgKHggPj0gbG93ZXJfbGltaXQpKQoKIyB0b3RhbF9hcmVhCnggPC0gZGF0YV9ub19vdXRsaWVycyR0b3RhbF9hcmVhCgpRMSA8LSBxdWFudGlsZSh4LCBwcm9icz0uMjUpClEzIDwtIHF1YW50aWxlKHgsIHByb2JzPS43NSkKaXFyID0gUTMtUTEKdXBwZXJfbGltaXQgPSBRMyArIChpcXIqMS41KQpsb3dlcl9saW1pdCA9IFExIC0gKGlxcioxLjUpCmRhdGFfbm9fb3V0bGllcnMgPC0gc3Vic2V0KGRhdGFfbm9fb3V0bGllcnMsICh4IDw9IHVwcGVyX2xpbWl0KSAmICh4ID49IGxvd2VyX2xpbWl0KSkKCgpwYXIobWZyb3cgPSBjKDIsMikpCmJveHBsb3QoZGF0YSRsaXZpbmdfYXJlYSwgeGxhYiA9ICJXaXRoIE91dGxpZXJzIiwgeWxhYiA9ICJsaXZpbmdfYXJlYSIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyRsaXZpbmdfYXJlYSwgeGxhYiA9ICJXaXRob3V0IE91dGxpZXJzIikKYm94cGxvdChkYXRhJHRvdGFsX2FyZWEsIHhsYWIgPSAiV2l0aCBPdXRsaWVycyIsIHlsYWIgPSAidG90YWxfYXJlYSIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyR0b3RhbF9hcmVhLCB4bGFiID0gIldpdGhvdXQgT3V0bGllcnMiKQpgYGAKCgpgYGB7cn0KIyBsYW5kX2FjcmVzCnggPC0gZGF0YV9ub19vdXRsaWVycyRsYW5kX2FjcmVzCgpRMSA8LSBxdWFudGlsZSh4LCBwcm9icz0uMjUpClEzIDwtIHF1YW50aWxlKHgsIHByb2JzPS43NSkKaXFyID0gUTMtUTEKdXBwZXJfbGltaXQgPSBRMyArIChpcXIqMS41KQpsb3dlcl9saW1pdCA9IFExIC0gKGlxcioxLjUpCmRhdGFfbm9fb3V0bGllcnMgPC0gc3Vic2V0KGRhdGFfbm9fb3V0bGllcnMsICh4IDw9IHVwcGVyX2xpbWl0KSAmICh4ID49IGxvd2VyX2xpbWl0KSkKCiMgYWdlCnggPC0gZGF0YV9ub19vdXRsaWVycyRhZ2UKClExIDwtIHF1YW50aWxlKHgsIHByb2JzPS4yNSkKUTMgPC0gcXVhbnRpbGUoeCwgcHJvYnM9Ljc1KQppcXIgPSBRMy1RMQp1cHBlcl9saW1pdCA9IFEzICsgKGlxciozLjUpCmxvd2VyX2xpbWl0ID0gUTEgLSAoaXFyKjMuNSkKZGF0YV9ub19vdXRsaWVycyA8LSBzdWJzZXQoZGF0YV9ub19vdXRsaWVycywgKHggPD0gdXBwZXJfbGltaXQpICYgKHggPj0gbG93ZXJfbGltaXQpKQoKcGFyKG1mcm93ID0gYygyLCAyKSkKYm94cGxvdChkYXRhJGxhbmRfYWNyZXMsIHhsYWIgPSAiV2l0aCBPdXRsaWVycyIsIHlsYWIgPSAibGFuZF9hY3JlcyIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyRsYW5kX2FjcmVzLCB4bGFiID0gIldpdGhvdXQgT3V0bGllcnMiKQpib3hwbG90KGRhdGEkYWdlLCB4bGFiID0gIldpdGggT3V0bGllcnMiLCB5bGFiID0gImFnZSIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyRhZ2UsIHhsYWIgPSAiV2l0aG91dCBPdXRsaWVycyIpCgoKYGBgCgoKYGBge3J9CiMgZGF5c19vbl9tYXJrZXQKeCA8LSBkYXRhX25vX291dGxpZXJzJGRheXNfb25fbWFya2V0CgpRMSA8LSBxdWFudGlsZSh4LCBwcm9icz0uMjUpClEzIDwtIHF1YW50aWxlKHgsIHByb2JzPS43NSkKaXFyID0gUTMtUTEKdXBwZXJfbGltaXQgPSBRMyArIChpcXIqMS41KQpsb3dlcl9saW1pdCA9IFExIC0gKGlxcioxLjUpCmRhdGFfbm9fb3V0bGllcnMgPC0gc3Vic2V0KGRhdGFfbm9fb3V0bGllcnMsICh4IDw9IHVwcGVyX2xpbWl0KSAmICh4ID49IGxvd2VyX2xpbWl0KSkKCiMgc3Rvcmllcwp4IDwtIGRhdGFfbm9fb3V0bGllcnMkc3RvcmllcwoKUTEgPC0gcXVhbnRpbGUoeCwgcHJvYnM9LjI1KQpRMyA8LSBxdWFudGlsZSh4LCBwcm9icz0uNzUpCmlxciA9IFEzLVExCnVwcGVyX2xpbWl0ID0gUTMgKyAoaXFyKjEuNSkKbG93ZXJfbGltaXQgPSBRMSAtIChpcXIqMS41KQpkYXRhX25vX291dGxpZXJzIDwtIHN1YnNldChkYXRhX25vX291dGxpZXJzLCAoeCA8PSAxNCkgJiAoeCA+PSAwKSkKCnBhcihtZnJvdyA9IGMoMiwgMikpCmJveHBsb3QoZGF0YSRkYXlzX29uX21hcmtldCwgeGxhYiA9ICJXaXRoIE91dGxpZXJzIiwgeWxhYiA9ICJkYXlzX29uX21hcmtldCIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyRkYXlzX29uX21hcmtldCwgeGxhYiA9ICJXaXRob3V0IE91dGxpZXJzIikKYm94cGxvdChkYXRhJHN0b3JpZXMsIHhsYWIgPSAiV2l0aCBPdXRsaWVycyIsIHlsYWIgPSAic3RvcmllcyIpCmJveHBsb3QoZGF0YV9ub19vdXRsaWVycyRzdG9yaWVzLCB4bGFiID0gIldpdGhvdXQgT3V0bGllcnMiKQoKYGBgCgoKKipOb3RlOioqIHRoZSBmb2xsb2l3bmcgdmFyaWFibGVzJyBvdXRsaWVycyB3ZXJlIGRlbHQgd2l0aCBpbiBFeGNlbAoKICogYmVkc190b3RhbAogKiBiYXRoX2Z1bGwKICogYmF0aF9oYWxmCgo8YnI+PGJyPgoKCiMjIyMjICoqNC4gQmFzZSBPTFM6IE5vIE91dGxpZXJzKioKCldlIG5vdyBjYW4gc2VlIHRoZSByZXN1bHRzIG9mIHJlbW92aW5nIG91dGxpZXJzIG9uIG91ciBtb3N0IGdlbmVyYWwgbW9kZWwuCgpUaGVzZSBkaWZmZXJlbmNlcyBhcmUgcGFydGljdWxhcnkgY2xlYXIgd2hlbiBjb21wYXJpbmcgdGhlIHR3byBncm91cGluZ3Mgb2YgZm91ciBkaWFnbm9zdGljIHBsb3RzIGF0IHRoZSBlbmQgb2YgdGhpcyByZWdyZXNzaW9uIHRvIHNlZSB0aGUgcmVkdWN0aW9uIGluIGJvdGggZXh0cmVtZSByZXNpZHVhbCB2YWx1ZXMgYW5kIGhpZ2gtbGV2ZXJhZ2UgcG9pbnRzLgoKYGBge3IsIGF0dHIub3V0cHV0PSdzdHlsZT0ibWF4LWhlaWdodDogMzUwcHg7Iid9CmxtX2NsZWFuIDwtIGxtKHNvbGRfcHJpY2UgfiAuIC1tbHNfbnVtYmVyIAogICAgICAgICAgICAgICAtIHNjaG9vbF9oaWdoIC0gc2Nob29sX2p1bmlvciAKICAgICAgICAgICAgICAgLSBzY2hvb2xfbWlkZGxlIC0gcm9vZl90eXBlX3NoaW5nbGVfYmkKICAgICAgICAgICAgICAgLSBzb2xkX2RhdGUgLSB0b3RhbF9hcmVhIC1jb3JvbmFfY3Jpc2lzCiAgICAgICAgICAgICAgIC0gcG9zdF9jb3JvbmFfYmksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBkYXRhX25vX291dGxpZXJzKQpzdW1tYXJ5KGxtX2NsZWFuKQpgYGAKCjxicj4KCiMjIyMjICoqNC4xIE9MUyBSZXZpZXc6IEV4Y2x1ZGluZyBPdXRsaWVycyoqCmBgYHtyfQpwYXIobWZyb3cgPSBjKDIsMikpCnBsb3QobG1fY2xlYW4pCmBgYAo8YnI+CgojIyMjIyAqKjQuMiBPTFMgUmV2aWV3OiBFeGNsdWRpbmcgT3V0bGllcnMqKgpgYGB7cn0KcGFyKG1mcm93ID0gYygyLDIpKQpwbG90KGxtX3JhdykKYGBgCgo8YnI+CgoKIyMjIyAqKjUuIFN1YnNldCBBbmFseXNpcyoqCgoqKkhlcmUgSSB0ZXN0IHRoZSBmb2xsb3dpbmc6KioKCjEuIElzIHByZSBhbmQgcG9zdC1jb3JvbmEgZGF0YSBzaWduaWZpY2FudGx5IGRpZmZlcmVudD8KICAgKyBJIHRlc3QgdGhpcyBieSB0ZXN0aW5nIHRoZSBzaWduaWZpY2FuY2Ugb2YgYSAnY29yb25hJyBkdW1teSB2YXJpYWJsZSAoMSBmb3IgcG9zdC1jb3JvbmEgYW5kIDAgZm9yIHByZS1jb3JvbmEpCjIuIEFyZSB0aGVyZSBzaWduaWZpY2FudCBkaWZmZXJlbmNlcyBpbiB0aGUgd2F5IGtleSB2YXJpYWJsZXMgKGUuZy4gcHJlbWl1bSBmb3IgYmVkcm9vbXMgb3IgJmJldGE7PHN1Yj5iZWRzX3RvdGFsPC9zdWI+KSBhcmUgaW1wYWN0ZWQgYnkgdGhlIENvcm9uYSBjcmlzaXM/CiAgICsgSSBmb2xsb3cgYSBtZXRob2Qgc3VnZ2VzdGVkIGJ5IFVDTEEncyBzdGF0aXN0aWNzIGRlcGFydG1lbnQgd2hpY2ggeW91IGNhbiByZWFkIHRocm91Z2ggWyoqaGVyZSoqXShodHRwczovL3N0YXRzLmlkcmUudWNsYS5lZHUvc2FzL2ZhcS9ob3ctY2FuLWktY29tcGFyZS1yZWdyZXNzaW9uLWNvZWZmaWNpZW50cy1hY3Jvc3MtdGhyZWUtb3ItbW9yZS1ncm91cHMvKSBpZiB5b3Ugd2lzaAoKPGJyPgoKIyMjIyMgKio1LjEgRHVtbXktVmFyaWFibGUgUmVncmVzc2lvbjogQ29yb25hID0gMSoqCgpXZSBzZWUgdGhhdCBiZWxvbmdpbmcgdG8gdGhlICdwb3N0LWNvcm9uYScgc3Vic2V0IGlzIGFzc29jaWF0ZWQgb24gYXZlcmFnZSB3aXRoIGEgbWFya2V0ICoqcHJpY2UgcHJlbWl1bSBvZiArJDI2LDc1MCoqLCBjZXRlcmlzIHBhcmFidXMuIFRoaXMgZmluZGluZyBpcyBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IHdpdGggKipwLXZhbHVlIDwgMC4wMCoqIHVuZGVyIHRoZSBtb3N0IGNvbnRyb2xsZWQgbW9kZWwuIEluIG90aGVyIHdvcmRzLCBvbmUgY291bGQgYWxzbyBzYXk6Cgo+IEEgaG91c2Ugd2l0aCBzaW1pbGFyIHBoeXNpY2FsIGZlYXR1cmVzIHNvbGQgKiphZnRlciB0aGUgZmlyc3QgbWFuZGF0ZWQgQ09WSURfMTkgbG9ja2Rvd24qKiB3aWxsLCBvbiBhdmVyYWdlLCBzZWxsIGZvciAqKiQyNiw3NTAgbW9yZSoqIHRoYW4gaXQgd291bGQgaGF2ZSBiZWZvcmUgdGhpcyBldmVudC4gCgpgYGB7ciwgYXR0ci5vdXRwdXQ9J3N0eWxlPSJtYXgtaGVpZ2h0OiAzNTBweDsiJ30KbG1fY29yb25hX3Rlc3QgPC0gbG0oc29sZF9wcmljZSB+IHBvc3RfY29yb25hX2JpICsgLiAKICAgICAgICAgICAgICAgICAgICAtbWxzX251bWJlciAKICAgICAgICAgICAgICAgICAgICAtIHNjaG9vbF9oaWdoIC0gc2Nob29sX2p1bmlvciAKICAgICAgICAgICAgICAgICAgICAtIHNjaG9vbF9taWRkbGUgLSByb29mX3R5cGVfc2hpbmdsZV9iaQogICAgICAgICAgICAgICAgICAgIC0gc29sZF9kYXRlIC0gdG90YWxfYXJlYSAtY29yb25hX2NyaXNpcywgCiAgICAgICAgICAgICAgICAgICAgZGF0YSA9IGRhdGFfbm9fb3V0bGllcnMpCnN1bW1hcnkobG1fY29yb25hX3Rlc3QpCiNSZXN1bHQ6IFRoZSBhZGRpdGlvbiBvZiB0aGUgdmFyaWFibGVzIHBvc3RfY29yb25hLCBsaXZpbmdfYXJlYSwgYW5kIGxpdmluZ19hcmVhKnBvc3RfY29yb25hCiMgICAgICAgIHJlc3VsdHMgaW4gdGhlIGludGVyYWN0aW9uIHRlcm0gKGJldGFfbGl2aW5nX2FyZWFfcG9zdF9jb3JvbmEgLSBiZXRhX2xpdmluZ19hcmVhX3ByZV9jb3JvbmEpIAojICAgICAgICBiZWluZyBzaWduaWZpY2FudGx5IHBvc2l0aXZlLgojICAgICAgICBOdWxsIGh5cG90aGVzaXMgaXMgcmVqZWN0ZWQgKGkuZS4gY29lZnMgYXJlIHNpZ25pZmljYW50bHkgZGlmZmVyZW50KQpgYGAKCmBgYHtyfQpwYXIobWZyb3cgPSBjKDIsMikpCnBsb3QobG1fY29yb25hX3Rlc3QpCmBgYAoKPGJyPgoKIyMjIyMgKio1LjIgVGVzdGluZyBEaWZmZXJlbmNlcyBvZiBTcGVjaWZpYyBWYXJpYWJsZXMgUG9zdCBhbmQgUHJlLUNvcm9uYSoqCgpUaGlzIFsqKlVDTEEgbWV0aG9kKipdKGh0dHBzOi8vc3RhdHMuaWRyZS51Y2xhLmVkdS9zYXMvZmFxL2hvdy1jYW4taS1jb21wYXJlLXJlZ3Jlc3Npb24tY29lZmZpY2llbnRzLWFjcm9zcy10aHJlZS1vci1tb3JlLWdyb3Vwcy8pIGlzIGltcGxpbWVudGVkIHdpdGggdGhlIGZvbGxvd2luZyBzdGVwczoKCjEuIFJ1biBhIHJlZHVjZWQgcmVncmVzc2lvbiBtb2RlbCB3aXRoIG9ubHk6CiAgICsgc3Vic2V0IGR1bW15IHZhcmlhYmxlIChlLmcuIGNvcm9uYSBkdW1teSkKICAgKyB2YXJpYWJsZSBvZiBpbnRlcmVzdCAoZS5nLiBiZWRzX3RvdGFsKQogICArIGludGVyYWN0aW9uIHRlcm06IChlLmcuIGNvcm9uYV9kdW1teSpiZWRzX3RvdGFsKQogICAKKipOb3RlOioqIFRoZSByZXN1bHRzIG9mIHRoZSBzaWduaWZpY2FuY2UgbGV2ZWwgZm9yIHRoZSBpbnRlcmFjdGlvbiB0ZXJtJ3MgYmV0YSBjb2VmZmljaWVudCB0ZXN0cyB0aGUgbnVsbCBoeXBvdGhlc2lzOiAqKkg8c3ViPjA8L3N1Yj46ICZiZXRhOzxzdWI+cHJlPC9zdWI+ID0gJmJldGE7PHN1Yj5wb3N0PC9zdWI+LioqCgo8YnI+CgojIyMjIyAqKjUuMyBFeGFtcGxlOiBOdW1iZXIgb2YgQmVkcm9vbXMqKgoKQXMgeW91IGNhbiBzZWUsIHRoZSBpbnRlcmFjdGlvbiB0ZXJtIChpLmUuIGJlZHNfdG90YWwqcG9zdF9jb3JvbmFfYmkpIGlzIHN0YXRpc3RpY2FsbHkgc2lnbmlmaWNhbnQgYXQgdGhlIDAuMDAgbGV2ZWwsIHN1Z2dlc3Rpbmcgd2UgcmVqZWN0IHRoZSBudWxsIGh5cG90aGVzaXMgdGhhdCAmYmV0YTs8c3ViPnByZTwvc3ViPiA9ICZiZXRhOzxzdWI+cG9zdDwvc3ViPi4KClRoaXMgcmVzdWx0IGNhbiBhbHNvIGJlIGludGVycHJldGVkIGFzOgoKPiAqVGhlIGF2ZXJhZ2UgcHJlbWl1bSBmb3IgZWFjaCBhZGRpdGlvbmFsIGJlZHJvb20gc2lnbmlmaWNhbnRseSBpbmNyZWFzZWQgYWZ0ZXIgdGhlIG91dGJyZWFrIG9mIENPVklELTE5IGluIE1hcmNoIDIzLCAyMDIwLCByZXN1bHRpbmcgaW4gYW4gKiphdmVyYWdlIHByZW1pdW0gaW5jcmVhc2Ugb2YgJDgsNTgzIHBlciByb29tKiosIGNldGVyaXMgcGFyYWJ1cy4qCgo+ICpNeSAodGhlIGF1dGhvcidzKSBoeXBvdGhlc2lzIGlzIHRoYXQgdGhpcyByZXN1bHQgaXMgbGlrZWx5IGRyaXZlbiBieSBhbiBpbmNyZWFzZWQgZGVtYW5kIGZvciBhbiBhZGRpdGlvbmFsIGJlZHJvb20gZnVuY3Rpb25pbmcgYXMgYSAqKidob21lIG9mZmljZScqKiBhcyBtYW55IHdvcmtlcnMgd2VyZSBsZWdhbGx5IGNvbXBlbGxlZCB0byB3b3JrIGZyb20gaG9tZSwgd2l0aCBtYW55IG9mIHRoZW0gKipuZXZlciByZXR1cm5pbmcqKiB0byBmdWxsLXRpbWUsIG9uLXNpdGUgd29ya2luZyBzY2hlZHVsZXMuKgoKYGBge3IsIGF0dHIub3V0cHV0PSdzdHlsZT0ibWF4LWhlaWdodDogMzUwcHg7Iid9CiMgVGVzdCBpZiBiZWRzX3RvdGFsIGlzIHNpZ25pZmljYW50bHkgZGlmZmVyZW50IHBvc3RfY29yb25hIHZzIHByZV9jb3JvbmEKbG1fYmVkc19jb3JvbmFfdGVzdCA8LSBsbShzb2xkX3ByaWNlIH4gcG9zdF9jb3JvbmFfYmkgKyBiZWRzX3RvdGFsICsgYmVkc190b3RhbCpwb3N0X2Nvcm9uYV9iaSwgZGF0YSA9IGRhdGFfbm9fb3V0bGllcnMpCnN1bW1hcnkobG1fYmVkc19jb3JvbmFfdGVzdCkKI1Jlc3VsdDogVGhlIGFkZGl0aW9uIG9mIHRoZSB2YXJpYWJsZXMgcG9zdF9jb3JvbmEsIGJlZHNfdG90YWwsIGFuZCBsaXZpbmdfYXJlYSpwb3N0X2Nvcm9uYQojICAgICAgICByZXN1bHRzIGluIHRoZSBpbnRlcmFjdGlvbiB0ZXJtIChiZXRhX2xpdmluZ19hcmVhX3Bvc3RfY29yb25hIC0gYmV0YV9saXZpbmdfYXJlYV9wcmVfY29yb25hKSAKIyAgICAgICAgYmVpbmcgc2lnbmlmaWNhbnRseSBwb3NpdGl2ZS4KIyAgICAgICAgTnVsbCBoeXBvdGhlc2lzIGlzIHJlamVjdGVkIChpLmUuIGNvZWZzIGFyZSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCkKYGBgCgpgYGB7cn0KcGFyKG1mcm93ID0gYygyLDIpKQpwbG90KGxtX2JlZHNfY29yb25hX3Rlc3QpCmBgYAoKCjxicj48YnI+CgoKCiMjIyMgKio2LiBRdWVzdGlvbnMgZm9yIFlvdSBhbmQgTmV4dCBTdGVwcyoqCgo8YnI+CgojIyMjIyAqKjYuMSBRdWVzdGlvbnMqKgoKMS4gQXJlIHlvdSBpbiBhZ3JlZW1lbnQgd2l0aCB0aGUgY3VycmVudCBtZXRob2Qgb2YgY29tcGFyaW5nIGJldGEgY29lZmZpY2llbnRzIGFjcm9zcyBtb2RlbHM/CiAgICsgSWYgeW91IGFyZSwgdGhlbiBJIHdpbGwgY29udGludWUgYnkgcmVwZWF0aW5nIHRoZSBwcm9jZXNzIGFjcm9zcyBzZXZlcmFsIGtleSB2YXJpYWJsZXMgYW5kIHdpdGggdGhlIG90aGVyIHN1YnNldHMgd2UndmUgYWxyZWFkeSBkaXNjdXNzZWQgKGUuZy4gdG9wIHZzIGJvdHRvbSAyNXRoIHBlcmNlbnRpbGUgb2Ygc2l6ZSwgcHJpY2UsIGV0Yy4pCjIuIERvIHlvdSBoYXZlIGFueSBzdWdnZXNpb25zIGZvciBzcGVjaWZpYyB2YXJpYWJsZSB0cmFuc2Zvcm1hdGlvbnMgaW4gdGhlc2UgT0xTIG1vZGVscyAoZS5nLiBsb2cocHJpY2UpICk/CiAgICsgSSB3aWxsIG1ha2UgbWFueSB0cmFuc2Zvcm1hdGlvbiBmb3IgdGhlIE1MIHByaWRpdGlvbiBtb2RlbHMsIGJ1dCB3YW50ZWQgdG8gZ2V0IHlvdXIgb3BpbmlvbiBoZXJlLgozLiBEbyB5b3UgaGF2ZSBhbnkgb3RoZXIgZXh0ZW50aW9ucyBvciBtb2RlbHMgd2hpY2ggY291bGQgbWFrZSB0aGlzIGFuYWx5c2lzIHJpdGNoZXIsIG1vcmUgcm9idXN0PwogICArIEJlY2F1c2UgdGhpcyAnY29yZScgcG9ydGlvbiBvZiB0aGUgYW5hbHlzaXMgd2VudCBzbyB3ZWxsLCBJJ20gaW50ZXJlc3RlZCBpbiBwdXNoaW5nIHRoZSBsaW1pdCBvZiBhY2FkZW1pYyB0ZWNobmljYWxsaXR5IGZvciB0aGUgc2FrZSBvZiB0aGUgcGFwZXIgYW5kIHRoZSBncmFkaW5nIHByb2Nlc3MgZG93biB0aGUgcm9hZC4KICAgCjxicj4KCiMjIyMjICoqNi4yIE5leHQgU3RlcHMqKgoKMS4gSSBhbSBhYm91dCA2MCUgb2YgdGhlIHdheSB0aHJvdWdoIGNvZGluZyB0aGUgTUwgbW9kZWxzIGFuZCB3aWxsIGxpa2VseSBtYWtlIGFub3RoZXIgUk1hcmtkb3duIGZpbGUgbGlrZSB0aGlzIG9uZSBmb3IgdGhhdCBwcm9jZXNzIGFzIHdlbGwKMi4gSSBhbSBjdXJyZW50bHkgY29sbGVjdGluZyBzcGVjaWZpYyBwYXBlcnMgZm9yIGEgbGl0ZXJhdHVyZSByZXZpZXcgYW5kIGFsc28gcHVsbGluZyBvdGhlciAnc29mdCcgZmlndXJlcyBhYm91dCB0aGUgY3Jpc2lzIGl0c2VsZiBpbiBMb3Vpc2lhbmEgZm9yIG5hcnJhdGl2ZSBhbmQgY29udGV4dC4KMy4gT25jZSBJIGhhdmUgY29tcGxldGVkIHRoZSBNTCBtb2RlbCBhbmQgaGF2ZSBjb21iaW5lZCB0aGUgZmluZGluZyBpbnRvIG9uZSwgc3VjY2luY3QgbGluZSBvZiBhbmFseXNpcywgSSBob3BlIHRvIGhhdmUgYSByZXZpZXcgbWVldGluZyB3aXRoIHlvdSB0byBkZXNjdXNzIGltcHJvdmVtZW50cy4KCjxicj48YnI+PGJyPgoKCiMjIyMgKipGaW5hbCBTdGF0ZW1lbnQqKgojIyMjIyBBcyBhbHdheXMsIHRoYW5rcyBmb3IgdGFraW5nIHRoZSB0aW1lIHRvIHJlYWQgdGhyb3VnaCB0aGlzIGFuZCBmb3IgeW91ciBndWlkZW5jZSEKCjxicj4KCiMjIyMjIEFsbCB0aGUgYmVzdCwKIyMjIyMgU2F3eWVyIAoKCjxicj48YnI+PGJyPjxicj48YnI+PGJyPgoKKkVuZCBvZiBEb2N1bWVudCoKCg==