1. Why this notebook?

This is an R Notebook created using R Studio Cloud. It illustrates several functionalities to obtain, process and visualize digital elevation models in R. It aims to help Geomatica Basica students at Universidad Nacional with their home activities during this time of social distancing. Everybody should publish a similar notebook, adapted to cover any municipality of your department (the one you like most), not later than on 2nd April 2020.

2. Introduction to elevatr

Elevation data is used for a wide array of applications, including, for example, visualization, hydrology, and ecological modelling. Gaining access to these data in R has not had a single interface, is made available through functions across many packages, or requires local access to the data. This is no longer required as a variety of APIs now exist that provide programmatic access to elevation data. The elevatr package was written to standarize access to elevation data from web APIs.

To access raster elevation data (e.g. a DEM) the elevatr package uses the Amazon Web Services Terrain Tiles. We will explore this functionality in this notebook.

3. Get Raster Elevation Data

There are several sources for digital elevation models such as the Shuttle Radar Topography Mission (SRTM), the USGS National Elevation Dataset (NED), Global DEM (GDEM), and others. Each of these DEMs has pros and cons for their use. Prior to its closure in January of 2018, Mapzen combined several of these sources to create a synthesis elevation product that utilizes the best available elevation data for a given region at given zoom level. Although closed, these data compiled by Mapzen are currently accessible through the Terrain Tiles on Amazon Web Services (AWS).

The input for get_elev_raster() is a data.frame with x (longitude) and y (latitude) locations as the first two columns, any sp object, or any raster object and it returns a RasterLayer of the tiles that overlap the bounding box of the input. If multiple tiles are retrieved, the resultant output is a merged Raster Layer.

Using get_elev_raster() to access the Terrain Tiles on AWS.

As mentioned a data frame with x and y columns, a sp object, or a raster object needs be the input and the src needs to be set to “mapzen” (this is the default).

There is no difference in using the sp and raster input data types. The data frame requires a prj. I will show examples using a SpatialPolygonsDataFrame. The zoom level (z) defaults to 9 (a trade off between resolution and time for download), but I could not get anything using this zoom. I had to use a lower zoom equal to 8. Otherwise, the RStudio Cloud notebook would crash again and again. Very often, I had to “reload” the page and start it again.

First, we need to load a shapefile representing municipalities of our department. In this notebook, we will load the shapefile downloaded from the DANE’s geoportal as requested in the virtual class on 19 March 2020. This shapefile was previously uploaded to RStudio Cloud under a folder named valledelcauca.

list.files("/cloud/project/valledelcauca/ADMINISTRATIVO")
 [1] "MGN_DPTO_POLITICO.cpg"     "MGN_DPTO_POLITICO.dbf"    
 [3] "MGN_DPTO_POLITICO.prj"     "MGN_DPTO_POLITICO.sbn"    
 [5] "MGN_DPTO_POLITICO.sbx"     "MGN_DPTO_POLITICO.shp"    
 [7] "MGN_DPTO_POLITICO.shp.xml" "MGN_DPTO_POLITICO.shx"    
 [9] "MGN_MPIO_POLITICO.cpg"     "MGN_MPIO_POLITICO.dbf"    
[11] "MGN_MPIO_POLITICO.prj"     "MGN_MPIO_POLITICO.sbn"    
[13] "MGN_MPIO_POLITICO.sbx"     "MGN_MPIO_POLITICO.shp"    
[15] "MGN_MPIO_POLITICO.shp.xml" "MGN_MPIO_POLITICO.shx"    

Now, let’s read the shapefile using a function provided by the raster package:

(munic <-  shapefile("/cloud/project/valledelcauca/ADMINISTRATIVO/MGN_MPIO_POLITICO.shp"))
class       : SpatialPolygonsDataFrame 
features    : 42 
extent      : -77.54977, -75.70724, 3.091239, 5.047394  (xmin, xmax, ymin, ymax)
crs         : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 
variables   : 9
names       : DPTO_CCDGO, MPIO_CCDGO, MPIO_CNMBR,                    MPIO_CRSLC,    MPIO_NAREA, MPIO_NANO,      DPTO_CNMBR,     Shape_Leng,       Shape_Area 
min values  :         76,      76001,     ALCALÁ,                          1536,   41.86090736,      2017, VALLE DEL CAUCA, 0.453826056161, 0.00340901158098 
max values  :         76,      76895,     ZARZAL, Ordenanza 9 de Diciembre 1954, 6292.50083741,      2017, VALLE DEL CAUCA,  6.59527269127,   0.510778519244 

What are the attributes of the munic object:

head(munic)

Let’s select only the capital city of this department.

cali <- munic[munic$MPIO_CNMBR=="CALI",]
plot(cali, main="", axes=TRUE)
plot(munic, add=TRUE)
invisible(text(coordinates(munic), labels=as.character(munic$MPIO_CNMBR), cex=0.8))

elevation <-get_elev_raster(cali, z = 8)

Downloading DEMs [===================>-------]  75% eta:  0s
Downloading DEMs [===========================] 100% eta:  0s
Merging DEMs
Reprojecting DEM to original projection
no non-missing arguments to min; returning Infno non-missing arguments to max; returning -InfNote: Elevation units are in meters.
Note: The coordinate reference system is:
 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0

As I have already downloaded and saved the elevation data, I need to read it using the raster function provided by the raster package.

Now, let’s check what is inside the elevation object:

elevation
class      : RasterLayer 
dimensions : 1035, 1033, 1069155  (nrow, ncol, ncell)
resolution : 0.00275, 0.00274  (x, y)
extent     : -77.3575, -74.51675, 1.392743, 4.228643  (xmin, xmax, ymin, ymax)
crs        : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 
source     : memory
names      : layer 
values     : -598.1536, 5331.562  (min, max)
plot(elevation, main="This the downloaded DEM [meters]")
plot(cali, add=TRUE)

4. Crop the elevation data to match the study area extent

Notice that the DEM covers a larger extent than the one we need. This is due to the spatial arrangement of the elevation tiles in AWS.

Anyway, it is a good idea to save the DEM for the future.

Now, let’s crop the elevation data corresponding to Cali:

elev_crop = crop(elevation, cali)
plot(elev_crop, main="Cropped Digital elevation model")
plot(cali, add=TRUE)

Let’s check the new object:

elev_crop
class      : RasterLayer 
dimensions : 101, 90, 9090  (nrow, ncol, ncell)
resolution : 0.00275, 0.00274  (x, y)
extent     : -76.70575, -76.45825, 3.272383, 3.549123  (xmin, xmax, ymin, ymax)
crs        : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 
source     : memory
names      : layer 
values     : 944.2498, 3979.236  (min, max)

5. Reproject the elevation data

geographic coordinates, horizontal dimensions units are decimal degrees BUT the vertical dimension unit is meters. Let’s reproject the elevation data.

We can go to epsg.io and look for the MAGNA Colombia Bogota zone projection. We need to get the definition for this spatial reference in PROJ.4 format (the one used for the sp and raster libraries. Let’s copy the PROJ.4 text and save it.

spatialref <- "+proj=tmerc +lat_0=4.596200416666666 +lon_0=-74.07750791666666 +k=1 +x_0=1000000 +y_0=1000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs"

Now, we can reproject the elevation data from WGS84 geographic coordinates into MAGNA Colombia Bogota zone.

# Project Raster
# to have control,  we create one raster object
# using projectExtent (no values are transferred)
pr3 <- projectExtent(elev_crop, spatialref)
# Adjust the cell size 
res(pr3) <- 100
# now project
rep_elev <- projectRaster(elev_crop, pr3)
no non-missing arguments to min; returning Infno non-missing arguments to max; returning -Inf
rep_elev
class      : RasterLayer 
dimensions : 307, 276, 84732  (nrow, ncol, ncell)
resolution : 100, 100  (x, y)
extent     : 707796.8, 735396.8, 853929.3, 884629.3  (xmin, xmax, ymin, ymax)
crs        : +proj=tmerc +lat_0=4.596200416666666 +lon_0=-74.07750791666666 +k=1 +x_0=1000000 +y_0=1000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs 
source     : memory
names      : layer 
values     : 943.4611, 3963.625  (min, max)

Now, let’s reproject the SpatialPolygonsDataFrame representing the capital of our department:

(rep_cali = spTransform(cali,spatialref))
class       : SpatialPolygonsDataFrame 
features    : 1 
extent      : 707694.8, 735285.7, 854051.1, 884460.6  (xmin, xmax, ymin, ymax)
crs         : +proj=tmerc +lat_0=4.596200416666666 +lon_0=-74.07750791666666 +k=1 +x_0=1000000 +y_0=1000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs 
variables   : 9
names       : DPTO_CCDGO, MPIO_CCDGO, MPIO_CNMBR, MPIO_CRSLC,   MPIO_NAREA, MPIO_NANO,      DPTO_CNMBR,    Shape_Leng,      Shape_Area 
value       :         76,      76001,       CALI,       1536, 563.04276445,      2017, VALLE DEL CAUCA, 1.16366974876, 0.0457332109324 

It is plotting time:

plot(rep_elev, main="Reprojected Digital elevation model")
plot(rep_cali, add=TRUE)

5. Basic statistics of elevation data

A quick exploration of the DEM statistics:

# histograma
hist(rep_elev)

# works for large files
promedio <- cellStats(rep_elev, 'mean')
minimo <- cellStats(rep_elev, 'min')
maximo <- cellStats(rep_elev, 'max')
desviacion  <- cellStats(rep_elev, 'sd')
# create unidimensional vector
metricas <- c('mean', 'min', 'max', 'std')
valores <- c(promedio, minimo, maximo, desviacion)
# creacion de data frame con estadisticas de elevacion [metros]
# el paréntesis externo sirve para "imprimir" el contenido de un objeto
(df_estadisticas <- data.frame(metricas, valores))

6. Obtention of geomorphometric variables

Before proceeding, make sure you have read the geomorphometry chapter written by Victor Olaya for his libro libre en sistemas de informacion geografica.

First, compute slope, aspect, and hillshade:

slope = terrain(rep_elev,opt='slope', unit='degrees')
aspect = terrain(rep_elev,opt='aspect',unit='degrees')
hill = hillShade(slope,aspect,40,315)

Plot elevation. Note the color palette used here.

plot(rep_elev,main="DEM for CALI [meters]", col=terrain.colors(25,alpha=0.7))

Plot slope. Note the color palette used here.

plot(slope,main="Slope for CALI [degrees]", col=topo.colors(25,alpha=0.7))

Plot aspect. Note the color palette used here.

plot(aspect,main="Aspect for CALI [degrees]", col=rainbow(25,alpha=0.7))

In case you are interested, read about color palettes in R.

A combined plot:

# plot DEM w/ hillshade
plot(hill,
        col=grey(1:100/100),  # create a color ramp of grey colors for hillshade
        legend=FALSE,         # no legend, we don't care about the grey of the hillshade
        main="DEM for CALI",
        axes=FALSE)           # makes for a cleaner plot, if the coordinates aren't necessary

plot(rep_elev, 
        axes=FALSE,
        col=terrain.colors(12, alpha=0.35), add=TRUE) # color method

7. Mapping elevation data with rayshader

The library rayshader is an open source package for producing 2D and 3D data visualizations in R. rayshader uses elevation data in a base R matrix and a combination of raytracing, spherical texture mapping, overlays, and ambient occlusion to generate beautiful topographic 2D and 3D maps. In addition to maps, rayshader also allows the user to translate ggplot2 objects into beautiful 3D data visualizations.

#Convert the DEM into a matrix:
elmat = raster_to_matrix(rep_elev)
[1] "Dimensions of matrix are: 276x307."
#We use another one of rayshader's built-in textures:
elmat %>%
  sphere_shade(texture = "imhof2") %>%
  plot_map()

Calculating Surface Normal [-----------------------------------] ETA:  0s
Calculating Surface Normal [===================================] ETA:  0s
                                                                         

#detect_water and add_water adds a water layer to the map:
elmat %>%
  sphere_shade(texture = "desert") %>%
  add_water(detect_water(elmat), color = "desert") %>%
  plot_map()

Calculating Surface Normal [-----------------------------------] ETA:  0s
Calculating Surface Normal [===================================] ETA:  0s
                                                                         

#And we can add a raytraced layer from that sun direction as well:
elmat %>%
  sphere_shade(texture = "desert") %>%
  add_water(detect_water(elmat), color = "desert") %>%
  add_shadow(ray_shade(elmat), 0.5) %>%
  plot_map()

Calculating Surface Normal [-----------------------------------] ETA:  0s
Calculating Surface Normal [===================================] ETA:  0s
                                                                         

Raytracing [---------------------------------------------------] ETA:  0s
Raytracing [===================================================] ETA:  0s
                                                                         

8. Another way of visualization

An R expert suggest another way of visualization here.

Let’s try it:

# Spherical environment mapping hillshade function
getv=function(i,a,s){
  ct = dim(i)[1:2]/2
  sx = values(s)/90 * ct[1]
  sy = values(s)/90 * ct[2]
  a = values(a) * 0.01745
  px = floor(ct[1] + sx * -sin(a))
  py = floor(ct[2] + sy * cos(a))
  
  
  template = brick(s,s,s)
  values(template)=NA
  
  cellr = px + py * ct[1]*2
  cellg = px + py * ct[1]*2 + (ct[1]*2*ct[2]*2)
  cellb = px + py * ct[1]*2 + 2*(ct[1]*2*ct[2]*2)
  
  template[[1]] = i[cellr]
  template[[2]] = i[cellg]
  template[[3]] = i[cellb]
  
  template = template * 256
  
  template
}
# Load an environment map image
# An example can be found at http://i.imgur.com/9pvbHjN.jpg
map=readJPEG("./valledelcauca/9pvbHjN.jpg")

# Do the mapping
out = getv(map, aspect, slope)
no non-missing arguments to min; returning Infno non-missing arguments to max; returning -Infno non-missing arguments to min; returning Infno non-missing arguments to max; returning -Infno non-missing arguments to min; returning Infno non-missing arguments to max; returning -Infno non-missing arguments to min; returning Infno non-missing arguments to max; returning -Infno non-missing arguments to min; returning Infno non-missing arguments to max; returning -Infno non-missing arguments to min; returning Infno non-missing arguments to max; returning -Inf
# Plot pretty mountains
plotRGB(out, main = "Supposedly pretty mountains in Medellin")

LS0tCnRpdGxlOiAiRWxldmF0aW9uIGRhdGEgaW4gUiIKYXV0aG9yOiAiTmljb2xhcyBDaWZ1ZW50ZXMiCmRhdGU6ICIzMS4wMy4yMDIwIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCiMjIDEuIFdoeSB0aGlzIG5vdGVib29rPwpUaGlzIGlzIGFuIFIgTm90ZWJvb2sgY3JlYXRlZCB1c2luZyBSIFN0dWRpbyBDbG91ZC4gSXQgaWxsdXN0cmF0ZXMgc2V2ZXJhbCBmdW5jdGlvbmFsaXRpZXMgdG8gb2J0YWluLCBwcm9jZXNzIGFuZCB2aXN1YWxpemUgZGlnaXRhbCBlbGV2YXRpb24gbW9kZWxzIGluIFIuIEl0IGFpbXMgdG8gaGVscCBHZW9tYXRpY2EgQmFzaWNhIHN0dWRlbnRzIGF0IFVuaXZlcnNpZGFkIE5hY2lvbmFsIHdpdGggdGhlaXIgaG9tZSBhY3Rpdml0aWVzIGR1cmluZyB0aGlzIHRpbWUgb2Ygc29jaWFsIGRpc3RhbmNpbmcuIEV2ZXJ5Ym9keSBzaG91bGQgcHVibGlzaCBhIHNpbWlsYXIgbm90ZWJvb2ssIGFkYXB0ZWQgdG8gY292ZXIgYW55IG11bmljaXBhbGl0eSBvZiB5b3VyIGRlcGFydG1lbnQgKHRoZSBvbmUgeW91IGxpa2UgbW9zdCksIG5vdCBsYXRlciB0aGFuIG9uIDJuZCBBcHJpbCAyMDIwLgoKIyMgMi4gSW50cm9kdWN0aW9uIHRvIGVsZXZhdHIKRWxldmF0aW9uIGRhdGEgaXMgdXNlZCBmb3IgYSB3aWRlIGFycmF5IG9mIGFwcGxpY2F0aW9ucywgaW5jbHVkaW5nLCBmb3IgZXhhbXBsZSwgdmlzdWFsaXphdGlvbiwgaHlkcm9sb2d5LCBhbmQgZWNvbG9naWNhbCBtb2RlbGxpbmcuIEdhaW5pbmcgYWNjZXNzIHRvIHRoZXNlIGRhdGEgaW4gUiBoYXMgbm90IGhhZCBhIHNpbmdsZSBpbnRlcmZhY2UsIGlzIG1hZGUgYXZhaWxhYmxlIHRocm91Z2ggZnVuY3Rpb25zIGFjcm9zcyBtYW55IHBhY2thZ2VzLCBvciByZXF1aXJlcyBsb2NhbCBhY2Nlc3MgdG8gdGhlIGRhdGEuIFRoaXMgaXMgbm8gbG9uZ2VyIHJlcXVpcmVkIGFzIGEgdmFyaWV0eSBvZiBBUElzIG5vdyBleGlzdCB0aGF0IHByb3ZpZGUgcHJvZ3JhbW1hdGljIGFjY2VzcyB0byBlbGV2YXRpb24gZGF0YS4gVGhlIGVsZXZhdHIgcGFja2FnZSB3YXMgd3JpdHRlbiB0byBzdGFuZGFyaXplIGFjY2VzcyB0byBlbGV2YXRpb24gZGF0YSBmcm9tIHdlYiBBUElzLgoKVG8gYWNjZXNzIHJhc3RlciBlbGV2YXRpb24gZGF0YSAoZS5nLiBhIERFTSkgdGhlIGVsZXZhdHIgcGFja2FnZSB1c2VzIHRoZSBBbWF6b24gV2ViIFNlcnZpY2VzIFRlcnJhaW4gVGlsZXMuIFdlIHdpbGwgZXhwbG9yZSB0aGlzIGZ1bmN0aW9uYWxpdHkgaW4gdGhpcyBub3RlYm9vay4KCiMjIDMuIEdldCBSYXN0ZXIgRWxldmF0aW9uIERhdGEKVGhlcmUgYXJlIHNldmVyYWwgc291cmNlcyBmb3IgZGlnaXRhbCBlbGV2YXRpb24gbW9kZWxzIHN1Y2ggYXMgdGhlIFNodXR0bGUgUmFkYXIgVG9wb2dyYXBoeSBNaXNzaW9uIChTUlRNKSwgdGhlIFVTR1MgTmF0aW9uYWwgRWxldmF0aW9uIERhdGFzZXQgKE5FRCksIEdsb2JhbCBERU0gKEdERU0pLCBhbmQgb3RoZXJzLiBFYWNoIG9mIHRoZXNlIERFTXMgaGFzIHByb3MgYW5kIGNvbnMgZm9yIHRoZWlyIHVzZS4gUHJpb3IgdG8gaXRzIGNsb3N1cmUgaW4gSmFudWFyeSBvZiAyMDE4LCBNYXB6ZW4gY29tYmluZWQgc2V2ZXJhbCBvZiB0aGVzZSBzb3VyY2VzIHRvIGNyZWF0ZSBhIHN5bnRoZXNpcyBlbGV2YXRpb24gcHJvZHVjdCB0aGF0IHV0aWxpemVzIHRoZSBiZXN0IGF2YWlsYWJsZSBlbGV2YXRpb24gZGF0YSBmb3IgYSBnaXZlbiByZWdpb24gYXQgZ2l2ZW4gem9vbSBsZXZlbC4gQWx0aG91Z2ggY2xvc2VkLCB0aGVzZSBkYXRhIGNvbXBpbGVkIGJ5IE1hcHplbiBhcmUgY3VycmVudGx5IGFjY2Vzc2libGUgdGhyb3VnaCB0aGUgVGVycmFpbiBUaWxlcyBvbiBBbWF6b24gV2ViIFNlcnZpY2VzIChBV1MpLgoKVGhlIGlucHV0IGZvciBnZXRfZWxldl9yYXN0ZXIoKSBpcyBhIGRhdGEuZnJhbWUgd2l0aCB4IChsb25naXR1ZGUpIGFuZCB5IChsYXRpdHVkZSkgbG9jYXRpb25zIGFzIHRoZSBmaXJzdCB0d28gY29sdW1ucywgYW55IHNwIG9iamVjdCwgb3IgYW55IHJhc3RlciBvYmplY3QgYW5kIGl0IHJldHVybnMgYSBSYXN0ZXJMYXllciBvZiB0aGUgdGlsZXMgdGhhdCBvdmVybGFwIHRoZSBib3VuZGluZyBib3ggb2YgdGhlIGlucHV0LiBJZiBtdWx0aXBsZSB0aWxlcyBhcmUgcmV0cmlldmVkLCB0aGUgcmVzdWx0YW50IG91dHB1dCBpcyBhIG1lcmdlZCBSYXN0ZXIgTGF5ZXIuCgojIyBVc2luZyBnZXRfZWxldl9yYXN0ZXIoKSB0byBhY2Nlc3MgdGhlIFRlcnJhaW4gVGlsZXMgb24gQVdTLgpBcyBtZW50aW9uZWQgYSBkYXRhIGZyYW1lIHdpdGggeCBhbmQgeSBjb2x1bW5zLCBhIHNwIG9iamVjdCwgb3IgYSByYXN0ZXIgb2JqZWN0IG5lZWRzIGJlIHRoZSBpbnB1dCBhbmQgdGhlIHNyYyBuZWVkcyB0byBiZSBzZXQgdG8g4oCcbWFwemVu4oCdICh0aGlzIGlzIHRoZSBkZWZhdWx0KS4KClRoZXJlIGlzIG5vIGRpZmZlcmVuY2UgaW4gdXNpbmcgdGhlIHNwIGFuZCByYXN0ZXIgaW5wdXQgZGF0YSB0eXBlcy4gVGhlIGRhdGEgZnJhbWUgcmVxdWlyZXMgYSBwcmouIEkgd2lsbCBzaG93IGV4YW1wbGVzIHVzaW5nIGEgU3BhdGlhbFBvbHlnb25zRGF0YUZyYW1lLiBUaGUgem9vbSBsZXZlbCAoeikgZGVmYXVsdHMgdG8gOSAoYSB0cmFkZSBvZmYgYmV0d2VlbiByZXNvbHV0aW9uIGFuZCB0aW1lIGZvciBkb3dubG9hZCksIGJ1dCBJIGNvdWxkIG5vdCBnZXQgYW55dGhpbmcgdXNpbmcgdGhpcyB6b29tLiBJIGhhZCB0byB1c2UgYSBsb3dlciB6b29tIGVxdWFsIHRvIDguIE90aGVyd2lzZSwgdGhlIFJTdHVkaW8gQ2xvdWQgbm90ZWJvb2sgd291bGQgY3Jhc2ggYWdhaW4gYW5kIGFnYWluLiBWZXJ5IG9mdGVuLCBJIGhhZCB0byDigJxyZWxvYWTigJ0gdGhlIHBhZ2UgYW5kIHN0YXJ0IGl0IGFnYWluLgoKRmlyc3QsIHdlIG5lZWQgdG8gbG9hZCBhIHNoYXBlZmlsZSByZXByZXNlbnRpbmcgbXVuaWNpcGFsaXRpZXMgb2Ygb3VyIGRlcGFydG1lbnQuIEluIHRoaXMgbm90ZWJvb2ssIHdlIHdpbGwgbG9hZCB0aGUgc2hhcGVmaWxlIGRvd25sb2FkZWQgZnJvbSB0aGUgREFOReKAmXMgZ2VvcG9ydGFsIGFzIHJlcXVlc3RlZCBpbiB0aGUgdmlydHVhbCBjbGFzcyBvbiAxOSBNYXJjaCAyMDIwLiBUaGlzIHNoYXBlZmlsZSB3YXMgcHJldmlvdXNseSB1cGxvYWRlZCB0byBSU3R1ZGlvIENsb3VkIHVuZGVyIGEgZm9sZGVyIG5hbWVkIHZhbGxlZGVsY2F1Y2EuCgpgYGB7cn0KbGlzdC5maWxlcygiL2Nsb3VkL3Byb2plY3QvdmFsbGVkZWxjYXVjYS9BRE1JTklTVFJBVElWTyIpCmBgYAoKTm93LCBsZXTigJlzIHJlYWQgdGhlIHNoYXBlZmlsZSB1c2luZyBhIGZ1bmN0aW9uIHByb3ZpZGVkIGJ5IHRoZSByYXN0ZXIgcGFja2FnZToKCmBgYHtyfQoobXVuaWMgPC0gIHNoYXBlZmlsZSgiL2Nsb3VkL3Byb2plY3QvdmFsbGVkZWxjYXVjYS9BRE1JTklTVFJBVElWTy9NR05fTVBJT19QT0xJVElDTy5zaHAiKSkKYGBgCldoYXQgYXJlIHRoZSBhdHRyaWJ1dGVzIG9mIHRoZSBtdW5pYyBvYmplY3Q6CmBgYHtyfQpoZWFkKG11bmljKQpgYGAKCkxldOKAmXMgc2VsZWN0IG9ubHkgdGhlIGNhcGl0YWwgY2l0eSBvZiB0aGlzIGRlcGFydG1lbnQuCgpgYGB7cn0KY2FsaSA8LSBtdW5pY1ttdW5pYyRNUElPX0NOTUJSPT0iQ0FMSSIsXQpwbG90KGNhbGksIG1haW49IiIsIGF4ZXM9VFJVRSkKcGxvdChtdW5pYywgYWRkPVRSVUUpCmludmlzaWJsZSh0ZXh0KGNvb3JkaW5hdGVzKG11bmljKSwgbGFiZWxzPWFzLmNoYXJhY3RlcihtdW5pYyRNUElPX0NOTUJSKSwgY2V4PTAuOCkpCmBgYAoKCmBgYHtyfQplbGV2YXRpb24gPC1nZXRfZWxldl9yYXN0ZXIoY2FsaSwgeiA9IDgpCmBgYAoKQXMgSSBoYXZlIGFscmVhZHkgZG93bmxvYWRlZCBhbmQgc2F2ZWQgdGhlIGVsZXZhdGlvbiBkYXRhLCBJIG5lZWQgdG8gcmVhZCBpdCB1c2luZyB0aGUgcmFzdGVyIGZ1bmN0aW9uIHByb3ZpZGVkIGJ5IHRoZSByYXN0ZXIgcGFja2FnZS4gCgpOb3csIGxldOKAmXMgY2hlY2sgd2hhdCBpcyBpbnNpZGUgdGhlIGVsZXZhdGlvbiBvYmplY3Q6CgpgYGB7cn0KZWxldmF0aW9uCmBgYAoKYGBge3J9CnBsb3QoZWxldmF0aW9uLCBtYWluPSJUaGlzIHRoZSBkb3dubG9hZGVkIERFTSBbbWV0ZXJzXSIpCnBsb3QoY2FsaSwgYWRkPVRSVUUpCmBgYAoKIyMgNC4gQ3JvcCB0aGUgZWxldmF0aW9uIGRhdGEgdG8gbWF0Y2ggdGhlIHN0dWR5IGFyZWEgZXh0ZW50Ck5vdGljZSB0aGF0IHRoZSBERU0gY292ZXJzIGEgbGFyZ2VyIGV4dGVudCB0aGFuIHRoZSBvbmUgd2UgbmVlZC4gVGhpcyBpcyBkdWUgdG8gdGhlIHNwYXRpYWwgYXJyYW5nZW1lbnQgb2YgdGhlIGVsZXZhdGlvbiB0aWxlcyBpbiBBV1MuCgpBbnl3YXksIGl0IGlzIGEgZ29vZCBpZGVhIHRvIHNhdmUgdGhlIERFTSBmb3IgdGhlIGZ1dHVyZS4KCk5vdywgbGV04oCZcyBjcm9wIHRoZSBlbGV2YXRpb24gZGF0YSBjb3JyZXNwb25kaW5nIHRvIENhbGk6CgpgYGB7cn0KZWxldl9jcm9wID0gY3JvcChlbGV2YXRpb24sIGNhbGkpCnBsb3QoZWxldl9jcm9wLCBtYWluPSJDcm9wcGVkIERpZ2l0YWwgZWxldmF0aW9uIG1vZGVsIikKcGxvdChjYWxpLCBhZGQ9VFJVRSkKYGBgCgpMZXTigJlzIGNoZWNrIHRoZSBuZXcgb2JqZWN0OgoKYGBge3J9CmVsZXZfY3JvcApgYGAKCiMjIDUuIFJlcHJvamVjdCB0aGUgZWxldmF0aW9uIGRhdGEKZ2VvZ3JhcGhpYyBjb29yZGluYXRlcywgaG9yaXpvbnRhbCBkaW1lbnNpb25zIHVuaXRzIGFyZSBkZWNpbWFsIGRlZ3JlZXMgQlVUIHRoZSB2ZXJ0aWNhbCBkaW1lbnNpb24gdW5pdCBpcyBtZXRlcnMuIExldOKAmXMgcmVwcm9qZWN0IHRoZSBlbGV2YXRpb24gZGF0YS4KCldlIGNhbiBnbyB0byBlcHNnLmlvIGFuZCBsb29rIGZvciB0aGUgTUFHTkEgQ29sb21iaWEgQm9nb3RhIHpvbmUgcHJvamVjdGlvbi4gV2UgbmVlZCB0byBnZXQgdGhlIGRlZmluaXRpb24gZm9yIHRoaXMgc3BhdGlhbCByZWZlcmVuY2UgaW4gUFJPSi40IGZvcm1hdCAodGhlIG9uZSB1c2VkIGZvciB0aGUgc3AgYW5kIHJhc3RlciBsaWJyYXJpZXMuIExldOKAmXMgY29weSB0aGUgUFJPSi40IHRleHQgYW5kIHNhdmUgaXQuCgpgYGB7cn0Kc3BhdGlhbHJlZiA8LSAiK3Byb2o9dG1lcmMgK2xhdF8wPTQuNTk2MjAwNDE2NjY2NjY2ICtsb25fMD0tNzQuMDc3NTA3OTE2NjY2NjYgK2s9MSAreF8wPTEwMDAwMDAgK3lfMD0xMDAwMDAwICtlbGxwcz1HUlM4MCArdG93Z3M4ND0wLDAsMCwwLDAsMCwwICt1bml0cz1tICtub19kZWZzIgpgYGAKCk5vdywgd2UgY2FuIHJlcHJvamVjdCB0aGUgZWxldmF0aW9uIGRhdGEgZnJvbSBXR1M4NCBnZW9ncmFwaGljIGNvb3JkaW5hdGVzIGludG8gTUFHTkEgQ29sb21iaWEgQm9nb3RhIHpvbmUuCgpgYGB7cn0KIyBQcm9qZWN0IFJhc3RlcgojIHRvIGhhdmUgY29udHJvbCwgIHdlIGNyZWF0ZSBvbmUgcmFzdGVyIG9iamVjdAojIHVzaW5nIHByb2plY3RFeHRlbnQgKG5vIHZhbHVlcyBhcmUgdHJhbnNmZXJyZWQpCnByMyA8LSBwcm9qZWN0RXh0ZW50KGVsZXZfY3JvcCwgc3BhdGlhbHJlZikKIyBBZGp1c3QgdGhlIGNlbGwgc2l6ZSAKcmVzKHByMykgPC0gMTAwCiMgbm93IHByb2plY3QKcmVwX2VsZXYgPC0gcHJvamVjdFJhc3RlcihlbGV2X2Nyb3AsIHByMykKYGBgCgpgYGB7cn0KcmVwX2VsZXYKYGBgCgpOb3csIGxldOKAmXMgcmVwcm9qZWN0IHRoZSBTcGF0aWFsUG9seWdvbnNEYXRhRnJhbWUgcmVwcmVzZW50aW5nIHRoZSBjYXBpdGFsIG9mIG91ciBkZXBhcnRtZW50OgpgYGB7cn0KKHJlcF9jYWxpID0gc3BUcmFuc2Zvcm0oY2FsaSxzcGF0aWFscmVmKSkKYGBgCgpJdCBpcyBwbG90dGluZyB0aW1lOgoKYGBge3J9CnBsb3QocmVwX2VsZXYsIG1haW49IlJlcHJvamVjdGVkIERpZ2l0YWwgZWxldmF0aW9uIG1vZGVsIikKcGxvdChyZXBfY2FsaSwgYWRkPVRSVUUpCmBgYAoKIyMgNS4gQmFzaWMgc3RhdGlzdGljcyBvZiBlbGV2YXRpb24gZGF0YQpBIHF1aWNrIGV4cGxvcmF0aW9uIG9mIHRoZSBERU0gc3RhdGlzdGljczoKCmBgYHtyfQojIGhpc3RvZ3JhbWEKaGlzdChyZXBfZWxldikKYGBgCgpgYGB7cn0KIyB3b3JrcyBmb3IgbGFyZ2UgZmlsZXMKcHJvbWVkaW8gPC0gY2VsbFN0YXRzKHJlcF9lbGV2LCAnbWVhbicpCm1pbmltbyA8LSBjZWxsU3RhdHMocmVwX2VsZXYsICdtaW4nKQptYXhpbW8gPC0gY2VsbFN0YXRzKHJlcF9lbGV2LCAnbWF4JykKZGVzdmlhY2lvbiAgPC0gY2VsbFN0YXRzKHJlcF9lbGV2LCAnc2QnKQpgYGAKCmBgYHtyfQojIGNyZWF0ZSB1bmlkaW1lbnNpb25hbCB2ZWN0b3IKbWV0cmljYXMgPC0gYygnbWVhbicsICdtaW4nLCAnbWF4JywgJ3N0ZCcpCnZhbG9yZXMgPC0gYyhwcm9tZWRpbywgbWluaW1vLCBtYXhpbW8sIGRlc3ZpYWNpb24pCmBgYAoKYGBge3J9CiMgY3JlYWNpb24gZGUgZGF0YSBmcmFtZSBjb24gZXN0YWRpc3RpY2FzIGRlIGVsZXZhY2lvbiBbbWV0cm9zXQojIGVsIHBhcsOpbnRlc2lzIGV4dGVybm8gc2lydmUgcGFyYSAiaW1wcmltaXIiIGVsIGNvbnRlbmlkbyBkZSB1biBvYmpldG8KKGRmX2VzdGFkaXN0aWNhcyA8LSBkYXRhLmZyYW1lKG1ldHJpY2FzLCB2YWxvcmVzKSkKYGBgCiMjIDYuIE9idGVudGlvbiBvZiBnZW9tb3JwaG9tZXRyaWMgdmFyaWFibGVzCkJlZm9yZSBwcm9jZWVkaW5nLCBtYWtlIHN1cmUgeW91IGhhdmUgcmVhZCB0aGUgZ2VvbW9ycGhvbWV0cnkgY2hhcHRlciB3cml0dGVuIGJ5IFZpY3RvciBPbGF5YSBmb3IgaGlzIGxpYnJvIGxpYnJlIGVuIHNpc3RlbWFzIGRlIGluZm9ybWFjaW9uIGdlb2dyYWZpY2EuCgpGaXJzdCwgY29tcHV0ZSBzbG9wZSwgYXNwZWN0LCBhbmQgaGlsbHNoYWRlOgoKYGBge3J9CnNsb3BlID0gdGVycmFpbihyZXBfZWxldixvcHQ9J3Nsb3BlJywgdW5pdD0nZGVncmVlcycpCmFzcGVjdCA9IHRlcnJhaW4ocmVwX2VsZXYsb3B0PSdhc3BlY3QnLHVuaXQ9J2RlZ3JlZXMnKQpoaWxsID0gaGlsbFNoYWRlKHNsb3BlLGFzcGVjdCw0MCwzMTUpCmBgYAoKUGxvdCBlbGV2YXRpb24uIE5vdGUgdGhlIGNvbG9yIHBhbGV0dGUgdXNlZCBoZXJlLgoKYGBge3J9CnBsb3QocmVwX2VsZXYsbWFpbj0iREVNIGZvciBDQUxJIFttZXRlcnNdIiwgY29sPXRlcnJhaW4uY29sb3JzKDI1LGFscGhhPTAuNykpCmBgYAoKUGxvdCBzbG9wZS4gTm90ZSB0aGUgY29sb3IgcGFsZXR0ZSB1c2VkIGhlcmUuCgpgYGB7cn0KcGxvdChzbG9wZSxtYWluPSJTbG9wZSBmb3IgQ0FMSSBbZGVncmVlc10iLCBjb2w9dG9wby5jb2xvcnMoMjUsYWxwaGE9MC43KSkKYGBgCgpQbG90IGFzcGVjdC4gTm90ZSB0aGUgY29sb3IgcGFsZXR0ZSB1c2VkIGhlcmUuCgpgYGB7cn0KcGxvdChhc3BlY3QsbWFpbj0iQXNwZWN0IGZvciBDQUxJIFtkZWdyZWVzXSIsIGNvbD1yYWluYm93KDI1LGFscGhhPTAuNykpCmBgYApJbiBjYXNlIHlvdSBhcmUgaW50ZXJlc3RlZCwgcmVhZCBhYm91dCBjb2xvciBwYWxldHRlcyBpbiBSLgoKQSBjb21iaW5lZCBwbG90OgoKYGBge3J9CiMgcGxvdCBERU0gdy8gaGlsbHNoYWRlCnBsb3QoaGlsbCwKICAgICAgICBjb2w9Z3JleSgxOjEwMC8xMDApLCAgIyBjcmVhdGUgYSBjb2xvciByYW1wIG9mIGdyZXkgY29sb3JzIGZvciBoaWxsc2hhZGUKICAgICAgICBsZWdlbmQ9RkFMU0UsICAgICAgICAgIyBubyBsZWdlbmQsIHdlIGRvbid0IGNhcmUgYWJvdXQgdGhlIGdyZXkgb2YgdGhlIGhpbGxzaGFkZQogICAgICAgIG1haW49IkRFTSBmb3IgQ0FMSSIsCiAgICAgICAgYXhlcz1GQUxTRSkgICAgICAgICAgICMgbWFrZXMgZm9yIGEgY2xlYW5lciBwbG90LCBpZiB0aGUgY29vcmRpbmF0ZXMgYXJlbid0IG5lY2Vzc2FyeQoKcGxvdChyZXBfZWxldiwgCiAgICAgICAgYXhlcz1GQUxTRSwKICAgICAgICBjb2w9dGVycmFpbi5jb2xvcnMoMTIsIGFscGhhPTAuMzUpLCBhZGQ9VFJVRSkgIyBjb2xvciBtZXRob2QKYGBgCiMjIDcuIE1hcHBpbmcgZWxldmF0aW9uIGRhdGEgd2l0aCByYXlzaGFkZXIKVGhlIGxpYnJhcnkgcmF5c2hhZGVyIGlzIGFuIG9wZW4gc291cmNlIHBhY2thZ2UgZm9yIHByb2R1Y2luZyAyRCBhbmQgM0QgZGF0YSB2aXN1YWxpemF0aW9ucyBpbiBSLiByYXlzaGFkZXIgdXNlcyBlbGV2YXRpb24gZGF0YSBpbiBhIGJhc2UgUiBtYXRyaXggYW5kIGEgY29tYmluYXRpb24gb2YgcmF5dHJhY2luZywgc3BoZXJpY2FsIHRleHR1cmUgbWFwcGluZywgb3ZlcmxheXMsIGFuZCBhbWJpZW50IG9jY2x1c2lvbiB0byBnZW5lcmF0ZSBiZWF1dGlmdWwgdG9wb2dyYXBoaWMgMkQgYW5kIDNEIG1hcHMuIEluIGFkZGl0aW9uIHRvIG1hcHMsIHJheXNoYWRlciBhbHNvIGFsbG93cyB0aGUgdXNlciB0byB0cmFuc2xhdGUgZ2dwbG90MiBvYmplY3RzIGludG8gYmVhdXRpZnVsIDNEIGRhdGEgdmlzdWFsaXphdGlvbnMuCgpgYGB7cn0KI0NvbnZlcnQgdGhlIERFTSBpbnRvIGEgbWF0cml4OgplbG1hdCA9IHJhc3Rlcl90b19tYXRyaXgocmVwX2VsZXYpCmBgYAoKYGBge3J9CiNXZSB1c2UgYW5vdGhlciBvbmUgb2YgcmF5c2hhZGVyJ3MgYnVpbHQtaW4gdGV4dHVyZXM6CmVsbWF0ICU+JQogIHNwaGVyZV9zaGFkZSh0ZXh0dXJlID0gImltaG9mMiIpICU+JQogIHBsb3RfbWFwKCkKYGBgCmBgYHtyfQojZGV0ZWN0X3dhdGVyIGFuZCBhZGRfd2F0ZXIgYWRkcyBhIHdhdGVyIGxheWVyIHRvIHRoZSBtYXA6CmVsbWF0ICU+JQogIHNwaGVyZV9zaGFkZSh0ZXh0dXJlID0gImRlc2VydCIpICU+JQogIGFkZF93YXRlcihkZXRlY3Rfd2F0ZXIoZWxtYXQpLCBjb2xvciA9ICJkZXNlcnQiKSAlPiUKICBwbG90X21hcCgpCmBgYApgYGB7cn0KI0FuZCB3ZSBjYW4gYWRkIGEgcmF5dHJhY2VkIGxheWVyIGZyb20gdGhhdCBzdW4gZGlyZWN0aW9uIGFzIHdlbGw6CmVsbWF0ICU+JQogIHNwaGVyZV9zaGFkZSh0ZXh0dXJlID0gImRlc2VydCIpICU+JQogIGFkZF93YXRlcihkZXRlY3Rfd2F0ZXIoZWxtYXQpLCBjb2xvciA9ICJkZXNlcnQiKSAlPiUKICBhZGRfc2hhZG93KHJheV9zaGFkZShlbG1hdCksIDAuNSkgJT4lCiAgcGxvdF9tYXAoKQpgYGAKCiMjIDguIEFub3RoZXIgd2F5IG9mIHZpc3VhbGl6YXRpb24KQW4gUiBleHBlcnQgc3VnZ2VzdCBhbm90aGVyIHdheSBvZiB2aXN1YWxpemF0aW9uIGhlcmUuCgpMZXTigJlzIHRyeSBpdDoKCmBgYHtyfQojIFNwaGVyaWNhbCBlbnZpcm9ubWVudCBtYXBwaW5nIGhpbGxzaGFkZSBmdW5jdGlvbgpnZXR2PWZ1bmN0aW9uKGksYSxzKXsKICBjdCA9IGRpbShpKVsxOjJdLzIKICBzeCA9IHZhbHVlcyhzKS85MCAqIGN0WzFdCiAgc3kgPSB2YWx1ZXMocykvOTAgKiBjdFsyXQogIGEgPSB2YWx1ZXMoYSkgKiAwLjAxNzQ1CiAgcHggPSBmbG9vcihjdFsxXSArIHN4ICogLXNpbihhKSkKICBweSA9IGZsb29yKGN0WzJdICsgc3kgKiBjb3MoYSkpCiAgCiAgCiAgdGVtcGxhdGUgPSBicmljayhzLHMscykKICB2YWx1ZXModGVtcGxhdGUpPU5BCiAgCiAgY2VsbHIgPSBweCArIHB5ICogY3RbMV0qMgogIGNlbGxnID0gcHggKyBweSAqIGN0WzFdKjIgKyAoY3RbMV0qMipjdFsyXSoyKQogIGNlbGxiID0gcHggKyBweSAqIGN0WzFdKjIgKyAyKihjdFsxXSoyKmN0WzJdKjIpCiAgCiAgdGVtcGxhdGVbWzFdXSA9IGlbY2VsbHJdCiAgdGVtcGxhdGVbWzJdXSA9IGlbY2VsbGddCiAgdGVtcGxhdGVbWzNdXSA9IGlbY2VsbGJdCiAgCiAgdGVtcGxhdGUgPSB0ZW1wbGF0ZSAqIDI1NgogIAogIHRlbXBsYXRlCn0KYGBgCgpgYGB7cn0KIyBMb2FkIGFuIGVudmlyb25tZW50IG1hcCBpbWFnZQojIEFuIGV4YW1wbGUgY2FuIGJlIGZvdW5kIGF0IGh0dHA6Ly9pLmltZ3VyLmNvbS85cHZiSGpOLmpwZwptYXA9cmVhZEpQRUcoIi4vdmFsbGVkZWxjYXVjYS85cHZiSGpOLmpwZyIpCgojIERvIHRoZSBtYXBwaW5nCm91dCA9IGdldHYobWFwLCBhc3BlY3QsIHNsb3BlKQoKIyBQbG90IHByZXR0eSBtb3VudGFpbnMKcGxvdFJHQihvdXQsIG1haW4gPSAiU3VwcG9zZWRseSBwcmV0dHkgbW91bnRhaW5zIGluIE1lZGVsbGluIikKYGBgCgoK