Angel Angelov | angelov@tum.de | 2017
based on the Growthcurver package by Sprouffske et. al.
https://www.ncbi.nlm.nih.gov/pubmed/27094401

This tutorial shows how to fit the logistic equation to microbial growth curve data (using the growthcurver package) and how to visualize the growth curves in R.

Part 1. Logistic regression with Growthcurver

Load required libraries, and evt. first install the Growthcurver package by running install.packages("growthcurver").

library(dplyr)
library(reshape2)
library(ggplot2)
library(growthcurver)
library(purrr)


Read in a tab-delimited text file with the following format -> first column is time, second and further columns are the samples (wild type, mutants, treatments…). It is important that the decimal separator is a . Change the code below accordingly if you use , as a separator. Also, if you have hours as time units you will of course get all calculations in hours later (like doubling time and the growth rate constant). If you have replicates, i.e. several measurements for one timepoint, just put them all in the column for the respective sample. In the example file there are for example replicates for the different timepoints, important is that each sample gets one column, in my case “wt”, “ope” and “dole” are the samples.

df <- read.csv(file.choose(), header = TRUE, sep = "\t", dec = ".")


Check how the file looks like after reading in R and take a look at the data points of the growth curve, in this case I plot the wt sample, which is in column 2 of the example file

ggplot(df, aes(x = time, y = wt)) + geom_point(alpha=0.7)

In this case, I am going to use the data up to 40 hours, shoud give a better fit to the logistic function. Followed by a simple call of the growthcurver package

df <- df %>% filter(time<=40)
model.wt <- SummarizeGrowth(df$time, df$wt)

Now the model for the wt sample is written to model.wt. From there, it is easy to obtain all the summary metrics of the fit

model.wt$vals # gives you all the values (the growth rate etc. See the Growthcurver manual for more info)
k   k_se    k_p n0  n0_se   n0_p
8.589   0.149   7e-41   0.047   0.026   8e-02

r   r_se    r_p sigma   df  t_mid
0.415   0.045   1e-11   0.621   41  12.55

t_gen   auc_l   auc_e
1.671   222.782 404.624
predict(model.wt$model) # gives you the predicted OD values (according to the model)
 [1] 0.0468966 3.8071200 5.5485934 6.9319781 7.7780261 8.2154843 8.4221437
 [8] 8.5155988 8.5888496 8.5891807 0.0468966 0.1067389 0.2407905 0.5326241
[15] 1.1302696 2.2142186 3.8071200 8.5155988 8.5630621 8.5800187 8.5866679
[22] 8.5884070 0.0468966 3.8071200 5.5485934 6.9319781 7.7780261 8.2154843
[29] 8.4221437 8.5155988 8.5888496 8.5891807 0.0468966 0.1067389 0.2407905
[36] 0.5326241 1.1302696 2.2142186 3.8071200 8.5155988 8.5630621 8.5800187
[43] 8.5866679 8.5884070

We can see that the logistic regression fit gives a growth rate constant (r) of 0.415 with a SE of 0.045. More info what all the values mean can be found in the Growthcurver vignette.

In the Growthcurver package there is a function for running the fit on many samples, called SummarizeGrowthByPlate. This function writes the values of the fitting in a data.frame, where each row is a sample and the columns are the obtained values. You can easily write this data.table to a *.csv file for further use.

growth.values.plate <- SummarizeGrowthByPlate(df)
write.csv2(growth.values.plate, file="growth-values-plate.tab")

In case you want plots of all the samples, call SummarizeGrowthByPlate with the plot_fit = TRUE argument. Also, I have made some modifications to this function in order to get more statistics about the fit and produce better plots. You can download the modified script , called SummarizeGrowthByPlateAA here and run it.

Part 2. Plotting

Here, you have two options:

  1. The easy one is to just use the basic plotting functions of R, which you can run directly on the generated model:
plot(model.wt)

This is not so nice, and there are not many options to change how it looks.

  1. Now the more interersting part - how to make nice plots with ggplot2. First I will make a plot of the raw data only (without the model fitting) and save it to p1.
p1 <- ggplot(df, aes(x=time,y=wt)) + geom_point(alpha=0.5) + theme_bw()
p1

If you want to put the predicted values from the logistic regression on the plot above, you will have to get them using predict and add them to a new datatable. After that just add another geom for the predicted values.

df.predicted <- data.frame(time = df$time, pred.wt = predict(model.wt$model))
p1 + geom_line(data=df.predicted, aes(y=df.predicted$pred.wt), color="red")

Part 3. Plotting many samples

Here again, two options:

  1. Use the basic R plotting function, i.e. run the SummarizeGrowthByPlate or my modified version SummarizeGrowthByPlateAA with the plot_fit = TRUE argument. A pdf file will be generated with all the plots in your project directory. The plotting here is amenable to modfication, but is somewhat complicated.
  2. If you have a lot of samples and want to do the plotting with ggplot2, you need to obtain the complete model data for each sample so that you can add the predicted values from the model to the plot. This means you have to run SummarizeGrowth on many samples (SummarizeGrowthByPlate will not work here, or at least has to be modified heavily to write the predicted “OD” values somewhere). In order to get the complete models for all samples I am using some lapply magic: first define a function, I name it summG here, and then use it in lapply to run SummarizeGrowth on any number of columns:
summG <- function(x) {SummarizeGrowth(df$time,x)}
lapply(df[2:ncol(df)], summG)
$wt
Fit data to K / (1 + ((K - N0) / N0) * exp(-r * t)): 
    K   N0  r
  val:  8.589   0.047   0.415
  Residual standard error: 0.6207026 on 41 degrees of freedom

Other useful metrics:
  DT    1 / DT  auc_l   auc_e
  1.67  6e-01   222.78  404.62

$ope
Fit data to K / (1 + ((K - N0) / N0) * exp(-r * t)): 
    K   N0  r
  val:  8.897   0.055   0.403
  Residual standard error: 0.7943527 on 41 degrees of freedom

Other useful metrics:
  DT    1 / DT  auc_l   auc_e
  1.72  5.8e-01 230.27  437.38

$dole
Fit data to K / (1 + ((K - N0) / N0) * exp(-r * t)): 
    K   N0  r
  val:  10.731  0.027   0.368
  Residual standard error: 0.5262141 on 41 degrees of freedom

Other useful metrics:
  DT    1 / DT  auc_l   auc_e
  1.88  5.3e-01 238.41  306.65

This needs some explanation, I guess. SummarizeGrowth needs 2 arguments - 1) the name of the column for time, here df$time and 2) the name of the column with the “OD” values for one sample, for example this was df$wt before. In order to do this for many samples, and avoiding for loops typical for some other languages, I am using vectorizing, a very common approach in R. Put shortly, I use lapply to apply a function to all the variables in a dataframe and return the results in a list. So in the lapply call here I am running the function summG for all the variables of the df (without the first one which is the time): df[2:ncol(df). Of course, this can be further improved by defining the summG function within the lapply call, and writing all models to a list like this:

models.all <- lapply(df[2:ncol(df)], function(x) SummarizeGrowth(df$time, x))

Now the models for all samples are written in models.all. We can take the predicted “OD” values from there and add them to a new dataframe called df.predicted.plate. This time I am using a for loop:)

df.predicted.plate <- data.frame(time = df$time)
for (i in names(df[2:ncol(df)])) 
  {df.predicted.plate[[i]] <- predict(models.all[[i]]$model)}

In case there are NA values in the columns, change the last two commands like this:

models.all <- lapply(df[2:ncol(df)], function(x) SummarizeGrowth(df[!is.na(x), 1], x[!is.na(x)]))
df.predicted.plate <- data.frame(time = df$time)
for (i in names(df[2:ncol(df)])) 
  {df.predicted.plate[[i]] <- predict(models.all[[i]]$model, newdata = list(t = df$time))}

Note that in order for predict to work, the new data supplied (time in this case) has to be named like in the model, hence the list(t = df$time).

Then I melt the two dataframes, df (the one with the measured OD values) and df.predicted.plate (the one with the predicted OD values), and merge them together. The melt is required for easier plotting in ggplot2 later.

melt1 <- melt(df, id.vars = "time", variable.name = "sample", value.name = "od")
melt2 <- melt(df.predicted.plate, id.vars = "time", variable.name = "sample", value.name = "pred.od")
df.final <- cbind(melt1, pred.od=melt2[,3])
rm(melt1)
rm(melt2)

Now the plots. Note how I have set the facet_wrap number of columns to 12. This is in case you want to plot a whole plate, will translate to plots organized in 12 columns and 8 rows. You can of course adjust the facet_wrap as you wish with ncol = and nrow=.

ggplot(df.final, aes(x=time, y=od)) + geom_point(aes(), alpha=0.5) + geom_line(aes(y=pred.od), color="red") + facet_wrap(~sample, ncol = 12) + theme_bw()

When you are happy with what comes out, don’t forget to save the plots.

If you want to get a table with all the values for all the samples:

vals_df <- map(models.all, "vals") %>% map_df(bind_rows, .id = "sample")
write.csv(vals_df, file = "output-predicted-values.csv")

Enjoy!

AA

PS As with any model, one can supply new data and let the model predict the outcome. In this imagenary case we can supply new time points and let the model predict the OD values. I used this approach above in order to deal with the NA values.

tt <- seq(0,10, length=66)
predict(model.wt$model,newdata=list(t=tt))
LS0tCnRpdGxlOiAiR3Jvd3RoIGN1cnZlIGFuYWx5c2lzIGFuZCBwbG90dGluZyB3aXRoIFIiCm91dHB1dDoKICBodG1sX25vdGVib29rOgogICAgdGhlbWU6IGZsYXRseQogIGh0bWxfZG9jdW1lbnQ6IGRlZmF1bHQKICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQKLS0tCgpBbmdlbCBBbmdlbG92IHwgYW5nZWxvdkB0dW0uZGUgfCAyMDE3PGJyPgpiYXNlZCBvbiB0aGUgR3Jvd3RoY3VydmVyIHBhY2thZ2UgYnkgU3Byb3VmZnNrZSBldC4gYWwuPGJyPgpodHRwczovL3d3dy5uY2JpLm5sbS5uaWguZ292L3B1Ym1lZC8yNzA5NDQwMQo8YnI+Cjxicj4KVGhpcyB0dXRvcmlhbCBzaG93cyBob3cgdG8gZml0IHRoZSBsb2dpc3RpYyBlcXVhdGlvbiB0byBtaWNyb2JpYWwgZ3Jvd3RoIGN1cnZlIGRhdGEgKHVzaW5nIHRoZSBgZ3Jvd3RoY3VydmVyYCBwYWNrYWdlKSBhbmQgaG93IHRvIHZpc3VhbGl6ZSB0aGUgZ3Jvd3RoIGN1cnZlcyBpbiBSLiA8YnI+CgojUGFydCAxLiBMb2dpc3RpYyByZWdyZXNzaW9uIHdpdGggYEdyb3d0aGN1cnZlcmAKTG9hZCByZXF1aXJlZCBsaWJyYXJpZXMsIGFuZCBldnQuIGZpcnN0IGluc3RhbGwgdGhlIEdyb3d0aGN1cnZlciBwYWNrYWdlIGJ5IHJ1bm5pbmcgYGluc3RhbGwucGFja2FnZXMoImdyb3d0aGN1cnZlciIpYC4KCmBgYHtyfQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KHJlc2hhcGUyKQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkoZ3Jvd3RoY3VydmVyKQpsaWJyYXJ5KHB1cnJyKQpgYGAKPGJyPgpSZWFkIGluIGEgdGFiLWRlbGltaXRlZCB0ZXh0IGZpbGUgd2l0aCB0aGUgZm9sbG93aW5nIGZvcm1hdCAtPiBmaXJzdCBjb2x1bW4gaXMgYHRpbWVgLCBzZWNvbmQgYW5kIGZ1cnRoZXIgY29sdW1ucyBhcmUgdGhlIHNhbXBsZXMgKHdpbGQgdHlwZSwgbXV0YW50cywgdHJlYXRtZW50cy4uLikuIEl0IGlzIGltcG9ydGFudCB0aGF0IHRoZSBkZWNpbWFsIHNlcGFyYXRvciBpcyBhIGAuYCBDaGFuZ2UgdGhlIGNvZGUgYmVsb3cgIGFjY29yZGluZ2x5IGlmIHlvdSB1c2UgYCxgIGFzIGEgc2VwYXJhdG9yLiBBbHNvLCBpZiB5b3UgaGF2ZSBob3VycyBhcyBgdGltZWAgdW5pdHMgeW91IHdpbGwgb2YgY291cnNlIGdldCBhbGwgY2FsY3VsYXRpb25zIGluIGhvdXJzIGxhdGVyIChsaWtlIGRvdWJsaW5nIHRpbWUgYW5kIHRoZSBncm93dGggcmF0ZSBjb25zdGFudCkuIElmIHlvdSBoYXZlIHJlcGxpY2F0ZXMsIGkuZS4gc2V2ZXJhbCBtZWFzdXJlbWVudHMgZm9yIG9uZSB0aW1lcG9pbnQsIGp1c3QgcHV0IHRoZW0gYWxsIGluIHRoZSBjb2x1bW4gZm9yIHRoZSByZXNwZWN0aXZlIHNhbXBsZS4gSW4gdGhlIGV4YW1wbGUgZmlsZSB0aGVyZSBhcmUgZm9yIGV4YW1wbGUgcmVwbGljYXRlcyBmb3IgdGhlIGRpZmZlcmVudCB0aW1lcG9pbnRzLCBpbXBvcnRhbnQgaXMgdGhhdCBlYWNoIHNhbXBsZSBnZXRzIG9uZSBjb2x1bW4sIGluIG15IGNhc2UgInd0IiwgIm9wZSIgYW5kICJkb2xlIiBhcmUgdGhlIHNhbXBsZXMuCiAKYGBge3J9CmRmIDwtIHJlYWQuY3N2KGZpbGUuY2hvb3NlKCksIGhlYWRlciA9IFRSVUUsIHNlcCA9ICJcdCIsIGRlYyA9ICIuIikKYGBgCjxicj4KQ2hlY2sgaG93IHRoZSBmaWxlIGxvb2tzIGxpa2UgYWZ0ZXIgcmVhZGluZyBpbiBSIGFuZCB0YWtlIGEgbG9vayBhdCB0aGUgZGF0YSBwb2ludHMgb2YgdGhlIGdyb3d0aCBjdXJ2ZSwgaW4gdGhpcyBjYXNlIEkgcGxvdCB0aGUgYHd0YCBzYW1wbGUsIHdoaWNoIGlzIGluIGNvbHVtbiAyIG9mIHRoZSBleGFtcGxlIGZpbGUKYGBge3J9CmdncGxvdChkZiwgYWVzKHggPSB0aW1lLCB5ID0gd3QpKSArIGdlb21fcG9pbnQoYWxwaGE9MC43KQpgYGAKSW4gdGhpcyBjYXNlLCBJIGFtIGdvaW5nIHRvIHVzZSB0aGUgZGF0YSB1cCB0byA0MCBob3Vycywgc2hvdWQgZ2l2ZSBhIGJldHRlciBmaXQgdG8gdGhlIGxvZ2lzdGljIGZ1bmN0aW9uLiBGb2xsb3dlZCBieSBhIHNpbXBsZSBjYWxsIG9mIHRoZSBgZ3Jvd3RoY3VydmVyYCBwYWNrYWdlIApgYGB7cn0KZGYgPC0gZGYgJT4lIGZpbHRlcih0aW1lPD00MCkKbW9kZWwud3QgPC0gU3VtbWFyaXplR3Jvd3RoKGRmJHRpbWUsIGRmJHd0KQpgYGAKTm93IHRoZSBtb2RlbCBmb3IgdGhlIGB3dGAgc2FtcGxlIGlzIHdyaXR0ZW4gdG8gYG1vZGVsLnd0YC4gRnJvbSB0aGVyZSwgaXQgaXMgZWFzeSB0byBvYnRhaW4gYWxsIHRoZSBzdW1tYXJ5IG1ldHJpY3Mgb2YgdGhlIGZpdApgYGB7cn0KbW9kZWwud3QkdmFscyAjIGdpdmVzIHlvdSBhbGwgdGhlIHZhbHVlcyAodGhlIGdyb3d0aCByYXRlIGV0Yy4gU2VlIHRoZSBHcm93dGhjdXJ2ZXIgbWFudWFsIGZvciBtb3JlIGluZm8pCnByZWRpY3QobW9kZWwud3QkbW9kZWwpICMgZ2l2ZXMgeW91IHRoZSBwcmVkaWN0ZWQgT0QgdmFsdWVzIChhY2NvcmRpbmcgdG8gdGhlIG1vZGVsKQpgYGAKV2UgY2FuIHNlZSB0aGF0IHRoZSBsb2dpc3RpYyByZWdyZXNzaW9uIGZpdCBnaXZlcyBhIGdyb3d0aCByYXRlIGNvbnN0YW50IChgcmApIG9mIGAwLjQxNWAgd2l0aCBhIGBTRWAgb2YgYDAuMDQ1YC4gTW9yZSBpbmZvIHdoYXQgYWxsIHRoZSB2YWx1ZXMgbWVhbiBjYW4gYmUgZm91bmQgaW4gdGhlIGBHcm93dGhjdXJ2ZXJgIHZpZ25ldHRlLgoKSW4gdGhlIGBHcm93dGhjdXJ2ZXJgIHBhY2thZ2UgdGhlcmUgaXMgYSBmdW5jdGlvbiBmb3IgcnVubmluZyB0aGUgZml0IG9uIG1hbnkgc2FtcGxlcywgY2FsbGVkIGBTdW1tYXJpemVHcm93dGhCeVBsYXRlYC4gVGhpcyBmdW5jdGlvbiB3cml0ZXMgdGhlIHZhbHVlcyBvZiB0aGUgZml0dGluZyBpbiBhIGBkYXRhLmZyYW1lYCwgd2hlcmUgZWFjaCByb3cgaXMgYSBzYW1wbGUgYW5kIHRoZSBjb2x1bW5zIGFyZSB0aGUgb2J0YWluZWQgdmFsdWVzLiBZb3UgY2FuIGVhc2lseSB3cml0ZSB0aGlzIGBkYXRhLnRhYmxlYCB0byBhICouY3N2IGZpbGUgZm9yIGZ1cnRoZXIgdXNlLgpgYGB7cn0KZ3Jvd3RoLnZhbHVlcy5wbGF0ZSA8LSBTdW1tYXJpemVHcm93dGhCeVBsYXRlKGRmKQp3cml0ZS5jc3YyKGdyb3d0aC52YWx1ZXMucGxhdGUsIGZpbGU9Imdyb3d0aC12YWx1ZXMtcGxhdGUudGFiIikKYGBgCkluIGNhc2UgeW91IHdhbnQgcGxvdHMgb2YgYWxsIHRoZSBzYW1wbGVzLCBjYWxsIGBTdW1tYXJpemVHcm93dGhCeVBsYXRlYCB3aXRoIHRoZSBgcGxvdF9maXQgPSBUUlVFYCBhcmd1bWVudC4gQWxzbywgSSBoYXZlIG1hZGUgc29tZSBtb2RpZmljYXRpb25zIHRvIHRoaXMgZnVuY3Rpb24gaW4gb3JkZXIgdG8gZ2V0IG1vcmUgc3RhdGlzdGljcyBhYm91dCB0aGUgZml0IGFuZCBwcm9kdWNlIGJldHRlciBwbG90cy4gWW91IGNhbiBkb3dubG9hZCB0aGUgbW9kaWZpZWQgc2NyaXB0ICwgY2FsbGVkIGBTdW1tYXJpemVHcm93dGhCeVBsYXRlQUFgIGhlcmUgYW5kIHJ1biBpdC4KCiNQYXJ0IDIuIFBsb3R0aW5nCkhlcmUsIHlvdSBoYXZlIHR3byBvcHRpb25zOgoKMS4gVGhlIGVhc3kgb25lIGlzIHRvIGp1c3QgdXNlIHRoZSBiYXNpYyBwbG90dGluZyBmdW5jdGlvbnMgb2YgYFJgLCB3aGljaCB5b3UgY2FuIHJ1biBkaXJlY3RseSBvbiB0aGUgZ2VuZXJhdGVkIG1vZGVsOgpgYGB7cn0KcGxvdChtb2RlbC53dCkKYGBgClRoaXMgaXMgbm90IHNvIG5pY2UsIGFuZCB0aGVyZSBhcmUgbm90IG1hbnkgb3B0aW9ucyB0byBjaGFuZ2UgaG93IGl0IGxvb2tzLgoKMi4gTm93IHRoZSBtb3JlIGludGVyZXJzdGluZyBwYXJ0IC0gaG93IHRvIG1ha2UgbmljZSBwbG90cyB3aXRoIGBnZ3Bsb3QyYC4gRmlyc3QgSSB3aWxsIG1ha2UgYSBwbG90IG9mIHRoZSByYXcgZGF0YSBvbmx5ICh3aXRob3V0IHRoZSBtb2RlbCBmaXR0aW5nKSBhbmQgc2F2ZSBpdCB0byBgcDFgLiAKYGBge3J9CnAxIDwtIGdncGxvdChkZiwgYWVzKHg9dGltZSx5PXd0KSkgKyBnZW9tX3BvaW50KGFscGhhPTAuNSkgKyB0aGVtZV9idygpCnAxCmBgYAoKSWYgeW91IHdhbnQgdG8gcHV0IHRoZSBwcmVkaWN0ZWQgdmFsdWVzIGZyb20gdGhlIGxvZ2lzdGljIHJlZ3Jlc3Npb24gb24gdGhlIHBsb3QgYWJvdmUsIHlvdSB3aWxsIGhhdmUgdG8gZ2V0IHRoZW0gdXNpbmcgYHByZWRpY3RgICBhbmQgYWRkIHRoZW0gdG8gYSBuZXcgZGF0YXRhYmxlLiBBZnRlciB0aGF0IGp1c3QgYWRkIGFub3RoZXIgYGdlb21gIGZvciB0aGUgcHJlZGljdGVkIHZhbHVlcy4KCmBgYHtyfQpkZi5wcmVkaWN0ZWQgPC0gZGF0YS5mcmFtZSh0aW1lID0gZGYkdGltZSwgcHJlZC53dCA9IHByZWRpY3QobW9kZWwud3QkbW9kZWwpKQpwMSArIGdlb21fbGluZShkYXRhPWRmLnByZWRpY3RlZCwgYWVzKHk9ZGYucHJlZGljdGVkJHByZWQud3QpLCBjb2xvcj0icmVkIikKYGBgCgojUGFydCAzLiBQbG90dGluZyBtYW55IHNhbXBsZXMKSGVyZSBhZ2FpbiwgdHdvIG9wdGlvbnM6CgoxLiBVc2UgdGhlIGJhc2ljIFIgcGxvdHRpbmcgZnVuY3Rpb24sIGkuZS4gcnVuIHRoZSBgU3VtbWFyaXplR3Jvd3RoQnlQbGF0ZWAgb3IgbXkgbW9kaWZpZWQgdmVyc2lvbiBgU3VtbWFyaXplR3Jvd3RoQnlQbGF0ZUFBYCB3aXRoIHRoZSBgcGxvdF9maXQgPSBUUlVFYCBhcmd1bWVudC4gQSBwZGYgZmlsZSB3aWxsIGJlIGdlbmVyYXRlZCB3aXRoIGFsbCB0aGUgcGxvdHMgaW4geW91ciBwcm9qZWN0IGRpcmVjdG9yeS4gVGhlIHBsb3R0aW5nIGhlcmUgaXMgYW1lbmFibGUgdG8gbW9kZmljYXRpb24sIGJ1dCBpcyBzb21ld2hhdCBjb21wbGljYXRlZC4KMi4gSWYgeW91IGhhdmUgYSBsb3Qgb2Ygc2FtcGxlcyBhbmQgd2FudCB0byBkbyB0aGUgcGxvdHRpbmcgd2l0aCBgZ2dwbG90MmAsIHlvdSBuZWVkIHRvIG9idGFpbiB0aGUgY29tcGxldGUgbW9kZWwgZGF0YSBmb3IgZWFjaCBzYW1wbGUgc28gdGhhdCB5b3UgY2FuIGFkZCB0aGUgcHJlZGljdGVkIHZhbHVlcyBmcm9tIHRoZSBtb2RlbCB0byB0aGUgcGxvdC4gVGhpcyBtZWFucyB5b3UgaGF2ZSB0byBydW4gYFN1bW1hcml6ZUdyb3d0aGAgb24gbWFueSBzYW1wbGVzIChgU3VtbWFyaXplR3Jvd3RoQnlQbGF0ZWAgd2lsbCBub3Qgd29yayBoZXJlLCBvciBhdCBsZWFzdCBoYXMgdG8gYmUgbW9kaWZpZWQgaGVhdmlseSB0byB3cml0ZSB0aGUgcHJlZGljdGVkICJPRCIgdmFsdWVzIHNvbWV3aGVyZSkuIApJbiBvcmRlciB0byBnZXQgdGhlIGNvbXBsZXRlIG1vZGVscyBmb3IgYWxsIHNhbXBsZXMgSSBhbSB1c2luZyBzb21lIGBsYXBwbHlgIG1hZ2ljOiBmaXJzdCBkZWZpbmUgYSBmdW5jdGlvbiwgSSBuYW1lIGl0IGBzdW1tR2AgaGVyZSwgYW5kIHRoZW4gdXNlIGl0IGluIGBsYXBwbHlgIHRvIHJ1biBgU3VtbWFyaXplR3Jvd3RoYCBvbiBhbnkgbnVtYmVyIG9mIGNvbHVtbnM6CmBgYHtyfQpzdW1tRyA8LSBmdW5jdGlvbih4KSB7U3VtbWFyaXplR3Jvd3RoKGRmJHRpbWUseCl9CmxhcHBseShkZlsyOm5jb2woZGYpXSwgc3VtbUcpCmBgYApUaGlzIG5lZWRzIHNvbWUgZXhwbGFuYXRpb24sIEkgZ3Vlc3MuIGBTdW1tYXJpemVHcm93dGhgIG5lZWRzIDIgYXJndW1lbnRzIC0gMSkgdGhlIG5hbWUgb2YgdGhlIGNvbHVtbiBmb3IgdGltZSwgaGVyZSBgZGYkdGltZWAgYW5kIDIpIHRoZSBuYW1lIG9mIHRoZSBjb2x1bW4gd2l0aCB0aGUgIk9EIiB2YWx1ZXMgZm9yIG9uZSBzYW1wbGUsIGZvciBleGFtcGxlIHRoaXMgd2FzIGBkZiR3dGAgYmVmb3JlLiBJbiBvcmRlciB0byBkbyB0aGlzIGZvciBtYW55IHNhbXBsZXMsIGFuZCBhdm9pZGluZyBgZm9yYCBsb29wcyB0eXBpY2FsIGZvciBzb21lIG90aGVyIGxhbmd1YWdlcywgSSBhbSB1c2luZyB2ZWN0b3JpemluZywgYSB2ZXJ5IGNvbW1vbiBhcHByb2FjaCBpbiBgUmAuIFB1dCBzaG9ydGx5LCBJIHVzZSBgbGFwcGx5YCB0byBhcHBseSBhIGZ1bmN0aW9uIHRvIGFsbCB0aGUgdmFyaWFibGVzIGluIGEgZGF0YWZyYW1lIGFuZCByZXR1cm4gdGhlIHJlc3VsdHMgaW4gYSBsaXN0LiBTbyBpbiB0aGUgYGxhcHBseWAgY2FsbCBoZXJlIEkgYW0gcnVubmluZyB0aGUgZnVuY3Rpb24gYHN1bW1HYCBmb3IgYWxsIHRoZSB2YXJpYWJsZXMgb2YgdGhlIGBkZmAgKHdpdGhvdXQgdGhlIGZpcnN0IG9uZSB3aGljaCBpcyB0aGUgdGltZSk6IGBkZlsyOm5jb2woZGYpYC4KT2YgY291cnNlLCB0aGlzIGNhbiBiZSBmdXJ0aGVyIGltcHJvdmVkIGJ5IGRlZmluaW5nIHRoZSBgc3VtbUdgIGZ1bmN0aW9uIHdpdGhpbiB0aGUgYGxhcHBseWAgY2FsbCwgYW5kIHdyaXRpbmcgYWxsIG1vZGVscyB0byBhIGxpc3QgbGlrZSB0aGlzOgpgYGB7cn0KbW9kZWxzLmFsbCA8LSBsYXBwbHkoZGZbMjpuY29sKGRmKV0sIGZ1bmN0aW9uKHgpIFN1bW1hcml6ZUdyb3d0aChkZiR0aW1lLCB4KSkKYGBgCgpOb3cgdGhlIG1vZGVscyBmb3IgYWxsIHNhbXBsZXMgYXJlIHdyaXR0ZW4gaW4gYG1vZGVscy5hbGxgLiBXZSBjYW4gdGFrZSB0aGUgcHJlZGljdGVkICJPRCIgdmFsdWVzIGZyb20gdGhlcmUgYW5kIGFkZCB0aGVtIHRvIGEgbmV3IGRhdGFmcmFtZSBjYWxsZWQgYGRmLnByZWRpY3RlZC5wbGF0ZWAuIFRoaXMgdGltZSBJIGFtIHVzaW5nIGEgYGZvcmAgbG9vcDopCmBgYHtyfQpkZi5wcmVkaWN0ZWQucGxhdGUgPC0gZGF0YS5mcmFtZSh0aW1lID0gZGYkdGltZSkKZm9yIChpIGluIG5hbWVzKGRmWzI6bmNvbChkZildKSkgCiAge2RmLnByZWRpY3RlZC5wbGF0ZVtbaV1dIDwtIHByZWRpY3QobW9kZWxzLmFsbFtbaV1dJG1vZGVsKX0KYGBgCgpJbiBjYXNlIHRoZXJlIGFyZSBOQSB2YWx1ZXMgaW4gdGhlIGNvbHVtbnMsIGNoYW5nZSB0aGUgbGFzdCB0d28gY29tbWFuZHMgbGlrZSB0aGlzOgpgYGB7cn0KbW9kZWxzLmFsbCA8LSBsYXBwbHkoZGZbMjpuY29sKGRmKV0sIGZ1bmN0aW9uKHgpIFN1bW1hcml6ZUdyb3d0aChkZlshaXMubmEoeCksIDFdLCB4WyFpcy5uYSh4KV0pKQoKZGYucHJlZGljdGVkLnBsYXRlIDwtIGRhdGEuZnJhbWUodGltZSA9IGRmJHRpbWUpCmZvciAoaSBpbiBuYW1lcyhkZlsyOm5jb2woZGYpXSkpIAogIHtkZi5wcmVkaWN0ZWQucGxhdGVbW2ldXSA8LSBwcmVkaWN0KG1vZGVscy5hbGxbW2ldXSRtb2RlbCwgbmV3ZGF0YSA9IGxpc3QodCA9IGRmJHRpbWUpKX0KYGBgCk5vdGUgdGhhdCBpbiBvcmRlciBmb3IgYHByZWRpY3RgIHRvIHdvcmssIHRoZSBuZXcgZGF0YSBzdXBwbGllZCAodGltZSBpbiB0aGlzIGNhc2UpIGhhcyB0byBiZSBuYW1lZCBsaWtlIGluIHRoZSBtb2RlbCwgaGVuY2UgdGhlIGBsaXN0KHQgPSBkZiR0aW1lKWAuCgpUaGVuIEkgbWVsdCB0aGUgdHdvIGRhdGFmcmFtZXMsIGBkZmAgKHRoZSBvbmUgd2l0aCB0aGUgbWVhc3VyZWQgT0QgdmFsdWVzKSBhbmQgYGRmLnByZWRpY3RlZC5wbGF0ZWAgKHRoZSBvbmUgd2l0aCB0aGUgcHJlZGljdGVkIE9EIHZhbHVlcyksIGFuZCBtZXJnZSB0aGVtIHRvZ2V0aGVyLiBUaGUgYG1lbHRgIGlzIHJlcXVpcmVkIGZvciBlYXNpZXIgcGxvdHRpbmcgaW4gYGdncGxvdDJgIGxhdGVyLgpgYGB7cn0KbWVsdDEgPC0gbWVsdChkZiwgaWQudmFycyA9ICJ0aW1lIiwgdmFyaWFibGUubmFtZSA9ICJzYW1wbGUiLCB2YWx1ZS5uYW1lID0gIm9kIikKbWVsdDIgPC0gbWVsdChkZi5wcmVkaWN0ZWQucGxhdGUsIGlkLnZhcnMgPSAidGltZSIsIHZhcmlhYmxlLm5hbWUgPSAic2FtcGxlIiwgdmFsdWUubmFtZSA9ICJwcmVkLm9kIikKZGYuZmluYWwgPC0gY2JpbmQobWVsdDEsIHByZWQub2Q9bWVsdDJbLDNdKQpybShtZWx0MSkKcm0obWVsdDIpCmBgYAoKTm93IHRoZSBwbG90cy4gTm90ZSBob3cgSSBoYXZlIHNldCB0aGUgYGZhY2V0X3dyYXBgIG51bWJlciBvZiBjb2x1bW5zIHRvIGAxMmAuIFRoaXMgaXMgaW4gY2FzZSB5b3Ugd2FudCB0byBwbG90IGEgd2hvbGUgcGxhdGUsIHdpbGwgdHJhbnNsYXRlIHRvIHBsb3RzIG9yZ2FuaXplZCBpbiAxMiBjb2x1bW5zIGFuZCA4IHJvd3MuIFlvdSBjYW4gb2YgY291cnNlIGFkanVzdCB0aGUgYGZhY2V0X3dyYXBgIGFzIHlvdSB3aXNoIHdpdGggYG5jb2wgPWAgYW5kIGBucm93PWAuCmBgYHtyLCBlY2hvPVRSVUV9CmdncGxvdChkZi5maW5hbCwgYWVzKHg9dGltZSwgeT1vZCkpICsgZ2VvbV9wb2ludChhZXMoKSwgYWxwaGE9MC41KSArIGdlb21fbGluZShhZXMoeT1wcmVkLm9kKSwgY29sb3I9InJlZCIpICsgZmFjZXRfd3JhcCh+c2FtcGxlLCBuY29sID0gMTIpICsgdGhlbWVfYncoKQpgYGAKV2hlbiB5b3UgYXJlIGhhcHB5IHdpdGggd2hhdCBjb21lcyBvdXQsIGRvbid0IGZvcmdldCB0byBzYXZlIHRoZSBwbG90cy4KCklmIHlvdSB3YW50IHRvIGdldCBhIHRhYmxlIHdpdGggYWxsIHRoZSB2YWx1ZXMgZm9yIGFsbCB0aGUgc2FtcGxlczoKYGBge3J9Cgp2YWxzX2RmIDwtIG1hcChtb2RlbHMuYWxsLCAidmFscyIpICU+JSBtYXBfZGYoYmluZF9yb3dzLCAuaWQgPSAic2FtcGxlIikKd3JpdGUuY3N2KHZhbHNfZGYsIGZpbGUgPSAib3V0cHV0LXByZWRpY3RlZC12YWx1ZXMuY3N2IikKYGBgCgoKRW5qb3khCgpBQQoKUFMKQXMgd2l0aCBhbnkgbW9kZWwsIG9uZSBjYW4gc3VwcGx5IG5ldyBkYXRhIGFuZCBsZXQgdGhlIG1vZGVsIHByZWRpY3QgdGhlIG91dGNvbWUuIEluIHRoaXMgaW1hZ2VuYXJ5IGNhc2Ugd2UgY2FuIHN1cHBseSBuZXcgdGltZSBwb2ludHMgYW5kIGxldCB0aGUgbW9kZWwgcHJlZGljdCB0aGUgT0QgdmFsdWVzLiBJIHVzZWQgdGhpcyBhcHByb2FjaCBhYm92ZSBpbiBvcmRlciB0byBkZWFsIHdpdGggdGhlIE5BIHZhbHVlcy4KYGBge3J9CnR0IDwtIHNlcSgwLDEwLCBsZW5ndGg9NjYpCnByZWRpY3QobW9kZWwud3QkbW9kZWwsbmV3ZGF0YT1saXN0KHQ9dHQpKQpgYGAKCgoK