Seurat_Tutorial_PBM
In this tutorial, we will be looking at a dataset of Peripheral Blood Mononuclear Cells (PBMC). PBMCs are like the army of immune cells that live in our blood and fight off any invaders like viruses or bacteria.
The dataset we have has 2,700 single PBMCs that have been sequenced using a fancy Illumina machine called the NextSeq 500. We’ll be using a function called Read10X() to read in the data from the 10X Genomics pipeline. This function will give us a matrix that tells us how many different molecules of each gene were detected in each of the 2,700 PBMCs.
We’ll then use this matrix to create a Seurat object. Think of this as creating a “fellowship of cells”, like in Lord of the Rings.
Here’s the code we’ll use to load in the dataset and create our Seurat object:
library(dplyr)
library(Seurat)
library(patchwork)
pbmc.data <- Read10X(data.dir = "pbmc3k/filtered_gene_bc_matrices/hg19/")
pbmc.data #32738 x 2700 sparse Matrix of class "dgCMatrix"32738 x 2700 sparse Matrix of class "dgCMatrix" [[ suppressing 74 column names ‘AAACATACAACCAC-1’, ‘AAACATTGAGCTAC-1’, ‘AAACATTGATCAGC-1’ ... ]]
[[ suppressing 74 column names ‘AAACATACAACCAC-1’, ‘AAACATTGAGCTAC-1’, ‘AAACATTGATCAGC-1’ ... ]]
MIR1302-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
FAM138A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
OR4F5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
RP11-34P13.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
RP11-34P13.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
AL627309.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
RP11-34P13.14 . . . . `<S4 [32738 x 2700] (Matrix::dgCMatrix)>` . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
..............................
........suppressing 2626 columns and 32725 rows in show(); maybe adjust 'options(max.print= *, width = *)'
.............................. [[ suppressing 74 column names ‘AAACATACAACCAC-1’, ‘AAACATTGAGCTAC-1’, ‘AAACATTGATCAGC-1’ ... ]]
SRSF10.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 . . . . . . . . . . 1 1 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
AC145205.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
BAGE5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
CU459201.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
AC002321.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......
AC002321.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......#a class of sparse numeric matrices in the compressed, sparse, column-oriented format. In this implementation the non-zero elements in the columns are sorted into increasing row order. dgCMatrix is the “standard” class for sparse numeric matrices in the Matrix package.
pbmc <- CreateSeuratObject(counts = pbmc.data, project = "pbmc3k", min.cells = 3, min.features = 200)Warning: Feature names cannot have underscores ('_'), replacing with dashes ('-')pbmcAn object of class Seurat
13714 features across 2700 samples within 1 assay
Active assay: RNA (13714 features, 0 variable features)Next, we’ll need to do some pre-processing on our data to make sure it’s ready for analysis. This involves selecting and filtering the best quality cells based on various metrics, normalizing and scaling the data, and identifying highly variable genes that will be important for our analysis. Think of this as preparing our fellowship for battle by making sure everyone is healthy and equipped with the right weapons.
Standard pre-processing workflow
A few QC metrics commonly used by the community include - the number of unique genes detected in each cell. - Low-quality cells or empty droplets will often have very few genes - Cell doublets or multiplets may exhibit an aberrantly high gene count - Similarly, the total number of molecules detected within a cell (correlates strongly with unique genes) - The percentage of reads that map to the mitochondrial genome - Low-quality / dying cells often exhibit extensive mitochondrial contamination - We calculate mitochondrial QC metrics with the PercentageFeatureSet() function, which calculates the percentage of counts originating from a set of features - We use the set of all genes starting with MT- as a set of mitochondrial genes
# The [[ operator can add columns to object metadata. This is a great place to stash QC stats
pbmc[["percent.mt"]] <- PercentageFeatureSet(pbmc, pattern = "^MT-")
head(pbmc@meta.data, 5)In the example below, we visualize QC metrics, and use these to filter cells. We filter cells that have unique feature counts over 2,500 or less than 200 We filter cells that have >5% mitochondrial counts
# Visualize QC metrics as a violin plot
VlnPlot(pbmc, features = c("nFeature_RNA", "nCount_RNA", "percent.mt"), ncol = 3)
# FeatureScatter is typically used to visualize feature-feature relationships, but can be used for anything calculated by the object, i.e. columns in object metadata, PC scores etc.
plot1 <- FeatureScatter(pbmc, feature1 = "nCount_RNA", feature2 = "percent.mt")
plot2 <- FeatureScatter(pbmc, feature1 = "nCount_RNA", feature2 = "nFeature_RNA")
plot1 + plot2
Pearson correlation between the two features is displayed above the plot. nFeature_RNA vs nCount_RNA:
nCount represents the total number of reads or molecules captured for each cell, while nFeature RNA corresponds to the number of unique genes detected in each cell. By plotting the correlation between these two variables, researchers can assess the overall data quality. Cells with low nCount and nFeature RNA may indicate poor capture or low-quality data, and they can be potentially excluded from downstream analyses to ensure robust results. Here we see that we do not poor quality data. In general we do want to see higher correlation between these two variables.
Outliers can arise due to various technical or biological factors, such as cell doublets or experimental artifacts. The correlation plot helps to identify such outliers by highlighting cells that deviate from the expected relationship between nCount and nFeature RNA.
pbmc <- subset(pbmc, subset = nFeature_RNA > 200 & nFeature_RNA < 2500 & percent.mt < 5)
VlnPlot(pbmc, features = c("nFeature_RNA", "nCount_RNA", "percent.mt"), ncol = 3)
Compare the above Violin plot to understand which cells were filtered.
#SCTransform(object = pbmc)
#Use this function as an alternative to the NormalizeData, FindVariableFeatures, ScaleData workflow
#Results are saved in a new assay (named SCT by default) with counts being (corrected) counts, data being log1p(counts), scale.data being pearson residuals; sctransform::vst intermediate results are saved in misc slot of new assay.Normalizing the data
Normalization is like making sure everyone in our fellowship is speaking the same language and using the same measuring system. This will help us compare gene expression levels between different cells.
We utilize a normalization technique called “LogNormalize” to standardize the feature expression measurements for each individual cell. This normalization method involves dividing the expression values of each feature by the total expression of that cell, followed by multiplication with a scaling factor (typically set to 10,000 by default). Finally, we apply a logarithmic transformation to the normalized values. This process helps ensure that the expression values are comparable across cells and enables us to effectively analyze and interpret the data.
#pbmc <- NormalizeData(pbmc, normalization.method = "RC", scale.factor = 1e6)
pbmc <- NormalizeData(pbmc, normalization.method = "LogNormalize", scale.factor = 10000)Performing log-normalization
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
par(mfrow=c(1,2))
# original expression distribution
raw_geneExp = as.vector(pbmc[['RNA']]@counts) %>% sample(10000)
raw_geneExp = raw_geneExp[raw_geneExp != 0]
hist(raw_geneExp)
# expression distribution after normalization
logNorm_geneExp = as.vector(pbmc[['RNA']]@data) %>% sample(10000)
logNorm_geneExp = logNorm_geneExp[logNorm_geneExp != 0]
hist(logNorm_geneExp)
Identification of highly variable features (feature selection) Now that we’ve normalized our data, we need to identify which genes are most important for our analysis. We do this by identifying highly variable genes.
Highly variable genes are like the real heroes in our fellowship, the ones who stand out from the rest and make a big impact. These genes exhibit a lot of variation in expression levels between cells, which can be a good indication of biological significance.
In ‘vst’ method it fits a line to the relationship of log(variance) and log(mean) using local polynomial regression (loess). Then standardizes the feature values using the observed mean and expected variance (given by the fitted line). Feature variance is then calculated on the standardized values after clipping to a maximum (see clip.max parameter).
pbmc <- FindVariableFeatures(pbmc, selection.method = "vst", nfeatures = 2000)Calculating gene variances
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Calculating feature variances of standardized and clipped values
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|# Identify the 10 most highly variable genes
top10 <- head(VariableFeatures(pbmc), 10)
By default, we return 2,000 features per dataset. You can adjust this number depending on the size and complexity of your dataset.
# plot variable features with and without labels
plot1 <- VariableFeaturePlot(pbmc)
plot2 <- LabelPoints(plot = plot1, points = top10, repel = TRUE, xnudge=0, ynudge=0)
plot2
Once we’ve identified our highly variable genes, we can use them to focus our downstream analysis and highlight the biological signal in our single-cell dataset.
Keep up the great work, and don’t forget that even the smallest heroes can have a big impact!
Scaling the data
The next step is to apply a linear transformation called “scaling”. This is a standard pre-processing step that we need to do before we can use techniques like PCA (principal component analysis) to analyze our data.
Scaling is like making sure that everyone’s voice is at the same volume in our fellowship. We want to make sure that each gene is given equal importance in downstream analysis, so that highly-expressed genes don’t dominate the results.
We’ll use the “ScaleData” function in Seurat to apply scaling to our dataset. Here’s the code we’ll use:
all.genes <- rownames(pbmc)
pbmc <- ScaleData(pbmc, features = all.genes, do.center = TRUE)Centering and scaling data matrix
|
| | 0%
|
|=========== | 7%
|
|====================== | 14%
|
|================================= | 21%
|
|============================================ | 29%
|
|======================================================= | 36%
|
|================================================================== | 43%
|
|============================================================================== | 50%
|
|========================================================================================= | 57%
|
|==================================================================================================== | 64%
|
|=============================================================================================================== | 71%
|
|========================================================================================================================== | 79%
|
|===================================================================================================================================== | 86%
|
|================================================================================================================================================ | 93%
|
|===========================================================================================================================================================| 100%In this code, we’re using the “ScaleData” function to shift the expression of each gene so that the mean expression across cells is 0, and scale the expression of each gene so that the variance across cells is 1. This step is important because it helps us compare gene expression levels between cells that might have different total amounts of RNA and so that highly-expressed genes do not dominate.
Now that we’ve scaled our data, we can move on to dimensional reduction techniques like PCA. Keep up the great work, and remember that in our fellowship, everyone’s voice is important!
Perform linear dimensional reduction
PCA helps us identify patterns in our data by reducing the number of features in our dataset. This makes it easier to visualize and analyze our data. After running PCA, we can move on to clustering our cells and identifying different cell types based on gene expression patterns. Keep up the great work, and remember, with PCA, one does not simply analyze all the features in a dataset!
pbmc <- RunPCA(pbmc, features = VariableFeatures(object = pbmc))PC_ 1
Positive: CST3, TYROBP, LST1, AIF1, FTL, FTH1, LYZ, FCN1, S100A9, TYMP
FCER1G, CFD, LGALS1, S100A8, CTSS, LGALS2, SERPINA1, IFITM3, SPI1, CFP
PSAP, IFI30, SAT1, COTL1, S100A11, NPC2, GRN, LGALS3, GSTP1, PYCARD
Negative: MALAT1, LTB, IL32, IL7R, CD2, B2M, ACAP1, CD27, STK17A, CTSW
CD247, GIMAP5, AQP3, CCL5, SELL, TRAF3IP3, GZMA, MAL, CST7, ITM2A
MYC, GIMAP7, HOPX, BEX2, LDLRAP1, GZMK, ETS1, ZAP70, TNFAIP8, RIC3
PC_ 2
Positive: CD79A, MS4A1, TCL1A, HLA-DQA1, HLA-DQB1, HLA-DRA, LINC00926, CD79B, HLA-DRB1, CD74
HLA-DMA, HLA-DPB1, HLA-DQA2, CD37, HLA-DRB5, HLA-DMB, HLA-DPA1, FCRLA, HVCN1, LTB
BLNK, P2RX5, IGLL5, IRF8, SWAP70, ARHGAP24, FCGR2B, SMIM14, PPP1R14A, C16orf74
Negative: NKG7, PRF1, CST7, GZMB, GZMA, FGFBP2, CTSW, GNLY, B2M, SPON2
CCL4, GZMH, FCGR3A, CCL5, CD247, XCL2, CLIC3, AKR1C3, SRGN, HOPX
TTC38, APMAP, CTSC, S100A4, IGFBP7, ANXA1, ID2, IL32, XCL1, RHOC
PC_ 3
Positive: HLA-DQA1, CD79A, CD79B, HLA-DQB1, HLA-DPB1, HLA-DPA1, CD74, MS4A1, HLA-DRB1, HLA-DRA
HLA-DRB5, HLA-DQA2, TCL1A, LINC00926, HLA-DMB, HLA-DMA, CD37, HVCN1, FCRLA, IRF8
PLAC8, BLNK, MALAT1, SMIM14, PLD4, P2RX5, IGLL5, LAT2, SWAP70, FCGR2B
Negative: PPBP, PF4, SDPR, SPARC, GNG11, NRGN, GP9, RGS18, TUBB1, CLU
HIST1H2AC, AP001189.4, ITGA2B, CD9, TMEM40, PTCRA, CA2, ACRBP, MMD, TREML1
NGFRAP1, F13A1, SEPT5, RUFY1, TSC22D1, MPP1, CMTM5, RP11-367G6.3, MYL9, GP1BA
PC_ 4
Positive: HLA-DQA1, CD79B, CD79A, MS4A1, HLA-DQB1, CD74, HIST1H2AC, HLA-DPB1, PF4, SDPR
TCL1A, HLA-DRB1, HLA-DPA1, HLA-DQA2, PPBP, HLA-DRA, LINC00926, GNG11, SPARC, HLA-DRB5
GP9, AP001189.4, CA2, PTCRA, CD9, NRGN, RGS18, CLU, TUBB1, GZMB
Negative: VIM, IL7R, S100A6, IL32, S100A8, S100A4, GIMAP7, S100A10, S100A9, MAL
AQP3, CD2, CD14, FYB, LGALS2, GIMAP4, ANXA1, CD27, FCN1, RBP7
LYZ, S100A11, GIMAP5, MS4A6A, S100A12, FOLR3, TRABD2A, AIF1, IL8, IFI6
PC_ 5
Positive: GZMB, NKG7, S100A8, FGFBP2, GNLY, CCL4, CST7, PRF1, GZMA, SPON2
GZMH, S100A9, LGALS2, CCL3, CTSW, XCL2, CD14, CLIC3, S100A12, RBP7
CCL5, MS4A6A, GSTP1, FOLR3, IGFBP7, TYROBP, TTC38, AKR1C3, XCL1, HOPX
Negative: LTB, IL7R, CKB, VIM, MS4A7, AQP3, CYTIP, RP11-290F20.3, SIGLEC10, HMOX1
LILRB2, PTGES3, MAL, CD27, HN1, CD2, GDI2, CORO1B, ANXA5, TUBA1B
FAM110A, ATP1A1, TRADD, PPA1, CCDC109B, ABRACL, CTD-2006K23.1, WARS, VMO1, FYB # Examine and visualize PCA results a few different ways
print(pbmc[["pca"]], dims = 1:5, nfeatures = 5)PC_ 1
Positive: CST3, TYROBP, LST1, AIF1, FTL
Negative: MALAT1, LTB, IL32, IL7R, CD2
PC_ 2
Positive: CD79A, MS4A1, TCL1A, HLA-DQA1, HLA-DQB1
Negative: NKG7, PRF1, CST7, GZMB, GZMA
PC_ 3
Positive: HLA-DQA1, CD79A, CD79B, HLA-DQB1, HLA-DPB1
Negative: PPBP, PF4, SDPR, SPARC, GNG11
PC_ 4
Positive: HLA-DQA1, CD79B, CD79A, MS4A1, HLA-DQB1
Negative: VIM, IL7R, S100A6, IL32, S100A8
PC_ 5
Positive: GZMB, NKG7, S100A8, FGFBP2, GNLY
Negative: LTB, IL7R, CKB, VIM, MS4A7 The VizDimLoadings function helps you understand which genes contribute the most to each dimension. It creates a plot where the genes are ranked based on their contribution to a particular dimension, and their loadings are shown as bar plots.
#Visualize top genes associated with reduction components
VizDimLoadings(pbmc, dims = 1:2, reduction = "pca")
X-axis: The x-axis represents the genes. The genes are usually sorted based on their contribution to the given dimension, from highest to lowest.
Y-axis: The y-axis represents the loadings of each gene on the given dimension. The loadings indicate the extent to which a gene contributes to the specific dimension. A higher loading value means the gene has a stronger influence on that dimension.
The DimPlot function takes this reduced-dimensional representation and creates a scatter plot where each point represents a cell, and its position is determined by the cell’s embeddings in the reduced space.
# Dimensional reduction plot
DimPlot(pbmc, reduction = "pca")
# The axes in the scatter plot correspond to the dimensions (components) obtained from the dimensionality reduction technique
# Each point in the scatter plot represents a cell from your datasetThe position of the point in this dimplot reflects the cell’s embeddings in the reduced space. Cells with similar gene expression profiles tend to cluster together in the plot.
DimHeatmap() allows for easy exploration of the primary sources of heterogeneity in a dataset, and can be useful when trying to decide which PCs to include for further downstream analyses.
It can be a valuable tool for exploring correlated feature sets.
#Dimensional reduction heatmap
# Explore the primary sources of variation in our dataset using DimHeatmap
DimHeatmap(pbmc, dims = 1, cells = 500, balanced = TRUE)
#The DimHeatmap function produces a heatmap where the rows represent the genes and the columns represent the cells. The “balanced” argument ensures that the plot has an equal number of genes with both + and - scores. The color intensity of each cell in the heatmap reflects the expression level of the corresponding gene in that cell (darker color for higher expression).
DimHeatmap(pbmc, dims = 1:9, cells = 500, balanced = TRUE)
The “position” of a gene along PC_n refers to the magnitude of its loading on this component. A gene with a higher positive loading has a strong positive association with PC_n, meaning its expression levels tend to increase along the direction of this component. Conversely, a gene with a higher negative loading has a strong negative association with PC_n, indicating its expression levels tend to decrease along the direction of this component.
By looking at the expression patterns of our top PCs, we can also start to get a sense of the different cell types present in our dataset.
Determine the ‘dimensionality’ of the dataset
When we analyze scRNA-seq data, we often want to identify groups of cells that behave similarly. Seurat can help us do this by clustering cells based on their similarities and differences. To do this, we first need to reduce the dimensionality of the data, which we do using PCA. Each PC is a ‘metafeature’ that combines information across a correlated feature set. The top principal components therefore represent a robust compression of the dataset.
Suppose they wanted to choose how many people should be in the fellowship of the ring, theoretically it’s like choosing the members of the “fellowship of the ring” based on their performance in a smaller task. We select the principal components that perform the best in distinguishing between cells and keep them for further analysis.
To answer this question, we use a technique called the JackStraw procedure. We randomly shuffle a small portion of the data (usually 1%) and run PCA on this shuffled data multiple times. By doing this, we create a null distribution of feature scores and identify the significant principal components based on the enrichment of low p-value features.
## NOTE: This process can take a long time for big datasets, comment out for expediency. More approximate techniques such as those implemented in ElbowPlot() can be used to reduce computation time
pbmc <- JackStraw(pbmc, num.replicate = 30) # num.replicate = 100 will take longer
| | 0 % ~calculating
|++ | 3 % ~01m 14s
|++++ | 7 % ~01m 08s
|+++++ | 10% ~01m 05s
|+++++++ | 13% ~01m 02s
|+++++++++ | 17% ~58s
|++++++++++ | 20% ~54s
|++++++++++++ | 23% ~51s
|++++++++++++++ | 27% ~48s
|+++++++++++++++ | 30% ~46s
|+++++++++++++++++ | 33% ~44s
|+++++++++++++++++++ | 37% ~42s
|++++++++++++++++++++ | 40% ~40s
|++++++++++++++++++++++ | 43% ~38s
|++++++++++++++++++++++++ | 47% ~35s
|+++++++++++++++++++++++++ | 50% ~33s
|+++++++++++++++++++++++++++ | 53% ~31s
|+++++++++++++++++++++++++++++ | 57% ~29s
|++++++++++++++++++++++++++++++ | 60% ~26s
|++++++++++++++++++++++++++++++++ | 63% ~24s
|++++++++++++++++++++++++++++++++++ | 67% ~22s
|+++++++++++++++++++++++++++++++++++ | 70% ~20s
|+++++++++++++++++++++++++++++++++++++ | 73% ~17s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~15s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~13s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~11s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~09s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~07s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~04s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~02s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01m 05s
| | 0 % ~calculating
|+++ | 5 % ~00s
|+++++ | 10% ~00s
|++++++++ | 15% ~00s
|++++++++++ | 20% ~00s
|+++++++++++++ | 25% ~00s
|+++++++++++++++ | 30% ~00s
|++++++++++++++++++ | 35% ~00s
|++++++++++++++++++++ | 40% ~00s
|+++++++++++++++++++++++ | 45% ~00s
|+++++++++++++++++++++++++ | 50% ~00s
|++++++++++++++++++++++++++++ | 55% ~00s
|++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|+++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=00s pbmc <- ScoreJackStraw(pbmc, dims = 1:20)
JackStrawPlot(pbmc, dims = 1:15)
This process can take a while for large datasets, but there are faster techniques like the ElbowPlot() function that can be used to reduce computation time. Once we have identified the significant principal components, we can use them to cluster cells based on their similarities and differences.
An alternative heuristic method generates an ‘Elbow plot’: a ranking of principle components based on the percentage of variance explained by each one.
ElbowPlot(pbmc)
Just like Gandalf had to choose the right hobbits to join the quest, we have to select the relevant sources of heterogeneity in our dataset.
Deciding on the number of principal components to keep can be as tricky as deciding on the number of members for a fellowship.
1)The first approach is more supervised, where we explore the principal components to identify relevant sources of heterogeneity, and can be used in conjunction with tools like GSEA, just like using Legolas’ keen eyes to explore each principal component and determine its relevance. 2) The second approach is a statistical test based on a random null model, but it may not return a clear cutoff and is time-consuming for large datasets. (JackStraw) This is very similar to Frodo’s journey to Mount Doom, random walking and way too time-consuming with no clear end. 3) Finally, the third approach is a heuristic (elbow plot) that is commonly used and can be calculated instantly, just like how Samwise used his intuition to make quick decisions.
In this study, all three approaches produced similar results, and we could have reasonably chosen a cutoff between PC 7-12.
Cluster the cells
Seurat v3 uses a graph-based clustering method to group cells with similar feature expression patterns.
First, a K-nearest neighbor (KNN) graph is constructed based on the euclidean distance in PCA space, and edge weights between cells are refined using shared overlap in their local neighborhoods. This step is performed using the FindNeighbors() function, and takes as input the previously defined dimensionality of the dataset (first 10 PCs). Next, modularity optimization techniques like the Louvain algorithm are applied to group cells together with the goal of optimizing the standard modularity function. This is done using the FindClusters() function, with a resolution parameter that sets the ‘granularity’ of the downstream clustering. The Idents() function can be used to find the clusters.
Just like Frodo needed a trusted community to complete his journey, we need to cluster cells into meaningful groups to complete downstream analyses.
pbmc <- FindNeighbors(pbmc, dims = 1:8)Computing nearest neighbor graph
Computing SNNpbmc <- FindClusters(pbmc, resolution = 0.7)Modularity Optimizer version 1.3.0 by Ludo Waltman and Nees Jan van Eck
Number of nodes: 2638
Number of edges: 91842
Running Louvain algorithm...0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|Maximum modularity in 10 random starts: 0.8534
Number of communities: 11
Elapsed time: 0 seconds# Look at cluster IDs of the first 5 cells
head(Idents(pbmc), 5)AAACATACAACCAC-1 AAACATTGAGCTAC-1 AAACATTGATCAGC-1 AAACCGTGCTTCCG-1 AAACCGTGTATGCG-1
0 2 0 5 8
Levels: 0 1 2 3 4 5 6 7 8 9 10Run non-linear dimensional reduction (UMAP/tSNE)
Imagine you have a vast kingdom with many cities and towns. Each city represents a group of cells with similar characteristics, and you want to explore them on a map. However, your map is limited to only two dimensions, which means you need a way to compress the information and place similar cities together.
That’s where tSNE and UMAP algorithms come in. They are like magical spells that allow you to transform the data into a lower-dimensional space, where similar cells are placed closer together. This helps you visualize and explore the data more easily.
To make sure the cells in each group (or cluster) are located near each other on the map, we suggest using the same “metafeatures” (i.e., PCs) that were used for clustering analysis as input to tSNE and UMAP. This will help you see the differences and similarities between the clusters more clearly.
# If you haven't installed UMAP, you can do so via reticulate::py_install(packages = 'umap-learn')
pbmc <- RunUMAP(pbmc, dims = 1:10)13:53:42 UMAP embedding parameters a = 0.9922 b = 1.112
13:53:42 Read 2638 rows and found 10 numeric columns
13:53:42 Using Annoy for neighbor search, n_neighbors = 30
13:53:42 Building Annoy index with metric = cosine, n_trees = 50
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
13:53:43 Writing NN index file to temp file /tmp/RtmphsS9HK/filee4c452293efc
13:53:43 Searching Annoy index using 1 thread, search_k = 3000
13:53:43 Annoy recall = 100%
13:53:44 Commencing smooth kNN distance calibration using 1 thread with target n_neighbors = 30
13:53:45 Initializing from normalized Laplacian + noise (using irlba)
13:53:45 Commencing optimization for 500 epochs, with 105140 positive edges
Using method 'umap'
0% 10 20 30 40 50 60 70 80 90 100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
13:53:48 Optimization finished# note that you can set `label = TRUE` or use the LabelClusters function to help label # individual clusters
DimPlot(pbmc, reduction = "umap", label = TRUE)
Finding differentially expressed features (cluster biomarkers)
If Seurat were a wizard, finding markers that define clusters via differential expression would be its signature spell - “Expecto Markerum!” By default, Seurat identifies positive and negative markers of a single cluster (specified in ident.1), compared to all other cells. But if you want to go on a more adventurous quest, FindAllMarkers() automates this process for all clusters.
# find all markers of cluster 2
cluster2.markers <- FindMarkers(pbmc, ident.1 = 2, min.pct = 0.25)
| | 0 % ~calculating
|+ | 1 % ~04s
|++ | 2 % ~03s
|++ | 3 % ~04s
|+++ | 4 % ~04s
|+++ | 5 % ~04s
|++++ | 6 % ~04s
|++++ | 7 % ~03s
|+++++ | 8 % ~03s
|+++++ | 9 % ~03s
|++++++ | 11% ~03s
|++++++ | 12% ~03s
|+++++++ | 13% ~03s
|+++++++ | 14% ~03s
|++++++++ | 15% ~03s
|++++++++ | 16% ~03s
|+++++++++ | 17% ~03s
|+++++++++ | 18% ~03s
|++++++++++ | 19% ~03s
|++++++++++ | 20% ~03s
|+++++++++++ | 21% ~03s
|++++++++++++ | 22% ~03s
|++++++++++++ | 23% ~03s
|+++++++++++++ | 24% ~03s
|+++++++++++++ | 25% ~03s
|++++++++++++++ | 26% ~03s
|++++++++++++++ | 27% ~03s
|+++++++++++++++ | 28% ~03s
|+++++++++++++++ | 29% ~02s
|++++++++++++++++ | 31% ~02s
|++++++++++++++++ | 32% ~02s
|+++++++++++++++++ | 33% ~02s
|+++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 37% ~02s
|+++++++++++++++++++ | 38% ~02s
|++++++++++++++++++++ | 39% ~02s
|++++++++++++++++++++ | 40% ~02s
|+++++++++++++++++++++ | 41% ~02s
|++++++++++++++++++++++ | 42% ~02s
|++++++++++++++++++++++ | 43% ~02s
|+++++++++++++++++++++++ | 44% ~02s
|+++++++++++++++++++++++ | 45% ~02s
|++++++++++++++++++++++++ | 46% ~02s
|++++++++++++++++++++++++ | 47% ~02s
|+++++++++++++++++++++++++ | 48% ~02s
|+++++++++++++++++++++++++ | 49% ~02s
|++++++++++++++++++++++++++ | 51% ~02s
|++++++++++++++++++++++++++ | 52% ~02s
|+++++++++++++++++++++++++++ | 53% ~02s
|+++++++++++++++++++++++++++ | 54% ~02s
|++++++++++++++++++++++++++++ | 55% ~01s
|++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|+++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|++++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|+++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|++++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|+++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 75% ~01s
|++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s head(cluster2.markers, n = 5)However, beware of the min.pct argument, which requires a feature to be detected at a minimum percentage in either of the two groups of cells. It’s like requiring a hobbit to have at least 50% hairy feet before letting them join the fellowship. And don’t forget the thresh.test argument, which requires a feature to be differentially expressed (on average) by some amount between the two groups. It’s like requiring Legolas to be at least 1 foot taller than Frodo before he can join the fellowship.
If you’re feeling brave, you can set both of these arguments to 0, but be prepared for a long and treacherous journey through a large number of features that are unlikely to be highly discriminatory. Alternatively, you can use the max.cells.per.ident argument to downsample each identity class to have no more cells than whatever this is set to. It’s like asking Gandalf to use his magic to shrink the fellowship down to a more manageable size. Yes, there may be a loss in power, but the speed increases can be significant, and the most highly differentially expressed features will likely still rise to the top.
# find all markers distinguishing cluster 5 from clusters 0 and 3
cluster5.markers <- FindMarkers(pbmc, ident.1 = 5, ident.2 = c(0, 3), min.pct = 0.25)
| | 0 % ~calculating
|+ | 1 % ~04s
|++ | 2 % ~03s
|++ | 3 % ~03s
|+++ | 4 % ~03s
|+++ | 5 % ~03s
|++++ | 6 % ~03s
|++++ | 7 % ~03s
|+++++ | 8 % ~03s
|+++++ | 9 % ~03s
|++++++ | 10% ~03s
|++++++ | 11% ~03s
|+++++++ | 12% ~03s
|+++++++ | 13% ~03s
|++++++++ | 14% ~03s
|++++++++ | 15% ~03s
|+++++++++ | 16% ~03s
|+++++++++ | 18% ~03s
|++++++++++ | 19% ~03s
|++++++++++ | 20% ~03s
|+++++++++++ | 21% ~03s
|+++++++++++ | 22% ~03s
|++++++++++++ | 23% ~02s
|++++++++++++ | 24% ~02s
|+++++++++++++ | 25% ~02s
|+++++++++++++ | 26% ~02s
|++++++++++++++ | 27% ~02s
|++++++++++++++ | 28% ~02s
|+++++++++++++++ | 29% ~02s
|+++++++++++++++ | 30% ~02s
|++++++++++++++++ | 31% ~02s
|++++++++++++++++ | 32% ~02s
|+++++++++++++++++ | 33% ~02s
|++++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|+++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 37% ~02s
|++++++++++++++++++++ | 38% ~02s
|++++++++++++++++++++ | 39% ~02s
|+++++++++++++++++++++ | 40% ~02s
|+++++++++++++++++++++ | 41% ~02s
|++++++++++++++++++++++ | 42% ~02s
|++++++++++++++++++++++ | 43% ~02s
|+++++++++++++++++++++++ | 44% ~02s
|+++++++++++++++++++++++ | 45% ~02s
|++++++++++++++++++++++++ | 46% ~02s
|++++++++++++++++++++++++ | 47% ~02s
|+++++++++++++++++++++++++ | 48% ~02s
|+++++++++++++++++++++++++ | 49% ~02s
|++++++++++++++++++++++++++ | 51% ~02s
|++++++++++++++++++++++++++ | 52% ~02s
|+++++++++++++++++++++++++++ | 53% ~01s
|+++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|+++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|+++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|+++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|++++++++++++++++++++++++++++++++++++ | 70% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|+++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|++++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 75% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s head(cluster5.markers, n = 5)NA# find markers for every cluster compared to all remaining cells, report only the positive ones
pbmc.markers <- FindAllMarkers(pbmc, only.pos = TRUE, min.pct = 0.25, logfc.threshold = 0.25)Calculating cluster 0
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 2 % ~01s
|++ | 3 % ~01s
|+++ | 4 % ~01s
|+++ | 5 % ~01s
|++++ | 7 % ~01s
|++++ | 8 % ~01s
|+++++ | 9 % ~01s
|+++++ | 10% ~01s
|++++++ | 11% ~01s
|++++++ | 12% ~01s
|+++++++ | 13% ~01s
|++++++++ | 14% ~01s
|++++++++ | 15% ~01s
|+++++++++ | 16% ~01s
|+++++++++ | 17% ~01s
|++++++++++ | 18% ~01s
|++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|+++++++++++ | 22% ~01s
|++++++++++++ | 23% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|++++++++++++++ | 26% ~01s
|++++++++++++++ | 27% ~01s
|+++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 30% ~01s
|++++++++++++++++ | 32% ~01s
|+++++++++++++++++ | 33% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|++++++++++++++++++ | 36% ~01s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|+++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 41% ~01s
|++++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|+++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|+++++++++++++++++++++++++ | 50% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|+++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|++++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~00s
|+++++++++++++++++++++++++++++ | 57% ~00s
|+++++++++++++++++++++++++++++ | 58% ~00s
|++++++++++++++++++++++++++++++ | 59% ~00s
|++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++ | 61% ~00s
|+++++++++++++++++++++++++++++++ | 62% ~00s
|++++++++++++++++++++++++++++++++ | 63% ~00s
|+++++++++++++++++++++++++++++++++ | 64% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|++++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 67% ~00s
|+++++++++++++++++++++++++++++++++++ | 68% ~00s
|+++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|+++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s Calculating cluster 1
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 2 % ~01s
|++ | 3 % ~01s
|+++ | 4 % ~01s
|+++ | 6 % ~01s
|++++ | 7 % ~01s
|++++ | 8 % ~01s
|+++++ | 9 % ~01s
|++++++ | 10% ~01s
|++++++ | 11% ~01s
|+++++++ | 12% ~01s
|+++++++ | 13% ~01s
|++++++++ | 15% ~01s
|++++++++ | 16% ~01s
|+++++++++ | 17% ~01s
|+++++++++ | 18% ~01s
|++++++++++ | 19% ~01s
|+++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|++++++++++++ | 22% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|+++++++++++++ | 26% ~01s
|++++++++++++++ | 27% ~01s
|+++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 30% ~01s
|++++++++++++++++ | 31% ~01s
|+++++++++++++++++ | 33% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|++++++++++++++++++ | 36% ~01s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|+++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|++++++++++++++++++++++ | 44% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|++++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|+++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|+++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|+++++++++++++++++++++++++++++ | 56% ~00s
|+++++++++++++++++++++++++++++ | 57% ~00s
|++++++++++++++++++++++++++++++ | 58% ~00s
|++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++ | 61% ~00s
|+++++++++++++++++++++++++++++++ | 62% ~00s
|++++++++++++++++++++++++++++++++ | 63% ~00s
|+++++++++++++++++++++++++++++++++ | 64% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|++++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 67% ~00s
|+++++++++++++++++++++++++++++++++++ | 69% ~00s
|+++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|++++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s Calculating cluster 2
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 2 % ~01s
|++ | 4 % ~01s
|+++ | 5 % ~01s
|++++ | 6 % ~01s
|++++ | 7 % ~01s
|+++++ | 8 % ~01s
|+++++ | 10% ~01s
|++++++ | 11% ~01s
|+++++++ | 12% ~01s
|+++++++ | 13% ~01s
|++++++++ | 14% ~01s
|++++++++ | 16% ~01s
|+++++++++ | 17% ~01s
|++++++++++ | 18% ~01s
|++++++++++ | 19% ~01s
|+++++++++++ | 20% ~01s
|+++++++++++ | 22% ~01s
|++++++++++++ | 23% ~01s
|+++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|++++++++++++++ | 27% ~01s
|++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 30% ~01s
|++++++++++++++++ | 31% ~01s
|+++++++++++++++++ | 33% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|+++++++++++++++++++ | 36% ~01s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 39% ~01s
|++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 41% ~01s
|++++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|+++++++++++++++++++++++ | 46% ~00s
|++++++++++++++++++++++++ | 47% ~00s
|+++++++++++++++++++++++++ | 48% ~00s
|+++++++++++++++++++++++++ | 49% ~00s
|++++++++++++++++++++++++++ | 51% ~00s
|++++++++++++++++++++++++++ | 52% ~00s
|+++++++++++++++++++++++++++ | 53% ~00s
|++++++++++++++++++++++++++++ | 54% ~00s
|++++++++++++++++++++++++++++ | 55% ~00s
|+++++++++++++++++++++++++++++ | 57% ~00s
|+++++++++++++++++++++++++++++ | 58% ~00s
|++++++++++++++++++++++++++++++ | 59% ~00s
|+++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++ | 61% ~00s
|++++++++++++++++++++++++++++++++ | 63% ~00s
|++++++++++++++++++++++++++++++++ | 64% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|++++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 67% ~00s
|+++++++++++++++++++++++++++++++++++ | 69% ~00s
|+++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|+++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s Calculating cluster 3
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 3 % ~01s
|+++ | 4 % ~01s
|+++ | 6 % ~01s
|++++ | 7 % ~01s
|+++++ | 9 % ~01s
|++++++ | 10% ~01s
|++++++ | 12% ~01s
|+++++++ | 13% ~01s
|++++++++ | 15% ~01s
|+++++++++ | 16% ~01s
|+++++++++ | 18% ~01s
|++++++++++ | 19% ~01s
|+++++++++++ | 21% ~01s
|++++++++++++ | 22% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|++++++++++++++ | 26% ~01s
|++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 31% ~01s
|+++++++++++++++++ | 32% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 41% ~00s
|++++++++++++++++++++++ | 43% ~00s
|+++++++++++++++++++++++ | 44% ~00s
|+++++++++++++++++++++++ | 46% ~00s
|++++++++++++++++++++++++ | 47% ~00s
|+++++++++++++++++++++++++ | 49% ~00s
|+++++++++++++++++++++++++ | 50% ~00s
|++++++++++++++++++++++++++ | 51% ~00s
|+++++++++++++++++++++++++++ | 53% ~00s
|++++++++++++++++++++++++++++ | 54% ~00s
|++++++++++++++++++++++++++++ | 56% ~00s
|+++++++++++++++++++++++++++++ | 57% ~00s
|++++++++++++++++++++++++++++++ | 59% ~00s
|+++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++ | 62% ~00s
|++++++++++++++++++++++++++++++++ | 63% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|++++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 68% ~00s
|+++++++++++++++++++++++++++++++++++ | 69% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|+++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s Calculating cluster 4
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 3 % ~01s
|+++ | 4 % ~01s
|+++ | 6 % ~01s
|++++ | 7 % ~01s
|+++++ | 8 % ~01s
|+++++ | 10% ~01s
|++++++ | 11% ~01s
|+++++++ | 13% ~01s
|++++++++ | 14% ~01s
|++++++++ | 15% ~01s
|+++++++++ | 17% ~01s
|++++++++++ | 18% ~01s
|++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|++++++++++++ | 23% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|++++++++++++++ | 27% ~01s
|+++++++++++++++ | 28% ~01s
|+++++++++++++++ | 30% ~01s
|++++++++++++++++ | 31% ~01s
|+++++++++++++++++ | 32% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|+++++++++++++++++++++ | 41% ~01s
|++++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 44% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|++++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|+++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 54% ~00s
|++++++++++++++++++++++++++++ | 55% ~00s
|+++++++++++++++++++++++++++++ | 56% ~00s
|+++++++++++++++++++++++++++++ | 58% ~00s
|++++++++++++++++++++++++++++++ | 59% ~00s
|+++++++++++++++++++++++++++++++ | 61% ~00s
|+++++++++++++++++++++++++++++++ | 62% ~00s
|++++++++++++++++++++++++++++++++ | 63% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|++++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 68% ~00s
|+++++++++++++++++++++++++++++++++++ | 69% ~00s
|++++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s Calculating cluster 5
| | 0 % ~calculating
|+ | 1 % ~03s
|++ | 2 % ~03s
|++ | 3 % ~03s
|+++ | 4 % ~03s
|+++ | 5 % ~03s
|++++ | 6 % ~03s
|++++ | 7 % ~03s
|+++++ | 8 % ~03s
|+++++ | 9 % ~03s
|++++++ | 10% ~03s
|++++++ | 11% ~02s
|+++++++ | 12% ~02s
|+++++++ | 13% ~02s
|++++++++ | 14% ~02s
|++++++++ | 15% ~02s
|+++++++++ | 16% ~02s
|+++++++++ | 17% ~02s
|++++++++++ | 18% ~02s
|++++++++++ | 19% ~02s
|+++++++++++ | 20% ~02s
|+++++++++++ | 21% ~02s
|++++++++++++ | 22% ~02s
|++++++++++++ | 23% ~02s
|+++++++++++++ | 24% ~02s
|+++++++++++++ | 26% ~02s
|++++++++++++++ | 27% ~02s
|++++++++++++++ | 28% ~02s
|+++++++++++++++ | 29% ~02s
|+++++++++++++++ | 30% ~02s
|++++++++++++++++ | 31% ~02s
|++++++++++++++++ | 32% ~02s
|+++++++++++++++++ | 33% ~02s
|+++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 37% ~02s
|+++++++++++++++++++ | 38% ~02s
|++++++++++++++++++++ | 39% ~02s
|++++++++++++++++++++ | 40% ~02s
|+++++++++++++++++++++ | 41% ~02s
|+++++++++++++++++++++ | 42% ~02s
|++++++++++++++++++++++ | 43% ~02s
|++++++++++++++++++++++ | 44% ~02s
|+++++++++++++++++++++++ | 45% ~02s
|+++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|+++++++++++++++++++++++++ | 50% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|+++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|++++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|+++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|++++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|+++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|++++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|+++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|++++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|++++++++++++++++++++++++++++++++++++ | 70% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|+++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|++++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s Calculating cluster 6
| | 0 % ~calculating
|+ | 1 % ~02s
|+ | 2 % ~01s
|++ | 3 % ~01s
|++ | 4 % ~01s
|+++ | 5 % ~01s
|+++ | 6 % ~01s
|++++ | 7 % ~02s
|++++ | 8 % ~02s
|+++++ | 9 % ~02s
|+++++ | 10% ~02s
|++++++ | 11% ~01s
|++++++ | 12% ~01s
|+++++++ | 13% ~01s
|+++++++ | 14% ~01s
|++++++++ | 15% ~01s
|++++++++ | 16% ~01s
|+++++++++ | 17% ~01s
|+++++++++ | 18% ~01s
|++++++++++ | 19% ~01s
|++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|+++++++++++ | 22% ~01s
|++++++++++++ | 23% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|+++++++++++++ | 26% ~01s
|++++++++++++++ | 27% ~01s
|++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|+++++++++++++++ | 30% ~01s
|++++++++++++++++ | 31% ~01s
|++++++++++++++++ | 32% ~01s
|+++++++++++++++++ | 33% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|++++++++++++++++++ | 36% ~01s
|+++++++++++++++++++ | 37% ~01s
|+++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 41% ~01s
|+++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|++++++++++++++++++++++ | 44% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|+++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|+++++++++++++++++++++++++ | 50% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|+++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|+++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|+++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|+++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|+++++++++++++++++++++++++++++++++++ | 70% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|+++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=02s Calculating cluster 7
| | 0 % ~calculating
|+ | 1 % ~03s
|++ | 2 % ~03s
|++ | 3 % ~03s
|+++ | 5 % ~03s
|+++ | 6 % ~03s
|++++ | 7 % ~03s
|++++ | 8 % ~03s
|+++++ | 9 % ~03s
|++++++ | 10% ~03s
|++++++ | 11% ~03s
|+++++++ | 12% ~03s
|+++++++ | 14% ~03s
|++++++++ | 15% ~03s
|++++++++ | 16% ~03s
|+++++++++ | 17% ~03s
|++++++++++ | 18% ~03s
|++++++++++ | 19% ~03s
|+++++++++++ | 20% ~03s
|+++++++++++ | 22% ~02s
|++++++++++++ | 23% ~02s
|++++++++++++ | 24% ~02s
|+++++++++++++ | 25% ~02s
|++++++++++++++ | 26% ~02s
|++++++++++++++ | 27% ~02s
|+++++++++++++++ | 28% ~02s
|+++++++++++++++ | 30% ~02s
|++++++++++++++++ | 31% ~02s
|++++++++++++++++ | 32% ~02s
|+++++++++++++++++ | 33% ~02s
|++++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|+++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 38% ~02s
|++++++++++++++++++++ | 39% ~02s
|++++++++++++++++++++ | 40% ~02s
|+++++++++++++++++++++ | 41% ~02s
|++++++++++++++++++++++ | 42% ~02s
|++++++++++++++++++++++ | 43% ~02s
|+++++++++++++++++++++++ | 44% ~02s
|+++++++++++++++++++++++ | 45% ~02s
|++++++++++++++++++++++++ | 47% ~02s
|++++++++++++++++++++++++ | 48% ~02s
|+++++++++++++++++++++++++ | 49% ~02s
|+++++++++++++++++++++++++ | 50% ~02s
|++++++++++++++++++++++++++ | 51% ~02s
|+++++++++++++++++++++++++++ | 52% ~02s
|+++++++++++++++++++++++++++ | 53% ~02s
|++++++++++++++++++++++++++++ | 55% ~02s
|++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|+++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|+++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|++++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|+++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|++++++++++++++++++++++++++++++++++++ | 70% ~01s
|++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|+++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 75% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 82% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s Calculating cluster 8
| | 0 % ~calculating
|+ | 1 % ~02s
|++ | 2 % ~02s
|++ | 3 % ~02s
|+++ | 4 % ~02s
|+++ | 6 % ~02s
|++++ | 7 % ~02s
|++++ | 8 % ~02s
|+++++ | 9 % ~02s
|++++++ | 10% ~02s
|++++++ | 11% ~02s
|+++++++ | 12% ~02s
|+++++++ | 13% ~02s
|++++++++ | 15% ~02s
|++++++++ | 16% ~02s
|+++++++++ | 17% ~02s
|+++++++++ | 18% ~02s
|++++++++++ | 19% ~02s
|+++++++++++ | 20% ~02s
|+++++++++++ | 21% ~02s
|++++++++++++ | 22% ~02s
|++++++++++++ | 24% ~02s
|+++++++++++++ | 25% ~02s
|+++++++++++++ | 26% ~02s
|++++++++++++++ | 27% ~02s
|+++++++++++++++ | 28% ~02s
|+++++++++++++++ | 29% ~02s
|++++++++++++++++ | 30% ~02s
|++++++++++++++++ | 31% ~02s
|+++++++++++++++++ | 33% ~02s
|+++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 37% ~01s
|++++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|+++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|++++++++++++++++++++++ | 44% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|++++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|+++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|+++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|+++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|++++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|+++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|+++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|++++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|+++++++++++++++++++++++++++++++++++ | 70% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|++++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 75% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=02s Calculating cluster 9
| | 0 % ~calculating
|+ | 1 % ~04s
|++ | 2 % ~04s
|++ | 3 % ~04s
|+++ | 4 % ~04s
|+++ | 5 % ~03s
|++++ | 6 % ~03s
|++++ | 7 % ~03s
|+++++ | 8 % ~03s
|+++++ | 9 % ~03s
|++++++ | 10% ~03s
|++++++ | 11% ~03s
|+++++++ | 12% ~03s
|+++++++ | 14% ~03s
|++++++++ | 15% ~03s
|++++++++ | 16% ~03s
|+++++++++ | 17% ~03s
|+++++++++ | 18% ~03s
|++++++++++ | 19% ~03s
|++++++++++ | 20% ~03s
|+++++++++++ | 21% ~03s
|+++++++++++ | 22% ~03s
|++++++++++++ | 23% ~03s
|++++++++++++ | 24% ~03s
|+++++++++++++ | 25% ~02s
|++++++++++++++ | 26% ~02s
|++++++++++++++ | 27% ~02s
|+++++++++++++++ | 28% ~02s
|+++++++++++++++ | 29% ~02s
|++++++++++++++++ | 30% ~02s
|++++++++++++++++ | 31% ~02s
|+++++++++++++++++ | 32% ~02s
|+++++++++++++++++ | 33% ~02s
|++++++++++++++++++ | 34% ~02s
|++++++++++++++++++ | 35% ~02s
|+++++++++++++++++++ | 36% ~02s
|+++++++++++++++++++ | 38% ~02s
|++++++++++++++++++++ | 39% ~02s
|++++++++++++++++++++ | 40% ~02s
|+++++++++++++++++++++ | 41% ~02s
|+++++++++++++++++++++ | 42% ~02s
|++++++++++++++++++++++ | 43% ~02s
|++++++++++++++++++++++ | 44% ~02s
|+++++++++++++++++++++++ | 45% ~02s
|+++++++++++++++++++++++ | 46% ~02s
|++++++++++++++++++++++++ | 47% ~02s
|++++++++++++++++++++++++ | 48% ~02s
|+++++++++++++++++++++++++ | 49% ~02s
|+++++++++++++++++++++++++ | 50% ~02s
|++++++++++++++++++++++++++ | 51% ~02s
|+++++++++++++++++++++++++++ | 52% ~02s
|+++++++++++++++++++++++++++ | 53% ~02s
|++++++++++++++++++++++++++++ | 54% ~02s
|++++++++++++++++++++++++++++ | 55% ~01s
|+++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|++++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|+++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|++++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|+++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|+++++++++++++++++++++++++++++++++++ | 70% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~01s
|+++++++++++++++++++++++++++++++++++++ | 74% ~01s
|++++++++++++++++++++++++++++++++++++++ | 75% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~01s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 78% ~01s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 80% ~01s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~01s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~01s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 90% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=03s Calculating cluster 10
| | 0 % ~calculating
|+ | 1 % ~02s
|++ | 2 % ~02s
|++ | 3 % ~02s
|+++ | 4 % ~02s
|+++ | 5 % ~02s
|++++ | 6 % ~02s
|++++ | 7 % ~02s
|+++++ | 8 % ~02s
|+++++ | 9 % ~02s
|++++++ | 11% ~02s
|++++++ | 12% ~02s
|+++++++ | 13% ~02s
|+++++++ | 14% ~02s
|++++++++ | 15% ~02s
|++++++++ | 16% ~02s
|+++++++++ | 17% ~02s
|+++++++++ | 18% ~02s
|++++++++++ | 19% ~02s
|++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|++++++++++++ | 22% ~01s
|++++++++++++ | 23% ~01s
|+++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|++++++++++++++ | 26% ~01s
|++++++++++++++ | 27% ~01s
|+++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 31% ~01s
|++++++++++++++++ | 32% ~01s
|+++++++++++++++++ | 33% ~01s
|+++++++++++++++++ | 34% ~01s
|++++++++++++++++++ | 35% ~01s
|++++++++++++++++++ | 36% ~01s
|+++++++++++++++++++ | 37% ~01s
|+++++++++++++++++++ | 38% ~01s
|++++++++++++++++++++ | 39% ~01s
|++++++++++++++++++++ | 40% ~01s
|+++++++++++++++++++++ | 41% ~01s
|++++++++++++++++++++++ | 42% ~01s
|++++++++++++++++++++++ | 43% ~01s
|+++++++++++++++++++++++ | 44% ~01s
|+++++++++++++++++++++++ | 45% ~01s
|++++++++++++++++++++++++ | 46% ~01s
|++++++++++++++++++++++++ | 47% ~01s
|+++++++++++++++++++++++++ | 48% ~01s
|+++++++++++++++++++++++++ | 49% ~01s
|++++++++++++++++++++++++++ | 51% ~01s
|++++++++++++++++++++++++++ | 52% ~01s
|+++++++++++++++++++++++++++ | 53% ~01s
|+++++++++++++++++++++++++++ | 54% ~01s
|++++++++++++++++++++++++++++ | 55% ~01s
|++++++++++++++++++++++++++++ | 56% ~01s
|+++++++++++++++++++++++++++++ | 57% ~01s
|+++++++++++++++++++++++++++++ | 58% ~01s
|++++++++++++++++++++++++++++++ | 59% ~01s
|++++++++++++++++++++++++++++++ | 60% ~01s
|+++++++++++++++++++++++++++++++ | 61% ~01s
|++++++++++++++++++++++++++++++++ | 62% ~01s
|++++++++++++++++++++++++++++++++ | 63% ~01s
|+++++++++++++++++++++++++++++++++ | 64% ~01s
|+++++++++++++++++++++++++++++++++ | 65% ~01s
|++++++++++++++++++++++++++++++++++ | 66% ~01s
|++++++++++++++++++++++++++++++++++ | 67% ~01s
|+++++++++++++++++++++++++++++++++++ | 68% ~01s
|+++++++++++++++++++++++++++++++++++ | 69% ~01s
|++++++++++++++++++++++++++++++++++++ | 71% ~01s
|++++++++++++++++++++++++++++++++++++ | 72% ~01s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|+++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 77% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 83% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 97% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=02s pbmc.markers %>%
group_by(cluster) %>%
slice_max(n = 2, order_by = avg_log2FC)cluster0.markers <- FindMarkers(pbmc, ident.1 = 0, logfc.threshold = 0.25, test.use = "roc", only.pos = TRUE)
| | 0 % ~calculating
|+ | 1 % ~01s
|++ | 2 % ~01s
|++ | 4 % ~01s
|+++ | 5 % ~01s
|+++ | 6 % ~01s
|++++ | 7 % ~01s
|+++++ | 8 % ~01s
|+++++ | 9 % ~01s
|++++++ | 11% ~01s
|++++++ | 12% ~01s
|+++++++ | 13% ~01s
|++++++++ | 14% ~01s
|++++++++ | 15% ~01s
|+++++++++ | 16% ~01s
|+++++++++ | 18% ~01s
|++++++++++ | 19% ~01s
|++++++++++ | 20% ~01s
|+++++++++++ | 21% ~01s
|++++++++++++ | 22% ~01s
|++++++++++++ | 24% ~01s
|+++++++++++++ | 25% ~01s
|+++++++++++++ | 26% ~01s
|++++++++++++++ | 27% ~01s
|+++++++++++++++ | 28% ~01s
|+++++++++++++++ | 29% ~01s
|++++++++++++++++ | 31% ~00s
|++++++++++++++++ | 32% ~00s
|+++++++++++++++++ | 33% ~00s
|++++++++++++++++++ | 34% ~00s
|++++++++++++++++++ | 35% ~00s
|+++++++++++++++++++ | 36% ~00s
|+++++++++++++++++++ | 38% ~00s
|++++++++++++++++++++ | 39% ~00s
|++++++++++++++++++++ | 40% ~00s
|+++++++++++++++++++++ | 41% ~00s
|++++++++++++++++++++++ | 42% ~00s
|++++++++++++++++++++++ | 44% ~00s
|+++++++++++++++++++++++ | 45% ~00s
|+++++++++++++++++++++++ | 46% ~00s
|++++++++++++++++++++++++ | 47% ~00s
|+++++++++++++++++++++++++ | 48% ~00s
|+++++++++++++++++++++++++ | 49% ~00s
|++++++++++++++++++++++++++ | 51% ~00s
|++++++++++++++++++++++++++ | 52% ~00s
|+++++++++++++++++++++++++++ | 53% ~00s
|++++++++++++++++++++++++++++ | 54% ~00s
|++++++++++++++++++++++++++++ | 55% ~00s
|+++++++++++++++++++++++++++++ | 56% ~00s
|+++++++++++++++++++++++++++++ | 58% ~00s
|++++++++++++++++++++++++++++++ | 59% ~00s
|++++++++++++++++++++++++++++++ | 60% ~00s
|+++++++++++++++++++++++++++++++ | 61% ~00s
|++++++++++++++++++++++++++++++++ | 62% ~00s
|++++++++++++++++++++++++++++++++ | 64% ~00s
|+++++++++++++++++++++++++++++++++ | 65% ~00s
|+++++++++++++++++++++++++++++++++ | 66% ~00s
|++++++++++++++++++++++++++++++++++ | 67% ~00s
|+++++++++++++++++++++++++++++++++++ | 68% ~00s
|+++++++++++++++++++++++++++++++++++ | 69% ~00s
|++++++++++++++++++++++++++++++++++++ | 71% ~00s
|++++++++++++++++++++++++++++++++++++ | 72% ~00s
|+++++++++++++++++++++++++++++++++++++ | 73% ~00s
|++++++++++++++++++++++++++++++++++++++ | 74% ~00s
|++++++++++++++++++++++++++++++++++++++ | 75% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 76% ~00s
|+++++++++++++++++++++++++++++++++++++++ | 78% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 79% ~00s
|++++++++++++++++++++++++++++++++++++++++ | 80% ~00s
|+++++++++++++++++++++++++++++++++++++++++ | 81% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 82% ~00s
|++++++++++++++++++++++++++++++++++++++++++ | 84% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 85% ~00s
|+++++++++++++++++++++++++++++++++++++++++++ | 86% ~00s
|++++++++++++++++++++++++++++++++++++++++++++ | 87% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 88% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++ | 89% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 91% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++ | 92% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++ | 93% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 94% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++ | 95% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 96% ~00s
|+++++++++++++++++++++++++++++++++++++++++++++++++ | 98% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 99% ~00s
|++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s VlnPlot(pbmc, features = c("S100A8", "S100A9"))
# you can plot raw counts as well
VlnPlot(pbmc, features = c("S100A8", "S100A9"), slot = "counts", log = TRUE)
FeaturePlot(pbmc, features = c("MS4A1", "GNLY", "CD3E", "CD14", "FCER1A", "FCGR3A", "LYZ", "PPBP", "CD8A"))
FeaturePlot(pbmc, features = c("MS4A1", "GNLY"))
pbmc.markers %>%
group_by(cluster) %>%
top_n(n = 10, wt = avg_log2FC) -> top10
DoHeatmap(pbmc, features = top10$gene) + NoLegend()
LS0tCnRpdGxlOiAiU2V1cmF0X1R1dG9yaWFsX1BCTSIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQpJbiB0aGlzIHR1dG9yaWFsLCB3ZSB3aWxsIGJlIGxvb2tpbmcgYXQgYSBkYXRhc2V0IG9mIFBlcmlwaGVyYWwgQmxvb2QgTW9ub251Y2xlYXIgQ2VsbHMgKFBCTUMpLiBQQk1DcyBhcmUgbGlrZSB0aGUgYXJteSBvZiBpbW11bmUgY2VsbHMgdGhhdCBsaXZlIGluIG91ciBibG9vZCBhbmQgZmlnaHQgb2ZmIGFueSBpbnZhZGVycyBsaWtlIHZpcnVzZXMgb3IgYmFjdGVyaWEuCgpUaGUgZGF0YXNldCB3ZSBoYXZlIGhhcyAyLDcwMCBzaW5nbGUgUEJNQ3MgdGhhdCBoYXZlIGJlZW4gc2VxdWVuY2VkIHVzaW5nIGEgZmFuY3kgSWxsdW1pbmEgbWFjaGluZSBjYWxsZWQgdGhlIE5leHRTZXEgNTAwLiBXZSdsbCBiZSB1c2luZyBhIGZ1bmN0aW9uIGNhbGxlZCBSZWFkMTBYKCkgdG8gcmVhZCBpbiB0aGUgZGF0YSBmcm9tIHRoZSAxMFggR2Vub21pY3MgcGlwZWxpbmUuIFRoaXMgZnVuY3Rpb24gd2lsbCBnaXZlIHVzIGEgbWF0cml4IHRoYXQgdGVsbHMgdXMgaG93IG1hbnkgZGlmZmVyZW50IG1vbGVjdWxlcyBvZiBlYWNoIGdlbmUgd2VyZSBkZXRlY3RlZCBpbiBlYWNoIG9mIHRoZSAyLDcwMCBQQk1Dcy4KCldlJ2xsIHRoZW4gdXNlIHRoaXMgbWF0cml4IHRvIGNyZWF0ZSBhIFNldXJhdCBvYmplY3QuIFRoaW5rIG9mIHRoaXMgYXMgY3JlYXRpbmcgYSAiZmVsbG93c2hpcCBvZiBjZWxscyIsIGxpa2UgaW4gTG9yZCBvZiB0aGUgUmluZ3MuCgpIZXJlJ3MgdGhlIGNvZGUgd2UnbGwgdXNlIHRvIGxvYWQgaW4gdGhlIGRhdGFzZXQgYW5kIGNyZWF0ZSBvdXIgU2V1cmF0IG9iamVjdDoKCmBgYHtyfQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KFNldXJhdCkKbGlicmFyeShwYXRjaHdvcmspCnBibWMuZGF0YSA8LSBSZWFkMTBYKGRhdGEuZGlyID0gInBibWMzay9maWx0ZXJlZF9nZW5lX2JjX21hdHJpY2VzL2hnMTkvIikKcGJtYy5kYXRhICMzMjczOCB4IDI3MDAgc3BhcnNlIE1hdHJpeCBvZiBjbGFzcyAiZGdDTWF0cml4IgojYSBjbGFzcyBvZiBzcGFyc2UgbnVtZXJpYyBtYXRyaWNlcyBpbiB0aGUgY29tcHJlc3NlZCwgc3BhcnNlLCBjb2x1bW4tb3JpZW50ZWQgZm9ybWF0LiBJbiB0aGlzIGltcGxlbWVudGF0aW9uIHRoZSBub24temVybyBlbGVtZW50cyBpbiB0aGUgY29sdW1ucyBhcmUgc29ydGVkIGludG8gaW5jcmVhc2luZyByb3cgb3JkZXIuIGRnQ01hdHJpeCBpcyB0aGUg4oCcc3RhbmRhcmTigJ0gY2xhc3MgZm9yIHNwYXJzZSBudW1lcmljIG1hdHJpY2VzIGluIHRoZSBNYXRyaXggcGFja2FnZS4KCmBgYAoKCmBgYHtyfQpwYm1jIDwtIENyZWF0ZVNldXJhdE9iamVjdChjb3VudHMgPSBwYm1jLmRhdGEsIHByb2plY3QgPSAicGJtYzNrIiwgbWluLmNlbGxzID0gMywgbWluLmZlYXR1cmVzID0gMjAwKQpwYm1jCmBgYAoKTmV4dCwgd2UnbGwgbmVlZCB0byBkbyBzb21lIHByZS1wcm9jZXNzaW5nIG9uIG91ciBkYXRhIHRvIG1ha2Ugc3VyZSBpdCdzIHJlYWR5IGZvciBhbmFseXNpcy4gVGhpcyBpbnZvbHZlcyBzZWxlY3RpbmcgYW5kIGZpbHRlcmluZyB0aGUgYmVzdCBxdWFsaXR5IGNlbGxzIGJhc2VkIG9uIHZhcmlvdXMgbWV0cmljcywgbm9ybWFsaXppbmcgYW5kIHNjYWxpbmcgdGhlIGRhdGEsIGFuZCBpZGVudGlmeWluZyBoaWdobHkgdmFyaWFibGUgZ2VuZXMgdGhhdCB3aWxsIGJlIGltcG9ydGFudCBmb3Igb3VyIGFuYWx5c2lzLiAKVGhpbmsgb2YgdGhpcyBhcyBwcmVwYXJpbmcgb3VyIGZlbGxvd3NoaXAgZm9yIGJhdHRsZSBieSBtYWtpbmcgc3VyZSBldmVyeW9uZSBpcyBoZWFsdGh5IGFuZCBlcXVpcHBlZCB3aXRoIHRoZSByaWdodCB3ZWFwb25zLgoKKipTdGFuZGFyZCBwcmUtcHJvY2Vzc2luZyB3b3JrZmxvdyoqCgpBIGZldyBRQyBtZXRyaWNzIGNvbW1vbmx5IHVzZWQgYnkgdGhlIGNvbW11bml0eSBpbmNsdWRlIAotIHRoZSBudW1iZXIgb2YgdW5pcXVlIGdlbmVzIGRldGVjdGVkIGluIGVhY2ggY2VsbC4KLSBMb3ctcXVhbGl0eSBjZWxscyBvciBlbXB0eSBkcm9wbGV0cyB3aWxsIG9mdGVuIGhhdmUgdmVyeSBmZXcgZ2VuZXMKLSBDZWxsIGRvdWJsZXRzIG9yIG11bHRpcGxldHMgbWF5IGV4aGliaXQgYW4gYWJlcnJhbnRseSBoaWdoIGdlbmUgY291bnQKLSBTaW1pbGFybHksIHRoZSB0b3RhbCBudW1iZXIgb2YgbW9sZWN1bGVzIGRldGVjdGVkIHdpdGhpbiBhIGNlbGwgKGNvcnJlbGF0ZXMgc3Ryb25nbHkgd2l0aCB1bmlxdWUgZ2VuZXMpCi0gVGhlIHBlcmNlbnRhZ2Ugb2YgcmVhZHMgdGhhdCBtYXAgdG8gdGhlIG1pdG9jaG9uZHJpYWwgZ2Vub21lCi0gTG93LXF1YWxpdHkgLyBkeWluZyBjZWxscyBvZnRlbiBleGhpYml0IGV4dGVuc2l2ZSBtaXRvY2hvbmRyaWFsIGNvbnRhbWluYXRpb24KLSBXZSBjYWxjdWxhdGUgbWl0b2Nob25kcmlhbCBRQyBtZXRyaWNzIHdpdGggdGhlIFBlcmNlbnRhZ2VGZWF0dXJlU2V0KCkgZnVuY3Rpb24sIHdoaWNoIGNhbGN1bGF0ZXMgdGhlIHBlcmNlbnRhZ2Ugb2YgY291bnRzIG9yaWdpbmF0aW5nIGZyb20gYSBzZXQgb2YgZmVhdHVyZXMKLSBXZSB1c2UgdGhlIHNldCBvZiBhbGwgZ2VuZXMgc3RhcnRpbmcgd2l0aCBNVC0gYXMgYSBzZXQgb2YgbWl0b2Nob25kcmlhbCBnZW5lcwogICAgIApgYGB7cn0KIyBUaGUgW1sgb3BlcmF0b3IgY2FuIGFkZCBjb2x1bW5zIHRvIG9iamVjdCBtZXRhZGF0YS4gVGhpcyBpcyBhIGdyZWF0IHBsYWNlIHRvIHN0YXNoIFFDIHN0YXRzCnBibWNbWyJwZXJjZW50Lm10Il1dIDwtIFBlcmNlbnRhZ2VGZWF0dXJlU2V0KHBibWMsIHBhdHRlcm4gPSAiXk1ULSIpCmhlYWQocGJtY0BtZXRhLmRhdGEsIDUpCmBgYAogICAKSW4gdGhlIGV4YW1wbGUgYmVsb3csIHdlIHZpc3VhbGl6ZSBRQyBtZXRyaWNzLCBhbmQgdXNlIHRoZXNlIHRvIGZpbHRlciBjZWxscy4KV2UgZmlsdGVyIGNlbGxzIHRoYXQgaGF2ZSB1bmlxdWUgZmVhdHVyZSBjb3VudHMgb3ZlciAyLDUwMCBvciBsZXNzIHRoYW4gMjAwCldlIGZpbHRlciBjZWxscyB0aGF0IGhhdmUgPjUlIG1pdG9jaG9uZHJpYWwgY291bnRzCiAgICAKICAKYGBge3J9CiMgVmlzdWFsaXplIFFDIG1ldHJpY3MgYXMgYSB2aW9saW4gcGxvdApWbG5QbG90KHBibWMsIGZlYXR1cmVzID0gYygibkZlYXR1cmVfUk5BIiwgIm5Db3VudF9STkEiLCAicGVyY2VudC5tdCIpLCBuY29sID0gMykKYGBgCgpgYGB7cn0KIyBGZWF0dXJlU2NhdHRlciBpcyB0eXBpY2FsbHkgdXNlZCB0byB2aXN1YWxpemUgZmVhdHVyZS1mZWF0dXJlIHJlbGF0aW9uc2hpcHMsIGJ1dCBjYW4gYmUgdXNlZCBmb3IgYW55dGhpbmcgY2FsY3VsYXRlZCBieSB0aGUgb2JqZWN0LCBpLmUuIGNvbHVtbnMgaW4gb2JqZWN0IG1ldGFkYXRhLCBQQyBzY29yZXMgZXRjLgoKcGxvdDEgPC0gRmVhdHVyZVNjYXR0ZXIocGJtYywgZmVhdHVyZTEgPSAibkNvdW50X1JOQSIsIGZlYXR1cmUyID0gInBlcmNlbnQubXQiKQpwbG90MiA8LSBGZWF0dXJlU2NhdHRlcihwYm1jLCBmZWF0dXJlMSA9ICJuQ291bnRfUk5BIiwgZmVhdHVyZTIgPSAibkZlYXR1cmVfUk5BIikKcGxvdDEgKyBwbG90MgpgYGAKUGVhcnNvbiBjb3JyZWxhdGlvbiBiZXR3ZWVuIHRoZSB0d28gZmVhdHVyZXMgaXMgZGlzcGxheWVkIGFib3ZlIHRoZSBwbG90LgoqbkZlYXR1cmVfUk5BIHZzIG5Db3VudF9STkEqOgoKbkNvdW50IHJlcHJlc2VudHMgdGhlIHRvdGFsIG51bWJlciBvZiByZWFkcyBvciBtb2xlY3VsZXMgY2FwdHVyZWQgZm9yIGVhY2ggY2VsbCwgd2hpbGUgbkZlYXR1cmUgUk5BIGNvcnJlc3BvbmRzIHRvIHRoZSBudW1iZXIgb2YgdW5pcXVlIGdlbmVzIGRldGVjdGVkIGluIGVhY2ggY2VsbC4gQnkgcGxvdHRpbmcgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdGhlc2UgdHdvIHZhcmlhYmxlcywgcmVzZWFyY2hlcnMgY2FuIGFzc2VzcyB0aGUgb3ZlcmFsbCBkYXRhIHF1YWxpdHkuIENlbGxzIHdpdGggbG93IG5Db3VudCBhbmQgbkZlYXR1cmUgUk5BIG1heSBpbmRpY2F0ZSBwb29yIGNhcHR1cmUgb3IgbG93LXF1YWxpdHkgZGF0YSwgYW5kIHRoZXkgY2FuIGJlIHBvdGVudGlhbGx5IGV4Y2x1ZGVkIGZyb20gZG93bnN0cmVhbSBhbmFseXNlcyB0byBlbnN1cmUgcm9idXN0IHJlc3VsdHMuIEhlcmUgd2Ugc2VlIHRoYXQgd2UgZG8gbm90IHBvb3IgcXVhbGl0eSBkYXRhLiBJbiBnZW5lcmFsIHdlIGRvIHdhbnQgdG8gc2VlIGhpZ2hlciBjb3JyZWxhdGlvbiBiZXR3ZWVuIHRoZXNlIHR3byB2YXJpYWJsZXMuCgpPdXRsaWVycyBjYW4gYXJpc2UgZHVlIHRvIHZhcmlvdXMgdGVjaG5pY2FsIG9yIGJpb2xvZ2ljYWwgZmFjdG9ycywgc3VjaCBhcyBjZWxsIGRvdWJsZXRzIG9yIGV4cGVyaW1lbnRhbCBhcnRpZmFjdHMuIFRoZSBjb3JyZWxhdGlvbiBwbG90IGhlbHBzIHRvIGlkZW50aWZ5IHN1Y2ggb3V0bGllcnMgYnkgaGlnaGxpZ2h0aW5nIGNlbGxzIHRoYXQgZGV2aWF0ZSBmcm9tIHRoZSBleHBlY3RlZCByZWxhdGlvbnNoaXAgYmV0d2VlbiBuQ291bnQgYW5kIG5GZWF0dXJlIFJOQS4KCiAKYGBge3J9CnBibWMgPC0gc3Vic2V0KHBibWMsIHN1YnNldCA9IG5GZWF0dXJlX1JOQSA+IDIwMCAmIG5GZWF0dXJlX1JOQSA8IDI1MDAgJiBwZXJjZW50Lm10IDwgNSkKVmxuUGxvdChwYm1jLCBmZWF0dXJlcyA9IGMoIm5GZWF0dXJlX1JOQSIsICJuQ291bnRfUk5BIiwgInBlcmNlbnQubXQiKSwgbmNvbCA9IDMpCgpgYGAKQ29tcGFyZSB0aGUgYWJvdmUgVmlvbGluIHBsb3QgdG8gdW5kZXJzdGFuZCB3aGljaCBjZWxscyB3ZXJlIGZpbHRlcmVkLgoKYGBge3J9CiNTQ1RyYW5zZm9ybShvYmplY3QgPSBwYm1jKQojVXNlIHRoaXMgZnVuY3Rpb24gYXMgYW4gYWx0ZXJuYXRpdmUgdG8gdGhlIE5vcm1hbGl6ZURhdGEsIEZpbmRWYXJpYWJsZUZlYXR1cmVzLCBTY2FsZURhdGEgd29ya2Zsb3cKI1Jlc3VsdHMgYXJlIHNhdmVkIGluIGEgbmV3IGFzc2F5IChuYW1lZCBTQ1QgYnkgZGVmYXVsdCkgd2l0aCBjb3VudHMgYmVpbmcgKGNvcnJlY3RlZCkgY291bnRzLCBkYXRhIGJlaW5nIGxvZzFwKGNvdW50cyksIHNjYWxlLmRhdGEgYmVpbmcgcGVhcnNvbiByZXNpZHVhbHM7IHNjdHJhbnNmb3JtOjp2c3QgaW50ZXJtZWRpYXRlIHJlc3VsdHMgYXJlIHNhdmVkIGluIG1pc2Mgc2xvdCBvZiBuZXcgYXNzYXkuCmBgYAoKCioqTm9ybWFsaXppbmcgdGhlIGRhdGEqKgoKTm9ybWFsaXphdGlvbiBpcyBsaWtlIG1ha2luZyBzdXJlIGV2ZXJ5b25lIGluIG91ciBmZWxsb3dzaGlwIGlzIHNwZWFraW5nIHRoZSBzYW1lIGxhbmd1YWdlIGFuZCB1c2luZyB0aGUgc2FtZSBtZWFzdXJpbmcgc3lzdGVtLiBUaGlzIHdpbGwgaGVscCB1cyBjb21wYXJlIGdlbmUgZXhwcmVzc2lvbiBsZXZlbHMgYmV0d2VlbiBkaWZmZXJlbnQgY2VsbHMuCgpXZSB1dGlsaXplIGEgbm9ybWFsaXphdGlvbiB0ZWNobmlxdWUgY2FsbGVkICJMb2dOb3JtYWxpemUiIHRvIHN0YW5kYXJkaXplIHRoZSBmZWF0dXJlIGV4cHJlc3Npb24gbWVhc3VyZW1lbnRzIGZvciBlYWNoIGluZGl2aWR1YWwgY2VsbC4gVGhpcyBub3JtYWxpemF0aW9uIG1ldGhvZCBpbnZvbHZlcyBkaXZpZGluZyB0aGUgZXhwcmVzc2lvbiB2YWx1ZXMgb2YgZWFjaCBmZWF0dXJlIGJ5IHRoZSB0b3RhbCBleHByZXNzaW9uIG9mIHRoYXQgY2VsbCwgZm9sbG93ZWQgYnkgbXVsdGlwbGljYXRpb24gd2l0aCBhIHNjYWxpbmcgZmFjdG9yICh0eXBpY2FsbHkgc2V0IHRvIDEwLDAwMCBieSBkZWZhdWx0KS4gRmluYWxseSwgd2UgYXBwbHkgYSBsb2dhcml0aG1pYyB0cmFuc2Zvcm1hdGlvbiB0byB0aGUgbm9ybWFsaXplZCB2YWx1ZXMuIFRoaXMgcHJvY2VzcyBoZWxwcyBlbnN1cmUgdGhhdCB0aGUgZXhwcmVzc2lvbiB2YWx1ZXMgYXJlIGNvbXBhcmFibGUgYWNyb3NzIGNlbGxzIGFuZCBlbmFibGVzIHVzIHRvIGVmZmVjdGl2ZWx5IGFuYWx5emUgYW5kIGludGVycHJldCB0aGUgZGF0YS4KCmBgYHtyfQojcGJtYyA8LSBOb3JtYWxpemVEYXRhKHBibWMsIG5vcm1hbGl6YXRpb24ubWV0aG9kID0gIlJDIiwgc2NhbGUuZmFjdG9yID0gMWU2KQoKYGBgCgpgYGB7cn0KcGJtYyA8LSBOb3JtYWxpemVEYXRhKHBibWMsIG5vcm1hbGl6YXRpb24ubWV0aG9kID0gIkxvZ05vcm1hbGl6ZSIsIHNjYWxlLmZhY3RvciA9IDEwMDAwKQoKYGBgCmBgYHtyfQoKcGFyKG1mcm93PWMoMSwyKSkKIyBvcmlnaW5hbCBleHByZXNzaW9uIGRpc3RyaWJ1dGlvbgpyYXdfZ2VuZUV4cCA9IGFzLnZlY3RvcihwYm1jW1snUk5BJ11dQGNvdW50cykgJT4lIHNhbXBsZSgxMDAwMCkKcmF3X2dlbmVFeHAgPSByYXdfZ2VuZUV4cFtyYXdfZ2VuZUV4cCAhPSAwXQpoaXN0KHJhd19nZW5lRXhwKQojIGV4cHJlc3Npb24gZGlzdHJpYnV0aW9uIGFmdGVyIG5vcm1hbGl6YXRpb24KbG9nTm9ybV9nZW5lRXhwID0gYXMudmVjdG9yKHBibWNbWydSTkEnXV1AZGF0YSkgJT4lIHNhbXBsZSgxMDAwMCkKbG9nTm9ybV9nZW5lRXhwID0gbG9nTm9ybV9nZW5lRXhwW2xvZ05vcm1fZ2VuZUV4cCAhPSAwXQpoaXN0KGxvZ05vcm1fZ2VuZUV4cCkKYGBgCgoKKipJZGVudGlmaWNhdGlvbiBvZiBoaWdobHkgdmFyaWFibGUgZmVhdHVyZXMgKGZlYXR1cmUgc2VsZWN0aW9uKSoqCk5vdyB0aGF0IHdlJ3ZlIG5vcm1hbGl6ZWQgb3VyIGRhdGEsIHdlIG5lZWQgdG8gaWRlbnRpZnkgd2hpY2ggZ2VuZXMgYXJlIG1vc3QgaW1wb3J0YW50IGZvciBvdXIgYW5hbHlzaXMuIFdlIGRvIHRoaXMgYnkgaWRlbnRpZnlpbmcgaGlnaGx5IHZhcmlhYmxlIGdlbmVzLgoKSGlnaGx5IHZhcmlhYmxlIGdlbmVzIGFyZSBsaWtlIHRoZSByZWFsIGhlcm9lcyBpbiBvdXIgZmVsbG93c2hpcCwgdGhlIG9uZXMgd2hvIHN0YW5kIG91dCBmcm9tIHRoZSByZXN0IGFuZCBtYWtlIGEgYmlnIGltcGFjdC4gVGhlc2UgZ2VuZXMgZXhoaWJpdCBhIGxvdCBvZiB2YXJpYXRpb24gaW4gZXhwcmVzc2lvbiBsZXZlbHMgYmV0d2VlbiBjZWxscywgd2hpY2ggY2FuIGJlIGEgZ29vZCBpbmRpY2F0aW9uIG9mIGJpb2xvZ2ljYWwgc2lnbmlmaWNhbmNlLgoKSW4gJ3ZzdCcgbWV0aG9kIGl0IGZpdHMgYSBsaW5lIHRvIHRoZSByZWxhdGlvbnNoaXAgb2YgbG9nKHZhcmlhbmNlKSBhbmQgbG9nKG1lYW4pIHVzaW5nIGxvY2FsIHBvbHlub21pYWwgcmVncmVzc2lvbiAobG9lc3MpLiBUaGVuIHN0YW5kYXJkaXplcyB0aGUgZmVhdHVyZSB2YWx1ZXMgdXNpbmcgdGhlIG9ic2VydmVkIG1lYW4gYW5kIGV4cGVjdGVkIHZhcmlhbmNlIChnaXZlbiBieSB0aGUgZml0dGVkIGxpbmUpLiBGZWF0dXJlIHZhcmlhbmNlIGlzIHRoZW4gY2FsY3VsYXRlZCBvbiB0aGUgc3RhbmRhcmRpemVkIHZhbHVlcyBhZnRlciBjbGlwcGluZyB0byBhIG1heGltdW0gKHNlZSBjbGlwLm1heCBwYXJhbWV0ZXIpLgoKYGBge3J9CnBibWMgPC0gRmluZFZhcmlhYmxlRmVhdHVyZXMocGJtYywgc2VsZWN0aW9uLm1ldGhvZCA9ICJ2c3QiLCBuZmVhdHVyZXMgPSAyMDAwKQoKIyBJZGVudGlmeSB0aGUgMTAgbW9zdCBoaWdobHkgdmFyaWFibGUgZ2VuZXMKdG9wMTAgPC0gaGVhZChWYXJpYWJsZUZlYXR1cmVzKHBibWMpLCAxMCkKCmBgYAogQnkgZGVmYXVsdCwgd2UgcmV0dXJuIDIsMDAwIGZlYXR1cmVzIHBlciBkYXRhc2V0LiAgWW91IGNhbiBhZGp1c3QgdGhpcyBudW1iZXIgZGVwZW5kaW5nIG9uIHRoZSBzaXplIGFuZCBjb21wbGV4aXR5IG9mIHlvdXIgZGF0YXNldC4gCiAKYGBge3J9CiMgcGxvdCB2YXJpYWJsZSBmZWF0dXJlcyB3aXRoIGFuZCB3aXRob3V0IGxhYmVscwpwbG90MSA8LSBWYXJpYWJsZUZlYXR1cmVQbG90KHBibWMpCnBsb3QyIDwtIExhYmVsUG9pbnRzKHBsb3QgPSBwbG90MSwgcG9pbnRzID0gdG9wMTAsIHJlcGVsID0gVFJVRSwgeG51ZGdlPTAsIHludWRnZT0wKQpwbG90MgpgYGAKCk9uY2Ugd2UndmUgaWRlbnRpZmllZCBvdXIgaGlnaGx5IHZhcmlhYmxlIGdlbmVzLCB3ZSBjYW4gdXNlIHRoZW0gdG8gZm9jdXMgb3VyIGRvd25zdHJlYW0gYW5hbHlzaXMgYW5kIGhpZ2hsaWdodCB0aGUgYmlvbG9naWNhbCBzaWduYWwgaW4gb3VyIHNpbmdsZS1jZWxsIGRhdGFzZXQuIAoKS2VlcCB1cCB0aGUgZ3JlYXQgd29yaywgYW5kIGRvbid0IGZvcmdldCB0aGF0IGV2ZW4gdGhlIHNtYWxsZXN0IGhlcm9lcyBjYW4gaGF2ZSBhIGJpZyBpbXBhY3QhCgoqKlNjYWxpbmcgdGhlIGRhdGEqKgoKVGhlIG5leHQgc3RlcCBpcyB0byBhcHBseSBhIGxpbmVhciB0cmFuc2Zvcm1hdGlvbiBjYWxsZWQgInNjYWxpbmciLiBUaGlzIGlzIGEgc3RhbmRhcmQgcHJlLXByb2Nlc3Npbmcgc3RlcCB0aGF0IHdlIG5lZWQgdG8gZG8gYmVmb3JlIHdlIGNhbiB1c2UgdGVjaG5pcXVlcyBsaWtlIFBDQSAocHJpbmNpcGFsIGNvbXBvbmVudCBhbmFseXNpcykgdG8gYW5hbHl6ZSBvdXIgZGF0YS4KClNjYWxpbmcgaXMgbGlrZSBtYWtpbmcgc3VyZSB0aGF0IGV2ZXJ5b25lJ3Mgdm9pY2UgaXMgYXQgdGhlIHNhbWUgdm9sdW1lIGluIG91ciBmZWxsb3dzaGlwLiBXZSB3YW50IHRvIG1ha2Ugc3VyZSB0aGF0IGVhY2ggZ2VuZSBpcyBnaXZlbiBlcXVhbCBpbXBvcnRhbmNlIGluIGRvd25zdHJlYW0gYW5hbHlzaXMsIHNvIHRoYXQgaGlnaGx5LWV4cHJlc3NlZCBnZW5lcyBkb24ndCBkb21pbmF0ZSB0aGUgcmVzdWx0cy4KCldlJ2xsIHVzZSB0aGUgIlNjYWxlRGF0YSIgZnVuY3Rpb24gaW4gU2V1cmF0IHRvIGFwcGx5IHNjYWxpbmcgdG8gb3VyIGRhdGFzZXQuIEhlcmUncyB0aGUgY29kZSB3ZSdsbCB1c2U6CgpgYGB7cn0KYWxsLmdlbmVzIDwtIHJvd25hbWVzKHBibWMpCnBibWMgPC0gU2NhbGVEYXRhKHBibWMsIGZlYXR1cmVzID0gYWxsLmdlbmVzLCBkby5jZW50ZXIgPSBUUlVFKQpgYGAKSW4gdGhpcyBjb2RlLCB3ZSdyZSB1c2luZyB0aGUgIlNjYWxlRGF0YSIgZnVuY3Rpb24gdG8gc2hpZnQgdGhlIGV4cHJlc3Npb24gb2YgZWFjaCBnZW5lIHNvIHRoYXQgdGhlIG1lYW4gZXhwcmVzc2lvbiBhY3Jvc3MgY2VsbHMgaXMgMCwgYW5kIHNjYWxlIHRoZSBleHByZXNzaW9uIG9mIGVhY2ggZ2VuZSBzbyB0aGF0IHRoZSB2YXJpYW5jZSBhY3Jvc3MgY2VsbHMgaXMgMS4gVGhpcyBzdGVwIGlzIGltcG9ydGFudCBiZWNhdXNlIGl0IGhlbHBzIHVzIGNvbXBhcmUgZ2VuZSBleHByZXNzaW9uIGxldmVscyBiZXR3ZWVuIGNlbGxzIHRoYXQgbWlnaHQgaGF2ZSBkaWZmZXJlbnQgdG90YWwgYW1vdW50cyBvZiBSTkEgYW5kIHNvIHRoYXQgaGlnaGx5LWV4cHJlc3NlZCBnZW5lcyBkbyBub3QgZG9taW5hdGUuCgpOb3cgdGhhdCB3ZSd2ZSBzY2FsZWQgb3VyIGRhdGEsIHdlIGNhbiBtb3ZlIG9uIHRvIGRpbWVuc2lvbmFsIHJlZHVjdGlvbiB0ZWNobmlxdWVzIGxpa2UgUENBLiBLZWVwIHVwIHRoZSBncmVhdCB3b3JrLCBhbmQgcmVtZW1iZXIgdGhhdCBpbiBvdXIgZmVsbG93c2hpcCwgZXZlcnlvbmUncyB2b2ljZSBpcyBpbXBvcnRhbnQhCgoqKlBlcmZvcm0gbGluZWFyIGRpbWVuc2lvbmFsIHJlZHVjdGlvbioqCgpQQ0EgaGVscHMgdXMgaWRlbnRpZnkgcGF0dGVybnMgaW4gb3VyIGRhdGEgYnkgcmVkdWNpbmcgdGhlIG51bWJlciBvZiBmZWF0dXJlcyBpbiBvdXIgZGF0YXNldC4gVGhpcyBtYWtlcyBpdCBlYXNpZXIgdG8gdmlzdWFsaXplIGFuZCBhbmFseXplIG91ciBkYXRhLiBBZnRlciBydW5uaW5nIFBDQSwgd2UgY2FuIG1vdmUgb24gdG8gY2x1c3RlcmluZyBvdXIgY2VsbHMgYW5kIGlkZW50aWZ5aW5nIGRpZmZlcmVudCBjZWxsIHR5cGVzIGJhc2VkIG9uIGdlbmUgZXhwcmVzc2lvbiBwYXR0ZXJucy4gS2VlcCB1cCB0aGUgZ3JlYXQgd29yaywgYW5kIHJlbWVtYmVyLCB3aXRoIFBDQSwgb25lIGRvZXMgbm90IHNpbXBseSBhbmFseXplIGFsbCB0aGUgZmVhdHVyZXMgaW4gYSBkYXRhc2V0IQoKYGBge3J9CnBibWMgPC0gUnVuUENBKHBibWMsIGZlYXR1cmVzID0gVmFyaWFibGVGZWF0dXJlcyhvYmplY3QgPSBwYm1jKSkKYGBgCgpgYGB7cn0KIyBFeGFtaW5lIGFuZCB2aXN1YWxpemUgUENBIHJlc3VsdHMgYSBmZXcgZGlmZmVyZW50IHdheXMKcHJpbnQocGJtY1tbInBjYSJdXSwgZGltcyA9IDE6NSwgbmZlYXR1cmVzID0gNSkKYGBgCgoKVGhlIFZpekRpbUxvYWRpbmdzIGZ1bmN0aW9uIGhlbHBzIHlvdSB1bmRlcnN0YW5kIHdoaWNoIGdlbmVzIGNvbnRyaWJ1dGUgdGhlIG1vc3QgdG8gZWFjaCBkaW1lbnNpb24uIEl0IGNyZWF0ZXMgYSBwbG90IHdoZXJlIHRoZSBnZW5lcyBhcmUgcmFua2VkIGJhc2VkIG9uIHRoZWlyIGNvbnRyaWJ1dGlvbiB0byBhIHBhcnRpY3VsYXIgZGltZW5zaW9uLCBhbmQgdGhlaXIgbG9hZGluZ3MgYXJlIHNob3duIGFzIGJhciBwbG90cy4KCmBgYHtyfQojVmlzdWFsaXplIHRvcCBnZW5lcyBhc3NvY2lhdGVkIHdpdGggcmVkdWN0aW9uIGNvbXBvbmVudHMKVml6RGltTG9hZGluZ3MocGJtYywgZGltcyA9IDE6MiwgcmVkdWN0aW9uID0gInBjYSIpCmBgYApYLWF4aXM6IFRoZSB4LWF4aXMgcmVwcmVzZW50cyB0aGUgZ2VuZXMuIFRoZSBnZW5lcyBhcmUgdXN1YWxseSBzb3J0ZWQgYmFzZWQgb24gdGhlaXIgY29udHJpYnV0aW9uIHRvIHRoZSBnaXZlbiBkaW1lbnNpb24sIGZyb20gaGlnaGVzdCB0byBsb3dlc3QuCgpZLWF4aXM6IFRoZSB5LWF4aXMgcmVwcmVzZW50cyB0aGUgbG9hZGluZ3Mgb2YgZWFjaCBnZW5lIG9uIHRoZSBnaXZlbiBkaW1lbnNpb24uIFRoZSBsb2FkaW5ncyBpbmRpY2F0ZSB0aGUgZXh0ZW50IHRvIHdoaWNoIGEgZ2VuZSBjb250cmlidXRlcyB0byB0aGUgc3BlY2lmaWMgZGltZW5zaW9uLiBBIGhpZ2hlciBsb2FkaW5nIHZhbHVlIG1lYW5zIHRoZSBnZW5lIGhhcyBhIHN0cm9uZ2VyIGluZmx1ZW5jZSBvbiB0aGF0IGRpbWVuc2lvbi4KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgpUaGUgRGltUGxvdCBmdW5jdGlvbiB0YWtlcyB0aGlzIHJlZHVjZWQtZGltZW5zaW9uYWwgcmVwcmVzZW50YXRpb24gYW5kIGNyZWF0ZXMgYSBzY2F0dGVyIHBsb3Qgd2hlcmUgZWFjaCBwb2ludCByZXByZXNlbnRzIGEgY2VsbCwgYW5kIGl0cyBwb3NpdGlvbiBpcyBkZXRlcm1pbmVkIGJ5IHRoZSBjZWxsJ3MgZW1iZWRkaW5ncyBpbiB0aGUgcmVkdWNlZCBzcGFjZS4KCgpgYGB7cn0KIyBEaW1lbnNpb25hbCByZWR1Y3Rpb24gcGxvdApEaW1QbG90KHBibWMsIHJlZHVjdGlvbiA9ICJwY2EiKQojIFRoZSBheGVzIGluIHRoZSBzY2F0dGVyIHBsb3QgY29ycmVzcG9uZCB0byB0aGUgZGltZW5zaW9ucyAoY29tcG9uZW50cykgb2J0YWluZWQgZnJvbSB0aGUgZGltZW5zaW9uYWxpdHkgcmVkdWN0aW9uIHRlY2huaXF1ZQojIEVhY2ggcG9pbnQgaW4gdGhlIHNjYXR0ZXIgcGxvdCByZXByZXNlbnRzIGEgY2VsbCBmcm9tIHlvdXIgZGF0YXNldApgYGAKVGhlIHBvc2l0aW9uIG9mIHRoZSBwb2ludCBpbiB0aGlzIGRpbXBsb3QgcmVmbGVjdHMgdGhlIGNlbGwncyBlbWJlZGRpbmdzIGluIHRoZSByZWR1Y2VkIHNwYWNlLiBDZWxscyB3aXRoIHNpbWlsYXIgZ2VuZSBleHByZXNzaW9uIHByb2ZpbGVzIHRlbmQgdG8gY2x1c3RlciB0b2dldGhlciBpbiB0aGUgcGxvdC4KCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KRGltSGVhdG1hcCgpIGFsbG93cyBmb3IgZWFzeSBleHBsb3JhdGlvbiBvZiB0aGUgcHJpbWFyeSBzb3VyY2VzIG9mIGhldGVyb2dlbmVpdHkgaW4gYSBkYXRhc2V0LCBhbmQgY2FuIGJlIHVzZWZ1bCB3aGVuIHRyeWluZyB0byBkZWNpZGUgd2hpY2ggUENzIHRvIGluY2x1ZGUgZm9yIGZ1cnRoZXIgZG93bnN0cmVhbSBhbmFseXNlcy4KCkl0IGNhbiBiZSBhIHZhbHVhYmxlIHRvb2wgZm9yIGV4cGxvcmluZyBjb3JyZWxhdGVkIGZlYXR1cmUgc2V0cy4KCmBgYHtyfQojRGltZW5zaW9uYWwgcmVkdWN0aW9uIGhlYXRtYXAKIyBFeHBsb3JlIHRoZSBwcmltYXJ5IHNvdXJjZXMgb2YgdmFyaWF0aW9uIGluIG91ciBkYXRhc2V0IHVzaW5nIERpbUhlYXRtYXAKRGltSGVhdG1hcChwYm1jLCBkaW1zID0gMSwgY2VsbHMgPSA1MDAsIGJhbGFuY2VkID0gVFJVRSkKI1RoZSBEaW1IZWF0bWFwIGZ1bmN0aW9uIHByb2R1Y2VzIGEgaGVhdG1hcCB3aGVyZSB0aGUgcm93cyByZXByZXNlbnQgdGhlIGdlbmVzIGFuZCB0aGUgY29sdW1ucyByZXByZXNlbnQgdGhlIGNlbGxzLiAKYGBgClRoZSAiYmFsYW5jZWQiIGFyZ3VtZW50IGVuc3VyZXMgdGhhdCB0aGUgcGxvdCBoYXMgYW4gZXF1YWwgbnVtYmVyIG9mIGdlbmVzIHdpdGggYm90aCArIGFuZCAtIHNjb3Jlcy4KVGhlIGNvbG9yIGludGVuc2l0eSBvZiBlYWNoIGNlbGwgaW4gdGhlIGhlYXRtYXAgcmVmbGVjdHMgdGhlIGV4cHJlc3Npb24gbGV2ZWwgb2YgdGhlIGNvcnJlc3BvbmRpbmcgZ2VuZSBpbiB0aGF0IGNlbGwgKGRhcmtlciBjb2xvciBmb3IgaGlnaGVyIGV4cHJlc3Npb24pLgoKYGBge3J9CkRpbUhlYXRtYXAocGJtYywgZGltcyA9IDE6OSwgY2VsbHMgPSA1MDAsIGJhbGFuY2VkID0gVFJVRSkKYGBgClRoZSAicG9zaXRpb24iIG9mIGEgZ2VuZSBhbG9uZyBQQ19uIHJlZmVycyB0byB0aGUgbWFnbml0dWRlIG9mIGl0cyBsb2FkaW5nIG9uIHRoaXMgY29tcG9uZW50LiBBIGdlbmUgd2l0aCBhIGhpZ2hlciBwb3NpdGl2ZSBsb2FkaW5nIGhhcyBhIHN0cm9uZyBwb3NpdGl2ZSBhc3NvY2lhdGlvbiB3aXRoIFBDX24sIG1lYW5pbmcgaXRzIGV4cHJlc3Npb24gbGV2ZWxzIHRlbmQgdG8gaW5jcmVhc2UgYWxvbmcgdGhlIGRpcmVjdGlvbiBvZiB0aGlzIGNvbXBvbmVudC4gQ29udmVyc2VseSwgYSBnZW5lIHdpdGggYSBoaWdoZXIgbmVnYXRpdmUgbG9hZGluZyBoYXMgYSBzdHJvbmcgbmVnYXRpdmUgYXNzb2NpYXRpb24gd2l0aCBQQ19uLCBpbmRpY2F0aW5nIGl0cyBleHByZXNzaW9uIGxldmVscyB0ZW5kIHRvIGRlY3JlYXNlIGFsb25nIHRoZSBkaXJlY3Rpb24gb2YgdGhpcyBjb21wb25lbnQuCgpCeSBsb29raW5nIGF0IHRoZSBleHByZXNzaW9uIHBhdHRlcm5zIG9mIG91ciB0b3AgUENzLCB3ZSBjYW4gYWxzbyBzdGFydCB0byBnZXQgYSBzZW5zZSBvZiB0aGUgZGlmZmVyZW50IGNlbGwgdHlwZXMgcHJlc2VudCBpbiBvdXIgZGF0YXNldC4KCioqRGV0ZXJtaW5lIHRoZSDigJhkaW1lbnNpb25hbGl0eeKAmSBvZiB0aGUgZGF0YXNldCoqCgpXaGVuIHdlIGFuYWx5emUgc2NSTkEtc2VxIGRhdGEsIHdlIG9mdGVuIHdhbnQgdG8gaWRlbnRpZnkgZ3JvdXBzIG9mIGNlbGxzIHRoYXQgYmVoYXZlIHNpbWlsYXJseS4gU2V1cmF0IGNhbiBoZWxwIHVzIGRvIHRoaXMgYnkgY2x1c3RlcmluZyBjZWxscyBiYXNlZCBvbiB0aGVpciBzaW1pbGFyaXRpZXMgYW5kIGRpZmZlcmVuY2VzLiBUbyBkbyB0aGlzLCB3ZSBmaXJzdCBuZWVkIHRvIHJlZHVjZSB0aGUgZGltZW5zaW9uYWxpdHkgb2YgdGhlIGRhdGEsIHdoaWNoIHdlIGRvIHVzaW5nIFBDQS4KRWFjaCBQQyBpcyBhIOKAmG1ldGFmZWF0dXJl4oCZIHRoYXQgY29tYmluZXMgaW5mb3JtYXRpb24gYWNyb3NzIGEgY29ycmVsYXRlZCBmZWF0dXJlIHNldC4gVGhlIHRvcCBwcmluY2lwYWwgY29tcG9uZW50cyB0aGVyZWZvcmUgcmVwcmVzZW50IGEgcm9idXN0IGNvbXByZXNzaW9uIG9mIHRoZSBkYXRhc2V0LiAKClN1cHBvc2UgdGhleSB3YW50ZWQgdG8gY2hvb3NlIGhvdyBtYW55IHBlb3BsZSBzaG91bGQgYmUgaW4gdGhlIGZlbGxvd3NoaXAgb2YgdGhlIHJpbmcsIHRoZW9yZXRpY2FsbHkgaXQncyBsaWtlIGNob29zaW5nIHRoZSBtZW1iZXJzIG9mIHRoZSAiZmVsbG93c2hpcCBvZiB0aGUgcmluZyIgYmFzZWQgb24gdGhlaXIgcGVyZm9ybWFuY2UgaW4gYSBzbWFsbGVyIHRhc2suIFdlIHNlbGVjdCB0aGUgcHJpbmNpcGFsIGNvbXBvbmVudHMgdGhhdCBwZXJmb3JtIHRoZSBiZXN0IGluIGRpc3Rpbmd1aXNoaW5nIGJldHdlZW4gY2VsbHMgYW5kIGtlZXAgdGhlbSBmb3IgZnVydGhlciBhbmFseXNpcy4KClRvIGFuc3dlciB0aGlzIHF1ZXN0aW9uLCB3ZSB1c2UgYSB0ZWNobmlxdWUgY2FsbGVkIHRoZSBKYWNrU3RyYXcgcHJvY2VkdXJlLiBXZSByYW5kb21seSBzaHVmZmxlIGEgc21hbGwgcG9ydGlvbiBvZiB0aGUgZGF0YSAodXN1YWxseSAxJSkgYW5kIHJ1biBQQ0Egb24gdGhpcyBzaHVmZmxlZCBkYXRhIG11bHRpcGxlIHRpbWVzLiBCeSBkb2luZyB0aGlzLCB3ZSBjcmVhdGUgYSBudWxsIGRpc3RyaWJ1dGlvbiBvZiBmZWF0dXJlIHNjb3JlcyBhbmQgaWRlbnRpZnkgdGhlIHNpZ25pZmljYW50IHByaW5jaXBhbCBjb21wb25lbnRzIGJhc2VkIG9uIHRoZSBlbnJpY2htZW50IG9mIGxvdyBwLXZhbHVlIGZlYXR1cmVzLgoKYGBge3J9CiMjIE5PVEU6IFRoaXMgcHJvY2VzcyBjYW4gdGFrZSBhIGxvbmcgdGltZSBmb3IgYmlnIGRhdGFzZXRzLCBjb21tZW50IG91dCBmb3IgZXhwZWRpZW5jeS4gTW9yZSBhcHByb3hpbWF0ZSB0ZWNobmlxdWVzIHN1Y2ggYXMgdGhvc2UgaW1wbGVtZW50ZWQgaW4gRWxib3dQbG90KCkgY2FuIGJlIHVzZWQgdG8gcmVkdWNlIGNvbXB1dGF0aW9uIHRpbWUKCnBibWMgPC0gSmFja1N0cmF3KHBibWMsIG51bS5yZXBsaWNhdGUgPSAzMCkgIyBudW0ucmVwbGljYXRlID0gMTAwIHdpbGwgdGFrZSBsb25nZXIKcGJtYyA8LSBTY29yZUphY2tTdHJhdyhwYm1jLCBkaW1zID0gMToyMCkKCmBgYAoKYGBge3J9CkphY2tTdHJhd1Bsb3QocGJtYywgZGltcyA9IDE6MTUpCmBgYAoKClRoaXMgcHJvY2VzcyBjYW4gdGFrZSBhIHdoaWxlIGZvciBsYXJnZSBkYXRhc2V0cywgYnV0IHRoZXJlIGFyZSBmYXN0ZXIgdGVjaG5pcXVlcyBsaWtlIHRoZSBFbGJvd1Bsb3QoKSBmdW5jdGlvbiB0aGF0IGNhbiBiZSB1c2VkIHRvIHJlZHVjZSBjb21wdXRhdGlvbiB0aW1lLiBPbmNlIHdlIGhhdmUgaWRlbnRpZmllZCB0aGUgc2lnbmlmaWNhbnQgcHJpbmNpcGFsIGNvbXBvbmVudHMsIHdlIGNhbiB1c2UgdGhlbSB0byBjbHVzdGVyIGNlbGxzIGJhc2VkIG9uIHRoZWlyIHNpbWlsYXJpdGllcyBhbmQgZGlmZmVyZW5jZXMuCgoKQW4gYWx0ZXJuYXRpdmUgaGV1cmlzdGljIG1ldGhvZCBnZW5lcmF0ZXMgYW4g4oCYRWxib3cgcGxvdOKAmTogYSByYW5raW5nIG9mIHByaW5jaXBsZSBjb21wb25lbnRzIGJhc2VkIG9uIHRoZSBwZXJjZW50YWdlIG9mIHZhcmlhbmNlIGV4cGxhaW5lZCBieSBlYWNoIG9uZS4KCmBgYHtyfQpFbGJvd1Bsb3QocGJtYykKYGBgCkp1c3QgbGlrZSBHYW5kYWxmIGhhZCB0byBjaG9vc2UgdGhlIHJpZ2h0IGhvYmJpdHMgdG8gam9pbiB0aGUgcXVlc3QsIHdlIGhhdmUgdG8gc2VsZWN0IHRoZSByZWxldmFudCBzb3VyY2VzIG9mIGhldGVyb2dlbmVpdHkgaW4gb3VyIGRhdGFzZXQuIAoKRGVjaWRpbmcgb24gdGhlIG51bWJlciBvZiBwcmluY2lwYWwgY29tcG9uZW50cyB0byBrZWVwIGNhbiBiZSBhcyB0cmlja3kgYXMgZGVjaWRpbmcgb24gdGhlIG51bWJlciBvZiBtZW1iZXJzIGZvciBhIGZlbGxvd3NoaXAuCgoxKVRoZSBmaXJzdCBhcHByb2FjaCBpcyBtb3JlIHN1cGVydmlzZWQsIHdoZXJlIHdlIGV4cGxvcmUgdGhlIHByaW5jaXBhbCBjb21wb25lbnRzIHRvIGlkZW50aWZ5IHJlbGV2YW50IHNvdXJjZXMgb2YgaGV0ZXJvZ2VuZWl0eSwgYW5kIGNhbiBiZSB1c2VkIGluIGNvbmp1bmN0aW9uIHdpdGggdG9vbHMgbGlrZSBHU0VBLCBqdXN0IGxpa2UgdXNpbmcgTGVnb2xhcycga2VlbiBleWVzIHRvIGV4cGxvcmUgZWFjaCBwcmluY2lwYWwgY29tcG9uZW50IGFuZCBkZXRlcm1pbmUgaXRzIHJlbGV2YW5jZS4KMikgVGhlIHNlY29uZCBhcHByb2FjaCBpcyBhIHN0YXRpc3RpY2FsIHRlc3QgYmFzZWQgb24gYSByYW5kb20gbnVsbCBtb2RlbCwgYnV0IGl0IG1heSBub3QgcmV0dXJuIGEgY2xlYXIgY3V0b2ZmIGFuZCBpcyB0aW1lLWNvbnN1bWluZyBmb3IgbGFyZ2UgZGF0YXNldHMuIChKYWNrU3RyYXcpClRoaXMgaXMgdmVyeSBzaW1pbGFyIHRvIEZyb2RvJ3Mgam91cm5leSB0byBNb3VudCBEb29tLCByYW5kb20gd2Fsa2luZyBhbmQgd2F5IHRvbyB0aW1lLWNvbnN1bWluZyB3aXRoIG5vIGNsZWFyIGVuZC4gCjMpIEZpbmFsbHksIHRoZSB0aGlyZCBhcHByb2FjaCBpcyBhIGhldXJpc3RpYyAoZWxib3cgcGxvdCkgdGhhdCBpcyBjb21tb25seSB1c2VkIGFuZCBjYW4gYmUgY2FsY3VsYXRlZCBpbnN0YW50bHksIGp1c3QgbGlrZSBob3cgU2Ftd2lzZSB1c2VkIGhpcyBpbnR1aXRpb24gdG8gbWFrZSBxdWljayBkZWNpc2lvbnMuICAKCkluIHRoaXMgc3R1ZHksIGFsbCB0aHJlZSBhcHByb2FjaGVzIHByb2R1Y2VkIHNpbWlsYXIgcmVzdWx0cywgYW5kIHdlIGNvdWxkIGhhdmUgcmVhc29uYWJseSBjaG9zZW4gYSBjdXRvZmYgYmV0d2VlbiBQQyA3LTEyLgoKKipDbHVzdGVyIHRoZSBjZWxscyoqCgpTZXVyYXQgdjMgdXNlcyBhIGdyYXBoLWJhc2VkIGNsdXN0ZXJpbmcgbWV0aG9kIHRvIGdyb3VwIGNlbGxzIHdpdGggc2ltaWxhciBmZWF0dXJlIGV4cHJlc3Npb24gcGF0dGVybnMuIAogIAogIEZpcnN0LCBhIEstbmVhcmVzdCBuZWlnaGJvciAoS05OKSBncmFwaCBpcyBjb25zdHJ1Y3RlZCBiYXNlZCBvbiB0aGUgZXVjbGlkZWFuIGRpc3RhbmNlIGluIFBDQSBzcGFjZSwgYW5kIGVkZ2Ugd2VpZ2h0cyBiZXR3ZWVuIGNlbGxzIGFyZSByZWZpbmVkIHVzaW5nIHNoYXJlZCBvdmVybGFwIGluIHRoZWlyIGxvY2FsIG5laWdoYm9yaG9vZHMuIFRoaXMgc3RlcCBpcyBwZXJmb3JtZWQgdXNpbmcgdGhlIEZpbmROZWlnaGJvcnMoKSBmdW5jdGlvbiwgYW5kIHRha2VzIGFzIGlucHV0IHRoZSBwcmV2aW91c2x5IGRlZmluZWQgZGltZW5zaW9uYWxpdHkgb2YgdGhlIGRhdGFzZXQgKGZpcnN0IDEwIFBDcykuIAogIE5leHQsIG1vZHVsYXJpdHkgb3B0aW1pemF0aW9uIHRlY2huaXF1ZXMgbGlrZSB0aGUgTG91dmFpbiBhbGdvcml0aG0gYXJlIGFwcGxpZWQgdG8gZ3JvdXAgY2VsbHMgdG9nZXRoZXIgd2l0aCB0aGUgZ29hbCBvZiBvcHRpbWl6aW5nIHRoZSBzdGFuZGFyZCBtb2R1bGFyaXR5IGZ1bmN0aW9uLiBUaGlzIGlzIGRvbmUgdXNpbmcgdGhlIEZpbmRDbHVzdGVycygpIGZ1bmN0aW9uLCB3aXRoIGEgcmVzb2x1dGlvbiBwYXJhbWV0ZXIgdGhhdCBzZXRzIHRoZSDigJhncmFudWxhcml0eeKAmSBvZiB0aGUgZG93bnN0cmVhbSBjbHVzdGVyaW5nLiBUaGUgSWRlbnRzKCkgZnVuY3Rpb24gY2FuIGJlIHVzZWQgdG8gZmluZCB0aGUgY2x1c3RlcnMuCgpKdXN0IGxpa2UgRnJvZG8gbmVlZGVkIGEgdHJ1c3RlZCBjb21tdW5pdHkgdG8gY29tcGxldGUgaGlzIGpvdXJuZXksIHdlIG5lZWQgdG8gY2x1c3RlciBjZWxscyBpbnRvIG1lYW5pbmdmdWwgZ3JvdXBzIHRvIGNvbXBsZXRlIGRvd25zdHJlYW0gYW5hbHlzZXMuCgpgYGB7cn0KcGJtYyA8LSBGaW5kTmVpZ2hib3JzKHBibWMsIGRpbXMgPSAxOjgpCnBibWMgPC0gRmluZENsdXN0ZXJzKHBibWMsIHJlc29sdXRpb24gPSAwLjcpCmBgYAoKYGBge3J9CiMgTG9vayBhdCBjbHVzdGVyIElEcyBvZiB0aGUgZmlyc3QgNSBjZWxscwpoZWFkKElkZW50cyhwYm1jKSwgNSkKYGBgCgoKKipSdW4gbm9uLWxpbmVhciBkaW1lbnNpb25hbCByZWR1Y3Rpb24gKFVNQVAvdFNORSkqKgoKSW1hZ2luZSB5b3UgaGF2ZSBhIHZhc3Qga2luZ2RvbSB3aXRoIG1hbnkgY2l0aWVzIGFuZCB0b3ducy4gRWFjaCBjaXR5IHJlcHJlc2VudHMgYSBncm91cCBvZiBjZWxscyB3aXRoIHNpbWlsYXIgY2hhcmFjdGVyaXN0aWNzLCBhbmQgeW91IHdhbnQgdG8gZXhwbG9yZSB0aGVtIG9uIGEgbWFwLiBIb3dldmVyLCB5b3VyIG1hcCBpcyBsaW1pdGVkIHRvIG9ubHkgdHdvIGRpbWVuc2lvbnMsIHdoaWNoIG1lYW5zIHlvdSBuZWVkIGEgd2F5IHRvIGNvbXByZXNzIHRoZSBpbmZvcm1hdGlvbiBhbmQgcGxhY2Ugc2ltaWxhciBjaXRpZXMgdG9nZXRoZXIuCgpUaGF0J3Mgd2hlcmUgdFNORSBhbmQgVU1BUCBhbGdvcml0aG1zIGNvbWUgaW4uIFRoZXkgYXJlIGxpa2UgbWFnaWNhbCBzcGVsbHMgdGhhdCBhbGxvdyB5b3UgdG8gdHJhbnNmb3JtIHRoZSBkYXRhIGludG8gYSBsb3dlci1kaW1lbnNpb25hbCBzcGFjZSwgd2hlcmUgc2ltaWxhciBjZWxscyBhcmUgcGxhY2VkIGNsb3NlciB0b2dldGhlci4gVGhpcyBoZWxwcyB5b3UgdmlzdWFsaXplIGFuZCBleHBsb3JlIHRoZSBkYXRhIG1vcmUgZWFzaWx5LgoKVG8gbWFrZSBzdXJlIHRoZSBjZWxscyBpbiBlYWNoIGdyb3VwIChvciBjbHVzdGVyKSBhcmUgbG9jYXRlZCBuZWFyIGVhY2ggb3RoZXIgb24gdGhlIG1hcCwgd2Ugc3VnZ2VzdCB1c2luZyB0aGUgc2FtZSAibWV0YWZlYXR1cmVzIiAoaS5lLiwgUENzKSB0aGF0IHdlcmUgdXNlZCBmb3IgY2x1c3RlcmluZyBhbmFseXNpcyBhcyBpbnB1dCB0byB0U05FIGFuZCBVTUFQLiBUaGlzIHdpbGwgaGVscCB5b3Ugc2VlIHRoZSBkaWZmZXJlbmNlcyBhbmQgc2ltaWxhcml0aWVzIGJldHdlZW4gdGhlIGNsdXN0ZXJzIG1vcmUgY2xlYXJseS4KCmBgYHtyfQojIElmIHlvdSBoYXZlbid0IGluc3RhbGxlZCBVTUFQLCB5b3UgY2FuIGRvIHNvIHZpYSByZXRpY3VsYXRlOjpweV9pbnN0YWxsKHBhY2thZ2VzID0gJ3VtYXAtbGVhcm4nKQpwYm1jIDwtIFJ1blVNQVAocGJtYywgZGltcyA9IDE6MTApCmBgYAoKYGBge3J9CiMgbm90ZSB0aGF0IHlvdSBjYW4gc2V0IGBsYWJlbCA9IFRSVUVgIG9yIHVzZSB0aGUgTGFiZWxDbHVzdGVycyBmdW5jdGlvbiB0byBoZWxwIGxhYmVsICMgaW5kaXZpZHVhbCBjbHVzdGVycwpEaW1QbG90KHBibWMsIHJlZHVjdGlvbiA9ICJ1bWFwIiwgbGFiZWwgPSBUUlVFKQpgYGAKCioqRmluZGluZyBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZmVhdHVyZXMgKGNsdXN0ZXIgYmlvbWFya2VycykqKgoKSWYgU2V1cmF0IHdlcmUgYSB3aXphcmQsIGZpbmRpbmcgbWFya2VycyB0aGF0IGRlZmluZSBjbHVzdGVycyB2aWEgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gd291bGQgYmUgaXRzIHNpZ25hdHVyZSBzcGVsbCAtICJFeHBlY3RvIE1hcmtlcnVtISIgQnkgZGVmYXVsdCwgU2V1cmF0IGlkZW50aWZpZXMgcG9zaXRpdmUgYW5kIG5lZ2F0aXZlIG1hcmtlcnMgb2YgYSBzaW5nbGUgY2x1c3RlciAoc3BlY2lmaWVkIGluIGlkZW50LjEpLCBjb21wYXJlZCB0byBhbGwgb3RoZXIgY2VsbHMuIEJ1dCBpZiB5b3Ugd2FudCB0byBnbyBvbiBhIG1vcmUgYWR2ZW50dXJvdXMgcXVlc3QsIEZpbmRBbGxNYXJrZXJzKCkgYXV0b21hdGVzIHRoaXMgcHJvY2VzcyBmb3IgYWxsIGNsdXN0ZXJzLgoKYGBge3J9CiMgZmluZCBhbGwgbWFya2VycyBvZiBjbHVzdGVyIDIKY2x1c3RlcjIubWFya2VycyA8LSBGaW5kTWFya2VycyhwYm1jLCBpZGVudC4xID0gMiwgbWluLnBjdCA9IDAuMjUpCmhlYWQoY2x1c3RlcjIubWFya2VycywgbiA9IDUpCmBgYApIb3dldmVyLCBiZXdhcmUgb2YgdGhlIG1pbi5wY3QgYXJndW1lbnQsIHdoaWNoIHJlcXVpcmVzIGEgZmVhdHVyZSB0byBiZSBkZXRlY3RlZCBhdCBhIG1pbmltdW0gcGVyY2VudGFnZSBpbiBlaXRoZXIgb2YgdGhlIHR3byBncm91cHMgb2YgY2VsbHMuIEl0J3MgbGlrZSByZXF1aXJpbmcgYSBob2JiaXQgdG8gaGF2ZSBhdCBsZWFzdCA1MCUgaGFpcnkgZmVldCBiZWZvcmUgbGV0dGluZyB0aGVtIGpvaW4gdGhlIGZlbGxvd3NoaXAuIEFuZCBkb24ndCBmb3JnZXQgdGhlIHRocmVzaC50ZXN0IGFyZ3VtZW50LCB3aGljaCByZXF1aXJlcyBhIGZlYXR1cmUgdG8gYmUgZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIChvbiBhdmVyYWdlKSBieSBzb21lIGFtb3VudCBiZXR3ZWVuIHRoZSB0d28gZ3JvdXBzLiBJdCdzIGxpa2UgcmVxdWlyaW5nIExlZ29sYXMgdG8gYmUgYXQgbGVhc3QgMSBmb290IHRhbGxlciB0aGFuIEZyb2RvIGJlZm9yZSBoZSBjYW4gam9pbiB0aGUgZmVsbG93c2hpcC4KCklmIHlvdSdyZSBmZWVsaW5nIGJyYXZlLCB5b3UgY2FuIHNldCBib3RoIG9mIHRoZXNlIGFyZ3VtZW50cyB0byAwLCBidXQgYmUgcHJlcGFyZWQgZm9yIGEgbG9uZyBhbmQgdHJlYWNoZXJvdXMgam91cm5leSB0aHJvdWdoIGEgbGFyZ2UgbnVtYmVyIG9mIGZlYXR1cmVzIHRoYXQgYXJlIHVubGlrZWx5IHRvIGJlIGhpZ2hseSBkaXNjcmltaW5hdG9yeS4gQWx0ZXJuYXRpdmVseSwgeW91IGNhbiB1c2UgdGhlIG1heC5jZWxscy5wZXIuaWRlbnQgYXJndW1lbnQgdG8gZG93bnNhbXBsZSBlYWNoIGlkZW50aXR5IGNsYXNzIHRvIGhhdmUgbm8gbW9yZSBjZWxscyB0aGFuIHdoYXRldmVyIHRoaXMgaXMgc2V0IHRvLiBJdCdzIGxpa2UgYXNraW5nIEdhbmRhbGYgdG8gdXNlIGhpcyBtYWdpYyB0byBzaHJpbmsgdGhlIGZlbGxvd3NoaXAgZG93biB0byBhIG1vcmUgbWFuYWdlYWJsZSBzaXplLiBZZXMsIHRoZXJlIG1heSBiZSBhIGxvc3MgaW4gcG93ZXIsIGJ1dCB0aGUgc3BlZWQgaW5jcmVhc2VzIGNhbiBiZSBzaWduaWZpY2FudCwgYW5kIHRoZSBtb3N0IGhpZ2hseSBkaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZmVhdHVyZXMgd2lsbCBsaWtlbHkgc3RpbGwgcmlzZSB0byB0aGUgdG9wLgoKYGBge3J9CiMgZmluZCBhbGwgbWFya2VycyBkaXN0aW5ndWlzaGluZyBjbHVzdGVyIDUgZnJvbSBjbHVzdGVycyAwIGFuZCAzCmNsdXN0ZXI1Lm1hcmtlcnMgPC0gRmluZE1hcmtlcnMocGJtYywgaWRlbnQuMSA9IDUsIGlkZW50LjIgPSBjKDAsIDMpLCBtaW4ucGN0ID0gMC4yNSkKaGVhZChjbHVzdGVyNS5tYXJrZXJzLCBuID0gNSkKCmBgYAoKYGBge3J9CiMgZmluZCBtYXJrZXJzIGZvciBldmVyeSBjbHVzdGVyIGNvbXBhcmVkIHRvIGFsbCByZW1haW5pbmcgY2VsbHMsIHJlcG9ydCBvbmx5IHRoZSBwb3NpdGl2ZSBvbmVzCnBibWMubWFya2VycyA8LSBGaW5kQWxsTWFya2VycyhwYm1jLCBvbmx5LnBvcyA9IFRSVUUsIG1pbi5wY3QgPSAwLjI1LCBsb2dmYy50aHJlc2hvbGQgPSAwLjI1KQpwYm1jLm1hcmtlcnMgJT4lCiAgICBncm91cF9ieShjbHVzdGVyKSAlPiUKICAgIHNsaWNlX21heChuID0gMiwgb3JkZXJfYnkgPSBhdmdfbG9nMkZDKQpgYGAKYGBge3J9CmNsdXN0ZXIwLm1hcmtlcnMgPC0gRmluZE1hcmtlcnMocGJtYywgaWRlbnQuMSA9IDAsIGxvZ2ZjLnRocmVzaG9sZCA9IDAuMjUsIHRlc3QudXNlID0gInJvYyIsIG9ubHkucG9zID0gVFJVRSkKYGBgCgpgYGB7cn0KVmxuUGxvdChwYm1jLCBmZWF0dXJlcyA9IGMoIlMxMDBBOCIsICJTMTAwQTkiKSkKCmBgYApgYGB7cn0KIyB5b3UgY2FuIHBsb3QgcmF3IGNvdW50cyBhcyB3ZWxsClZsblBsb3QocGJtYywgZmVhdHVyZXMgPSBjKCJTMTAwQTgiLCAiUzEwMEE5IiksIHNsb3QgPSAiY291bnRzIiwgbG9nID0gVFJVRSkKYGBgCgpgYGB7cn0KRmVhdHVyZVBsb3QocGJtYywgZmVhdHVyZXMgPSBjKCJNUzRBMSIsICJHTkxZIiwgIkNEM0UiLCAiQ0QxNCIsICJGQ0VSMUEiLCAiRkNHUjNBIiwgIkxZWiIsICJQUEJQIiwgIkNEOEEiKSkKYGBgCmBgYHtyfQpGZWF0dXJlUGxvdChwYm1jLCBmZWF0dXJlcyA9IGMoIk1TNEExIiwgIkdOTFkiKSkKYGBgCgpgYGB7cn0KcGJtYy5tYXJrZXJzICU+JQogICAgZ3JvdXBfYnkoY2x1c3RlcikgJT4lCiAgICB0b3BfbihuID0gMTAsIHd0ID0gYXZnX2xvZzJGQykgLT4gdG9wMTAKRG9IZWF0bWFwKHBibWMsIGZlYXR1cmVzID0gdG9wMTAkZ2VuZSkgKyBOb0xlZ2VuZCgpCmBgYAo=