k-Means Clustering Analysis on Washington CANS Data

Overview

This file is a summary of the k-means clustering analysis on Washington CANS.

• Motivation. While the 50 core items of CANS and 4-point potential response for each item is able to describe a huge range of diversity of youth disposition at intake to systems of care, it is likely that a smaller, more tractable set of essential patterns of presentation would emerge. Identifying and describing this distilled set of essential patterns of need can be a powerful guide for human interpretation — and thus actionability — of CANS data for system actors and clinicians.

• Applied Uses. A distillation of youth characteristics would allow a range of uses including:
  • The ability to classify youth at intake could help with
    • the referral and/or placement decision, such as matching clients to clinicians and/or agencies with particular expertise, based on judgment of facilities/experience and data-based characterization of client disposition.
    • promoting professional development of clinicians, by more easily identifying staff who have similar client caseloads (even/especially across agencies) and could productively compare practices.
  • The ability to describe trends in certain types of client clusters can help identify overall strategy in developing training, program support or design, and broader referral mechanisms.
  • The overall associations between clusters of clinical need and demographic or geographic factors can promote broader social discussions about the origin or nature of how harms initially occur that could be addressed by human service systems even before individuals enter systems of care that we work with.

k-Means Clustering Analysis

Statistical clustering methods are an “unsupervised learning” method for identifying the distilled set of essential patterns of need. In particular, in our analysis we chose the k-means clustering method.

Why k-means clustering?

• The k-means clustering method is probably the most widely-used clustering method, which aims to partition observations into \(k\) clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster.
• The method could perfectly treat the four-point scale for CANS items as continuous to identify combinations of responses based on intensity of need.
• The method is rather easy and efficient to implement and apply even on large data sets, which has been successfully used in various topics. It often serves as a preprocessing step for other algorithms.
• One can apply the 1-nearest neighbor classifier on the cluster centers obtained by k-means to classify new data into the existing clusters. This allows for further classification of the new clients and facilitate the design of training.

Sample set

The persons included in the analysis are the clients above 5 years old from the Washington CANS 11/06/2017 spreadsheet. Furthermore, we take the records of CANS screening assessment from this cohort.

• Total number of clients included: 6,429/6,905 (included/total)
• Age characteristic: all above 5 years old
• Gender characteristic: 3,696 males, 2,704 females, 29 Unknown or Unspecified
• Length of stay in treatment: average length of stay is 245.9 days (7.9 - 8.2 months)
• Number of days from referral to screening on average: 7.7 days
• Individuals excluded from WISe services based on the screener: 2,397/8,266 within all screening records
• Distribution of the sample set across regions:

Statistical Charts and Diagrams

Clients included

Gender

Length of stay

Sample distribution

Domains and Items

In our clustering analyses we include all items in Emotional Needs (BH) and Risk (RF), and then exclude those with missing percentage larger than 95% at intake. Specifically the resulting 14 items included in the analyses are as follows.

Implementation

We implement the k-means clustering with the aforementioned sample set based on the resulting 14 CANS items, using the \(\mathtt{kmeans}\) function in the \(\mathtt{stat}\) package. Notice that the number of clusters \(k\) is user-specified. Although there exists statistical methodology to choose the “best” \(k\), policy fellows should always be involved to provide the clinical rationale in this process (i.e, decide the \(k\) so that the resulting clusters are most clinically meaningful).

Clustering Results

In our analyses we loop the code across the number of clusters \(k\) from 1 to 8. Here we show and describe the clustering results with \(k=3\). The scree plot is shown below to legitimate the choice of \(k\) – we look at the percentage of total within-cluster sum of squares in the total sum of squares.

First we look at the size of each cluster.

Relative Size (pie chart)

Relative Size (bar chart)

Second, we show the comparisons of characteristics of each cluster with \(k=3\), where the vertical axis is the corresponding component for an item of the cluster center.

Observations

• First we provide a description of the clusters which emerged.
  • Cluster 1 is the “internal cluster”, which has high scores in “Anxiety”, “Mood”, “Trauma” and “Suicide” and low score in “Danger”.
  • Cluster 2 is the “mixing cluster”, which has low scores in “Mood”, “Non-suicidal” and “Suicide”.
  • Cluster 3 is the “external cluster”, which has high scores in “Anxiety”, “Attention”, “Disruptive”, “Emotional Control”, “Danger” and “Decision”.
• Second notice that some items included in the initial analyses are not retained in any of the three clusters. In particular, all three clusters have low scores in “Psychosis”, “Substance Abuse”, “Medication Management” and “Runaway”, therefore none of these items form the core of any cluster.

Further Analyses with the Clustering Results

With the clusters identified we can conduct several follow-up analyses. The results would facilitate the training development, placement decision and program design by providing more targeted and insightful description of down-the-road metrics for each cluster.

Scheme 1: Unconditional Analysis of Changes in Average Number of Actionable Items

To measure the outcomes of current children’s services, one natural metric is to look at the improvement in average number of actionable items along the whole treatment period. We would also want this measurement to be conditioned on the length of time in treatment. The result of the unconditional analysis without considering the clusters is displayed below. The sample size of each cohort is attached to the final point of the corresponding trajectory.

Observations

• The trajectories show that all the 5 cohorts included in the analysis benefit from the current treatment, since the average number of actionable items decreases during the entire treatment period.
• Furthermore we can conclude that in general the improvement speeds up along the treatment period by looking closer at the slopes.
• Also by observing the start points of the trajectories we see that the clients with higher initial number of actionable items stay longer in the treatment, a conclusion complying with our intuition.

However the unconditional analysis has a confounding factor of cluster at intake. Therefore an extension of this analysis is to look at the metric for each resulting cluster, which leads to the second scheme as described below.

Scheme 2: Changes in Average Number of Actionable Items for Each Cluster

Scheme 2 extends the unconditional analysis by further conditioning on the cluster, i.e, we extract all the clients within a certain cluster before conducting scheme 1. The sample size of each cohort is attached to the final point of the corresponding trajectory.

Cluster 1

Cluster 2

Cluster 3

Observations

The difference among the trajectories for each cluster is significant. In particular,

• for Cluster 1 (the “internal cluster”),
  • the treatment does not work well for the 9-month cohort within the first 3 months — the average number of actionable items increases by around 0.5 instead of decreasing.
  • there is also a platform between 3 to 6 months for the 12-month cohort.
  • clients with more actionable items at intake do not necessarily stay longer in the treatment — the 9-month cohort has lower start point than the 3- and 6-month cohorts.
• for Cluster 2 (the “mixing cluster”),
  • the decreasing trends of the trajectories are preserved, i.e, for the “mixing cluster” the current treatment reduces the average number of actionable items along the whole treatment period.
  • the 9-month cohort has slightly higher start point than the 12-month cohort.
• for Cluster 3 (the “external cluster”),
  • 9-month cohort starts with a moderate average number of actionable items, however barely benefits from the current treatment within the first 3 months.

Also if we compare among the clusters,

• we see that Cluster 2 is of the smallest start average number of actionable items (17.97 - 19.37), followed by Cluster 1 (18.10 - 19.66) and then Cluster 3 (19.06 - 20.58).
• however interestingly, we also observe that the 12-month cohort in Cluster 3 benefits the most from the current treatment, with an improvement from 20.5 actionable items on average at the beginning of treatment to around 12.5 at the end (8-point improvement) compared to 6- and 5.5-point improvements for Cluster 1 and Cluster 2 respectively.

Scheme 3: Changes in Average Number of Actionable Items for Each Cohort

Similarly, we would also like to know how the clients within the same cohort benefit differently based on the cluster they are in. Therefore we look at each cohort but further divide the sample sets by the resulting clusters.

Initial

3 Months

6 Months

9 Months

12 Months

Observations

• Except for the 9-month cohort, the clients within Cluster 3 always start from the largest number of actionable items and the clients within Cluster 2 always start from the smallest.
• For the 9-month cohort those within Cluster 1 do not get improved with the treatment in the first 3 months; same for those within Cluster 1 in the 12-month cohort between 3 to 6 months treatment period. Therefore the agencies might want to emphasize more on the program design for those within Cluster 1 in the first 6 months of treatment.

Scheme 4: The Probability of Still Being in the Treatment for Each Cluster

Another metric of the treatment performance is the probability of an individual still being in the treatment at time \(t\). This probability is estimated by dividing the number of clients who haven’t exited care at time \(t\) with the total number of clients who entered the treatment. We further enstimate this metric conditioned on clusters to determine whether there exists between-cluster difference. The resulting trajectories are shown below.

Surprsingly, the difference between the trajectories is negligible – the individuals should expect a same discharge rate during the entire treatment period, no matter which cluster they are in.

Note

• The criteria used to determine whether or not an individual exited care is: we look at all the full assessment records for a certain client, work out the longest length of stay in treatment T, and use this number as a threshold: we say a client is still in the treatment if the treatment period is shorter than T or otherwise he/she exited care.
• This criteria mignt not handle properly the censoring of observation, i.e, the case that an individual hasn’t yet exited care for the duration of our survey or an individual dropped out the treatment before being discharged. An advanced area of study, the Survival Analysis, can be further incorporated to effectively deal with those censored cases.

Scheme 5: Weighted Monthly Change (WMC) for Each Region

The last criteria we discuss in our analyses is the weighted monthly change in the number of actionable items. This single number can summarize the monthly changes in the number of actionable items for all cohorts with different lengths of stay in care. Specifically, first the weighted monthly change within each cohort is calculated by \[\text{WMC}_i= \frac{\text{Average start # actionable items - Average end # actionable items}}{\text{length of stay in care}}\]for the \(i^{th}\) cohort and further compute the arithmetic mean of the \(\text{WMC}_i\) or weight the \(\text{WMC}_i\) by the population size of the cohort. We calculate the unweighted WMC (i.e, taking the simple arithmetic mean first) for each region conditional/unconditional on the clusters and show the results in the Choropleths graphs below.

Cluster 1

Cluster 2

Cluster 3

Unconditional

Observations

• First notice that there are so few observations from the North Central Washington BHO in each cluster that no statistically powerful analysis can be conducted for this region. The region is shown in grey and marked as “NA”.
• For the other regions the difference between WMC within a certain cluster is significant. In particular,
  • with regard to Cluster 1, the Salish BHO, Greater Columbia BHO and Spokane County Regional BHO have strength in treating the clients and the King County BHO is performing relatively weaker.
  • with regard to Cluster 2, the Greater Columbia BHO, Southwest Washington BHO and Thurston-Mason BHO have strong treatment performance and the King County BHO has relatively weaker performance.
  • with regard to Cluster 3, the Spokane County Regional BHO is doing particularly good and the King County BHO has relatively weaker performance.
• The difference of WMC among clusters is also significant. For example, the Salish BHO treats best the clients in Cluster 1 while the Great River BHO has best care performance for Cluster 3. This information would serve as a guideline for making better referral and/or placement decision.

Diversity of Prevalence of Each Cluster

We show the differences in the proportions of clients that fall into each cluster for each region.

Conclusion

• The k-means clustering analysis only requires the clients’ information at intake to system of care, rendering itself an ideal tool to facilitate further treatment design and placement decision from the very beginning of the treatment period.
• The clustering results are powerful if combined with down-the-road metrics. In particular in our analyses,
  • Scheme 2 and 3 serve as guides for improving current treatment for system actors and clinicians and
  • Scheme 5 can potentially facilitate the referral and/or placement decision process.