Hi Reader, greetings!

In this notebook, I’m using library dplyr, ggplot2 and function plot() to demonstrate the plotting of pinch force along time when a healthy human subject was turning a force-embedded key.

Setting the stage: why force sensor for pinch?

  • The goal is to examine the sensor-aided approach to capture human hand dexterity, hence, to shed the light of sensor’s potential in measuring impairment before (baseline) and after going through rehabilitation therapy, pathing the way to evidence-based precision rehabilitation.

Is pinch force during the task of turning key useful?

  • The selected task is one of the functional tasks extracted from Wolf Motor Function Test-Item 15 to assess stroke survivor’s pinch force control/scaling ability.

  • This task requires participant to 1) first grip the key 2) perform “turn and return” and 3) release the key as fast as possible. Participant has to repeat it for 6 times.

  • The ability of turning key can reflect fine motor skill (human hand dexterity).

  • The data is collected from a healthy subject, obtained from a publicly available source (data source). In this section, we aim to understand trend of turning a key in a normal way (normative data).

Something technical - Automating the data cleaning and segmentation

The force sensor data is collected at 200Hz frequency, and post-processed with low pass Butterworth filter at 6Hz for noise cleaning (i.e. high frequency noises > 6hz are filtered out). After which, data are systematically segmented according to a customized algorithm.

1. Read file

Now, lets read the file from one human subject doing key turning.

# read file
comdf <- read.table(file=paste0("./processed_files/", filename),
                    header=TRUE, fill = TRUE)

#extract subject name
subject <- stringi::stri_sub(filename, 1, 5)
subject <- "SNXXX " #censor subject name

#extract handedness
hand <- stringi::stri_sub(filename, 6, 7)
hand <- "Right Hand"

2. Data wrangling

Here, the column P.V. (peak and valley location) and phases (turn key, return key & release key) are converted from character to factor(i.e. categorical attribute), to facilitate ease of data manipulation.

# Transform Data
library(dplyr)
comdf <- comdf %>%
    select(c("Time", "theta_lowpass", 
                   "phase", "wrist.deg_lowpass", 
                   "Key_Pinch_kg_lowpass",
                   "firstder.Pinch_kg_lowpass", 
                   "secder.Pinch_kg_lowpass", "P.V", "maxstaticpinch")) %>%
    mutate_at(vars(P.V, phase), list(factor))

# checking
glimpse(comdf)
## Rows: 7,441
## Columns: 9
## $ Time                      <dbl> 0.905, 0.910, 0.915, 0.920, 0.925, 0.930, 0.~
## $ theta_lowpass             <dbl> 0.03548162, 0.04071614, 0.04591240, 0.050908~
## $ phase                     <fct> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,~
## $ wrist.deg_lowpass         <dbl> 4.710036, 5.195445, 5.633970, 6.013274, 6.32~
## $ Key_Pinch_kg_lowpass      <dbl> 0.1414459, 0.1600956, 0.1781672, 0.1952391, ~
## $ firstder.Pinch_kg_lowpass <dbl> 0.000000000, 3.694167969, 3.536273440, 3.305~
## $ secder.Pinch_kg_lowpass   <dbl> 0.00000000, -21.61862451, -38.88305342, -51.~
## $ P.V                       <fct> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,~
## $ maxstaticpinch            <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,~

P.V. and phases are now converted to factor .

Below with the explanation of each variables: - Time: Measuring time (in seconds) - theta_lowpass: Angle of key while turning (in degree) - phase: - wrist.deg_lowpass: - Key_Pinch_kg_lowpass: Force - firstder.Pinch_kg_lowpass: - secder.Pinch_kg_lowpass - P.V: - maxstaticpinch:

3. Plots using plot() function

Let’s now look at the plots. Plot_1 shows the filtered pinch force while turning a key (embedded with force sensor) repeated 6 times.

The vertical dotted lines used to segment the signal are based on turning angle (not shown in the graph).

We get to see the trend that higher amount of force occurred at the time of returning key (light seagreen color).

Highlight: This process of segementation and visualisation is being automated and repeated for 288 profiles (process done in R = 3mins vs manual process done in excel: 2 weeks)

## [1] "SNXXX Right Hand Plot_1: Key Turning Pinch Force v.s Time"

4.Ensemble plot using ggplot function

In Plot_2, pinch force profile per each of 6 trials is extracted against normalized time. It gives us an idea how a pinch force is varying within each phase cycle per single participant.

The shape is generally consistent.

## [1] "SNXXX Right Hand Plot_2: Pinch Force v.s. Time"

Pinch force rate reflect the pinch force increase within unit time, namely “scaling”.

The shape of each trial is consistent and the fastest scaling rate is found at 15kg/s in positive (adding force) and negative (releasing force).

## [1] "SNXXX Right Hand Plot_3: Pinch Force Rate v.s. Time"

The pinch force is plotted against key turning angle. When key is turned forward (zero to 180 degree), the force increases non-linearly. It interestingly reflects how human hand gently control the force in a non-linear way which can be very different to traditional robot hand (linear open and close).

An insight is that robot hand can now mimic such similar curve to achieve smooth hand pinch motion!

## [1] "SNXXX Right Hand Pinch Force v.s. Key_angle"

5. Essemble plot of 288 pinch profiles using ggplot function

Here is a chunk of data wrangling code and I’m attempting to apply Fisher-EM algorithm to cluster high dimensional data (in this case, 288 layer of time series) into two clusters mean.

The plot visualises trend of normative pinch variation for key turning and returning only (excluding the release phase). The trajectory is a dual peak shape.

#plotting all trials

filename <- "_trialdata_pinch.tsv"
trialdata_pinch <- read.table(file=paste0("./processed_files/",
                                          filename), header=TRUE, fill = TRUE)

filename <- "newtrialdata_mean_pinch.tsv"
newtrialdata_mean_pinch <- read.table(file=paste0("./processed_files/", filename),
                    header=TRUE, fill = TRUE)

long_transpose_pinch <- reshape2::melt(trialdata_pinch, id.vars = "X")
colnames(long_transpose_pinch) <- c("time", "subject", "value")
View(long_transpose_pinch)



# Fisher EM clustering
newtrialdata_mean_pinch <- newtrialdata_mean_pinch[,2:102]
res = FisherEM::fem(newtrialdata_mean_pinch,K=2,model='AB',method='reg')
#plot(res)
fem_mean <- t(res$mean)    #get from following fem
long_fem_mean <- reshape2::melt(fem_mean)
colnames(long_fem_mean) <- c("time", "cluster", "value")



#---------------
# 288 profiles ensembles
#---------------

library(ggplot2)
library("ggthemes")  
ggbase <- ggplot(long_transpose_pinch,                            # Draw ggplot2 time series plot
                 aes(x = time,
                     y = value,
                     col = subject)) +
    geom_line() + 
    theme(legend.position = "none") 

gg1 <- ggbase+ labs(x = "Normalised Time(%)", y = "Pinch Force (kg)") +
    ggtitle("Absolute Pinch Force vs Normarlised Time") +
    theme_hc() + theme(legend.position = "none") +
    geom_point(data=long_fem_mean, aes(x = time, y = value), 
               pch = 13, size = 3.0, colour ="black", alpha = 1)
gg1

6. Wrap Up

[Update Jan 2023] Good news! The presented work eventually leads to publication in a Biomechanics conference. I’m grateful for this opportunity to promote the potential of sensor data in tracking hand dexterity recovery in rehabilitation field. Below with a sneak preview.

Sneak Preview

In this notebook, we get to see how normative key turning pinch profile is like. The next step is to compare it with stroke survivor pinch profile and to discover useful metric able to distinguish the healthy and post-stroke condition.

7. Something about statistics

To identify useful metric, we can preliminary employ Two Sample t-test (parametric test for normal distribution) or Wilcoxon-Mann-Whitney test (non-parametric without underlying distribution) to prove our hypothesis. Taking t-test as example:

  • let’s say mean time of key turning is considered as indicator, then our hypothesis will be:
    • H0: mean time in healthy population equals mean time in stroke survivor population: t0=t1
    • H1: mean time in healthy population does not equal mean time in stroke survivor population: t0≠t1

We can use t.test() or wilcox.test() in R to run hypothesis test. (note: wilcox has different hypothesis statement)

If the p-value is <0.05, we have sufficient evidence to reject null H0 and conclude mean time is significantly different between two groups. That gives an insight that such sensor measurement can be used to measure human hand dexterity performance. It will encourage sensor-based approach to monitor patient’s hand rehabilitation journey.