Process Mining - Relation to Robotic Process Automation

Robotic Process Automation

Robotic Process Automation (RPA) is the discipline of automating basic, high volume and potentially (though not necessarily) low value-add activities. The agent executing said activities are referred to as ‘bots’ or ‘robots’. However, a key pre requisite before process automation is a deepr understanding of the ‘as-is’ process. This is where Process Mining comes in.

The process (pun intended) for process automation could be described as below (referred from QPR website) * Understand the As-Is process. * Highlight the best candidates for automation * Locate deviations before automation * Use process maps to guide bots * Measure process efficiency at implementation

In all of these steps, process mining plays an important role. So, let us start with understanding what is it all about.

What is Process Mining?

Process mining is the analytical discipline of understanding real processes (vis a vis assumed processes) from event logs, thereby enabling comparison of the extracted process with ideal process. It can provide detailed objective & data driven view of how the processes are performing and answer both performance and compliance related questions.

Process mining can be used as a precursor to operational efficiency enhancement programs, resource allocation and process automation guidelines. It is closely related to business process management.

This notebook is intended for getting an introduction to process mining, understanding its basics and provide basic ground work for starting process mining.

Process management - key concerns

• A top concern is the creation of “current state” processes itself. Most process exercises in large corporations focus on the “to be” processes, which implies how something should be done, rather than how it is being done.

• Moreover, understanding the “as is” is justifiably skeptical for the executive given the sticky-note interview-juggling it is built upon. This provides a subjective view of the process. Although there has traditionally been no subsitute to first hand experience, with the data collection robustness in modern times there might be a better option.

• A lack of two-way connection between business processes and enterprise information systems. While ERP systems do capture process logs, they rarely provide interfaces to work with these logs. In many cases, the data collection and logging systems are not robust enough, or distributed. For instance, support cases being resolved and logged over email would never be amenable to any data mining approach.

The discipline that studies these concerns and allows actions which form the pre requisite to start the journey of RPA is essentially process mining.

Ecosystem activity in Process Mining - a brief literature survey

In 2011, a munich based company called Celonis was founded with Van Der Alst as the scientific advisor. Van der Alst has been the academic force behind research on process mining and is the instructor for arguably the best structured MOOC on the subject. There are also resources like ProM and fluxicon which have been active in the field since 2014.

Since 2016, IEEE has been maintaining the standard data format XES for event logs to enable streamlines process analysis. In 2018, Gartner published a market guide for process mining and hailed it as an enabler towards digital transformation. In April 2019, Harvard business review published an article which describes the need for mining processes and deriving useful insights from process data.

This makes process mining an interesting exploration for data science enthusiasts as well as business process owners.

Available tools

As software offerings, Celonis, Fluxicon and QPR provide solutions for process discovery. In the open source data science world, R and Python communities have libraries bupaR and pm4py respectively to work on the process data. While this notebook is built on R and works with bupaR, the fundamental priciples are common to other tools. The R library is more mature, while as a platform python has picked up in recent years. Python library built on the same principles is sure to catch-up.

Understanding process data

The essential data capture schema for process mining is an event log. Basically three aspects that should be necessarily captured are 1. Activity - a well understood step in the process, for instance in a IT process, calling help desk is an activity/ 2. Case identifier - the unique identifier to which multiple activities can be tagged and tracked. In the same example of an IT process, case identifier would be serial number assigned to the case. 3. Activity instance identifier (or sequence) - the connect between 1) and 2) above, for instance the starting and ending of phone call would be tagged with the instance identifier.

Time stamp, resource, location and other details are also sometimes available in event logs and can be used for selective filtering.

BUsinesss Process Analytics with R (BUPAR)

Introducing dataset

For this exercise, I have used bank transaction data from 4TU data repository. Data is available here

This data Event log concerning the ticketing management process of the Help desk of an Italian software company which could be a generic replacement for any help desk processes (including IT, HR, Finance etc.).

## [1] "Case.ID"            "Activity"           "Resource"          
## [4] "Complete.Timestamp" "Variant"

The column names from the above data need to be mapped to the standard event log nomenclature. This is done by using the event log creating command. The standard nomencalture includes following column inputs

• case_id - which is a unique identifier for the whole process sequence,
• activity_id - a description on the individual activity within a process,
• activity_instance_id - which instances of activity should be treated as different from others,
• lifecycle_id - status/outcome of the process,
• timestamp - time at which logging was done,
• resource_id - machine/individual responsible for the instance of the process/activity.

However, before that the date time data needs to be brought to a proper date time format. For this, lubridate library is used.

## Warning: package 'lubridate' was built under R version 3.4.3

At this point, an activity instance is added to the logs and an event log is created.

## [1] "There are a total of 14 activities, carried out by 22 resources for 4580 cases in 226 unique different combinations."

Activity Understanding & Analysis

Let us start what different level of activities are present and their relative frequency

This shows that take incharge ticket is the highest frequency activity. Around 1500 cases also have a wait activity which needs to be better understood. There are a few cases which required upgrade, or anomaly treatment as well.

Visualizing process maps

Let us start by viewing one of the process maps. Process maps show the flow of the process in a sequential sense. This is something which a subject matter expert could validate for a reasonable number of processes.

## Warning: Prefixing `UQ()` with the rlang namespace is deprecated as of rlang 0.3.0.
## Please use the non-prefixed form or `!!` instead.
## 
##   # Bad:
##   rlang::expr(mean(rlang::UQ(var) * 100))
## 
##   # Ok:
##   rlang::expr(mean(UQ(var) * 100))
## 
##   # Good:
##   rlang::expr(mean(!!var * 100))
## 
## This warning is displayed once per session.

This shows for one of the cases, how the steps involved in the process play out. As we add more data, we can start to see some of the possible iterations in the process play out. It appears that the steps for this case are 1. Assign seriousness 2. Take in charge ticket 3. Resolve Ticker 4. Close

Self loops represent that more than one logging of the same activity is observed, which might be due to error messages or logging discrepancy or different resources looking into the activity.

If we add two more cases to this, more insights start to appear, see below plot for 3 cases.

It is clear that one of the cases (1/3) went for wait rather than resolution.

Process maps can look at all of the cases data together.This however can be difficult to read at times.

Similar to process maps, resource maps are also another way of understanding flow. Interactions between resources should be understood to decide which resource activities can be helped through automation.

There are further ways available for exploring processes.

Precedence Matrix & Dotted Chart

Another way to visualize the process is precdence matrix which shows which steps tend to happen together.

Dotted chart adds an element of time taken to the activity order. Below plot shows for subset of data that time between reolution and closing is not marked. and the wide gap shows that between being marked resolved to closing the case can take a week!

## Warning in deprecated_y_arg(sort, ...): Arguments y is deprecated. Use sort
## instead.

## Joining, by = "Case.ID"

Resource & activity Analysis

Resource specialization and utilization is another key activity which can be helped by process analytics

The above plot for instance shows that Value 2 is a generalist, performing upto 12 activity types, Value 2, Value 22 are specialists looking into only about 2 activities.

In terms of activities, it would also be useful to understand which activities are always performed, and which are rare.

Based on these activities, it seems that

Trace analysis

Trace length and plots are used to see how much variation is there across cases.

##       min        q1    median      mean        q3       max    st_dev 
##  2.000000  4.000000  4.000000  4.661135  5.000000 15.000000  1.180026 
##       iqr 
##  1.000000

The plot shows that on an average, 75% of cases have 4-5 steos, although a maximum of 15 steps have been observed as well. Median number of steps is 4, and average number of steps is slightly higher.

Do all activities start and end at the same points? This can be visualized using bar plots as well

Assigning seriousness almost always seems to be the starting point.

As expected, closed is the final step.

The above plot shows how many cases can be described with a relatively small number of traces indicating the consistency in the process. Here it is visible that slightly more than 50% of cases can be described with a single trace. This trace is something which could then be automated.

A relevant trace

Below plot shows 75% most frequenct traces in all the traces. this shows that ~52% of cases directly go from assessment to raised ticked to resolution to close, without a wait time.

Interim Conclusion

Most activities start with assigning a seriousness score, that is also reasonably resource-agnostic. Although in this dataset, tha variable for seriousness and its rationale is missing (which does prophetically seem to point to deficiencies in the process documentation). But, with sufficient knowledge based rules (or text based learning) this is an activity which could be a candidate for automation.