Tableau to Redshift Writeback using R: Test Details

Type of Test: Load/Stress Test

Scope: Monitor and Assess the resource requirements on the backend R server and aid in estimating the costing for R component

Time Window : 27 May 2020, 9:00 am - 10:00 am EST

No. of Partcipants: 12

Participants:


Details:
The R based writeback functionality available for Tableau utlizes Rserve process started in daemon mode on the R server. Rserve acts as a socket server which allows binary requests to be sent to R. Every connection has a separate workspace and working directory.

NOTE: The statistics presented in this report are the closest possible approximations. The data collection is done at ‘per second’ frequency.

Summary Report of CPU and Memory Usage by Rserv


##        DT                           CPU               MEM        
##  Min.   :2020-05-27 10:00:00   Min.   :   0.00   Min.   :0.1000  
##  1st Qu.:2020-05-27 10:15:00   1st Qu.:   0.00   1st Qu.:0.1000  
##  Median :2020-05-27 10:30:00   Median :   0.00   Median :0.1000  
##  Mean   :2020-05-27 10:29:59   Mean   :  11.77   Mean   :0.4287  
##  3rd Qu.:2020-05-27 10:45:00   3rd Qu.:   0.00   3rd Qu.:0.1000  
##  Max.   :2020-05-27 11:00:00   Max.   :1065.20   Max.   :9.2000

Graphical Representation of CPU and Memory Utilization by rserve

Total Cores allocated on R Dev server: 8 Cores (1 core ~ 100% CPU )
Total RAM allocated on R Dev server : 32 GB

Zoom In view of Round 5 testing: Concurrent submissions

Plot1: Total CPU Vs.Time
Plot2: Total Memory Vs. Time
Plot3 : CPU Per Process(top 10) Vs. Time
Plot4: Memory Per Process(top 10) Vs. Time

Round 5 Submissions- Summary

##        DT                           CPU              MEM       
##  Min.   :2020-05-27 10:41:05   Min.   :   0.0   Min.   :0.100  
##  1st Qu.:2020-05-27 10:41:23   1st Qu.:   2.0   1st Qu.:5.425  
##  Median :2020-05-27 10:41:35   Median :   4.0   Median :6.100  
##  Mean   :2020-05-27 10:41:35   Mean   : 101.7   Mean   :6.244  
##  3rd Qu.:2020-05-27 10:41:47   3rd Qu.:   8.0   3rd Qu.:7.875  
##  Max.   :2020-05-27 10:42:00   Max.   :1065.2   Max.   :9.100

Table: Top 150 rserve processes identified on the server during the testing window


PID=14430 corresponds to the master rserv process which runs as a daemon on the server.

##Observations & Conclusion:

The total CPU usage by rserve was beyond 9 cores (during Round 5 Tetsing) for a second, whereas the actual number of cores is 8. This indicates that the CPU peak time for the process must be for fraction of a second and hence the per second monitor is reporting a CPU usage of more cores than available.

Overall, the load on the server looks ‘moderate’ as the processing takes a very short span of time. Total RAM usage at any instant was not beyond 3 GB which is considered ‘moderate’.

However any additional R processing for Tableau should be provisioned after doing a stress analysis similar to this.

——————————————–end of file——————————————————