BSAD343 Fall 2018 Lab Worksheet 02

Data Modeling & Exploration (bsad-lab02)

About

This worksheet includes three main tasks: data modeling (a key step to understand the data), basic steps to compute a simple signal-to-noise ratio, and data exploration to identify trends & patterns using Watson Analytics.

Setup

Remember to always set your working directory to the source file location. Go to ‘Session’, scroll down to ‘Set Working Directory’, and click ‘To Source File Location’. Read carefully the below and follow the instructions to complete the tasks and answer any questions. Submit your work to RPubs as detailed in previous notes.

Note

For your assignment you may be using different data sets than what is included here. Always read carefully the instructions on Sakai. For clarity, tasks/questions to be completed/answered are highlighted in red color (visible in preview) and numbered according to their particular placement in the task section. Quite often you will need to add your own code chunk.

Execute all code chunks, preview, publish, and submit link on Sakai.

---

Task 1: Data Modeling

To begin the Lab, examine the content of the csv file ‘creditrisk.csv’ by opening the file in RStudio. You can view the file separetely in Excel or use File -> Import Dataset in RStudio for that purpose.

##### 1A) Create a simple star relational schema in ERDPlus standalone feature https://erdplus.com/#/standalone, take a screenshot of the image, and add it below. Consider using one fact table for loan, one dimension table for customer profile, and one dimension table for credit risk.

To add a picture, use the directions found in Lab00. Below are steps and an example to create a simple star relational schema in ERDPlus.

Steps to create an star relation schema using erdplus.

From the drop-down option select New Start Schema

Example of how to create an start schema using erdplus

Completed Star schema example

Finally export the diagram as an image.

---

Task 2: Signal-to-Noise Ratio

Next, read the csv file into R Studio. It can be useful to name your data to create a shortcut to it. Here we will label the data, ‘mydata’. To see the top head data in the console, one can ‘call’ it using the function ‘head’ and referring to it by its given shortcut name.

mydata = read.csv(file="data/creditrisk.csv")
head(mydata)

To capture, or extract, the checking and savings columns and perform some analytics on them, we must first be able to extract the columns from the data separately. Using the ‘$’ sign following the label for the data extracts a specific column. For convenience, we relabel the extracted data. Below, we have extracted the checking column.

#Extracting the Checking Column
checking = mydata$Checking 
#Calling the Checking Column to display top head values
head(checking)

[1]    0    0    0  638  963 2827

##### 2A) Repeat here the above code chunk to extract instead the savings column. Be careful to use different variable naming.

#Extracting the Savings Column
savings = mydata$Savings 
#Calling the Savings Column to display top head values
head(savings)

[1]  739 1230  389  347 4754    0

In order to calculate the mean, or the average by hand of the checking column, one can add each individual row entry and divide by the total number of rows. Thankfully, R has a built-in command for this. We have done an example using the checking column.

#Using the 'mean' function on checking to calculate the checking average and naming the average 'meanChecking'
meanChecking = mean(checking)
#Calling the average
meanChecking

[1] 1048.014

We similarly compute the standard deviation or spread of the checking column

#Computing the standard deviation of checking
spreadChecking = sd(checking)
#Call sd_Chacking
spreadChecking

[1] 3147.183

Now, to compute the SNR, the signal to noise ratio, a formula is created because there is no built in function. SNR is the mean, or average, divided by the spread.

#Compute the snr of Checking and name it snr_Checking
snr_Checking = meanChecking/spreadChecking
#Call snr_Checking
snr_Checking

[1] 0.3330006

##### 2B) Repeat here the above code chunks calculations to derive instead the SNR for the savings column. Watch your variable namings to differentiate the checking calculations from the savings.

#Using the 'mean' function on savings to calculate the savings average and naming the average 'meanSavings'
meanSavings = mean(savings)
#Calling the average
meanSavings

[1] 1812.562

We similarly compute the standard deviation or spread of the checking column

#Computing the standard deviation of savings
spreadSavings = sd(savings)
#Call sd_Savings
spreadSavings

[1] 3597.285

Now, to compute the SNR, the signal to noise ratio, a formula is created because there is no built in function. SNR is the mean, or average, divided by the spread.

#Compute the snr of Savings and name it snr_Savings
snr_Savings = meanSavings/spreadSavings
#Call snr_Savings
snr_Savings

[1] 0.5038695

##### 2C) Of the checking and savings data , which one has a higher SNR? What does it mean in terms of possible data quality?

Savings has a higher SNR, which is meanin saving data is better and has higher quality than chenking data. The higher SNR means there is less distortion and the data is much better.

---

Task 3: Data Trends & Patterns

Login to Watson Analytics https://www.ibm.com/watson-analytics and upload the file creditrisk.csv to your account.

##### 3A) Consider the trend of ‘Months Employed over Age by Gender’. Include a screenshot of your plot. Refer to first lab session lab00 worksheet on how to include a picture. Write a small paragraph sharing your observations on the general data trends and data behavior.

From the graph, we can see that the male line is high than femeal line that menas male’s months employed is much higher than female’s, exspecially between age 34 to 41.