Austin Sensitivity Analysis

Author

Vadim Sokolov

Published

July 2, 2024

Exploratory analysis

X = load_json_training_data_x('data/timedep_training_data.json'); 
n = nrow(X); p = ncol(X)
X = cbind(X,1:n)
colnames(X)[p+1]="run_id"
Y = read_csv('data/vmt_vht.csv') %>% select(million_VMT,million_VHT,speed_mph,count,run_id)
X =  as_tibble(X)
d = inner_join(X,Y,by="run_id")
X = as.matrix(d[,1:p])
Ym = d[,(p+2):ncol(d)]

There are 12 variables used for this analysis. Below are the names of the variables along with the range used for sampling.

index name min max
1 S_AMPEAK_TT -1 0
2 S_AMOFFPEAK_TTV -1 0
3 S_AMOFFPEAK_TT -1 0
4 S_PMOFFPEAK_TT -1 0
5 S_PMPEAK_TT -1 0
6 S_EVENING_TT -1 0
7 S_AMOFFPEAK_OCCUPANCY -1 0
8 S_PMOFFPEAK_OCCUPANCY -1 0
9 S_PMPEAK_OCCUPANCY -1 0
10 D_AMPEAK_OCCUPANCY -1 0
11 D_AMOFFPEAK_OCCUPANCY -1 0
12 D_PMOFFPEAK_OCCUPANCY -1 0

We used Morris design of experiment. Here are the scatter plots for the first five variables

plot(d[,1:6])

Let’s plot the \(Y\) variables

plot(Y$million_VMT, type='b', pch=16)
plot(Y$million_VHT, type='b', pch=16)
plot(Y$speed_mph, type='b', pch=16)
plot(Y$count, type='b', pch=16)

Sensetivity Analysis

bartsens(X,Ym$million_VMT)
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 0.500303, 0.429788
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 200
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.052169,3.000000,0.040544
*****sigma: 0.456222
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 13s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 0.500303, 0.429788
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 20
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.164972,3.000000,0.040544
*****sigma: 0.456222
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 2s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
          S_AMPEAK_TT       S_AMOFFPEAK_TTV        S_AMOFFPEAK_TT 
           0.06951316            0.08563046            0.09941880 
       S_PMOFFPEAK_TT           S_PMPEAK_TT          S_EVENING_TT 
           0.07239115            0.04825223            0.14717750 
S_AMOFFPEAK_OCCUPANCY S_PMOFFPEAK_OCCUPANCY    S_PMPEAK_OCCUPANCY 
           0.14551839            0.05769956            0.04900640 
   D_AMPEAK_OCCUPANCY D_AMOFFPEAK_OCCUPANCY D_PMOFFPEAK_OCCUPANCY 
           0.03502843            0.09151225            0.09885168

Bart Sensetivity

Bart Sensetivity
bartsens(X,Ym$million_VHT)
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: -0.584622, -0.556985
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 200
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.051509,3.000000,0.073599
*****sigma: 0.614685
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 13s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: -0.584622, -0.556985
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 20
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.162886,3.000000,0.073599
*****sigma: 0.614685
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 1s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
          S_AMPEAK_TT       S_AMOFFPEAK_TTV        S_AMOFFPEAK_TT 
           0.11554162            0.08993050            0.05782963 
       S_PMOFFPEAK_TT           S_PMPEAK_TT          S_EVENING_TT 
           0.07558888            0.08187926            0.08638834 
S_AMOFFPEAK_OCCUPANCY S_PMOFFPEAK_OCCUPANCY    S_PMPEAK_OCCUPANCY 
           0.06193478            0.16846605            0.06661805 
   D_AMPEAK_OCCUPANCY D_AMOFFPEAK_OCCUPANCY D_PMOFFPEAK_OCCUPANCY 
           0.04651861            0.07534411            0.07396015

Bart Sensetivity

Bart Sensetivity
bartsens(X,Ym$speed_mph)
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 3.919231, 3.603805
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 200
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.274990,3.000000,2.759239
*****sigma: 3.763654
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 13s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 3.919231, 3.603805
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 20
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,0.869595,3.000000,2.759239
*****sigma: 3.763654
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 2s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
          S_AMPEAK_TT       S_AMOFFPEAK_TTV        S_AMOFFPEAK_TT 
           0.09210529            0.09633230            0.06695027 
       S_PMOFFPEAK_TT           S_PMPEAK_TT          S_EVENING_TT 
           0.08622125            0.07888994            0.08159454 
S_AMOFFPEAK_OCCUPANCY S_PMOFFPEAK_OCCUPANCY    S_PMPEAK_OCCUPANCY 
           0.07899246            0.15758113            0.06292884 
   D_AMPEAK_OCCUPANCY D_AMOFFPEAK_OCCUPANCY D_PMOFFPEAK_OCCUPANCY 
           0.04383208            0.07835724            0.07621466

Bart Sensetivity

Bart Sensetivity
bartsens(X,Ym$count)
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 180789.894737, 165581.894737
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 200
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,11016.157965,3.000000,3828878653.634756
*****sigma: 140200.914734
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 13s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
*****Into main of wbart
*****Data:
data:n,p,np: 95, 12, 0
y1,yn: 180789.894737, 165581.894737
x1,x[n*p]: -1.000000, -1.000000
*****Number of Trees: 20
*****Number of Cut Points: 3 ... 3
*****burn and ndpost: 10000, 10000
*****Prior:beta,alpha,tau,nu,lambda: 2.000000,0.950000,34836.150235,3.000000,3828878653.634756
*****sigma: 140200.914734
*****w (weights): 1.000000 ... 1.000000
*****Dirichlet:sparse,theta,omega,a,b,rho,augment: 0,0,1,0.5,1,12,0
*****nkeeptrain,nkeeptest,nkeeptestme,nkeeptreedraws: 10000,10000,10000,10000
*****printevery: 5000
*****skiptr,skipte,skipteme,skiptreedraws: 1,1,1,1

MCMC
done 0 (out of 20000)
done 5000 (out of 20000)
done 10000 (out of 20000)
done 15000 (out of 20000)
time: 1s
check counts
trcnt,tecnt,temecnt,treedrawscnt: 10000,0,0,10000
          S_AMPEAK_TT       S_AMOFFPEAK_TTV        S_AMOFFPEAK_TT 
           0.07941342            0.10099966            0.08950674 
       S_PMOFFPEAK_TT           S_PMPEAK_TT          S_EVENING_TT 
           0.08403033            0.08371647            0.09914001 
S_AMOFFPEAK_OCCUPANCY S_PMOFFPEAK_OCCUPANCY    S_PMPEAK_OCCUPANCY 
           0.10435869            0.13265792            0.05765022 
   D_AMPEAK_OCCUPANCY D_AMOFFPEAK_OCCUPANCY D_PMOFFPEAK_OCCUPANCY 
           0.04082779            0.06843312            0.05926562

Bart Sensetivity

Bart Sensetivity