- Previous calibraiton efforts relied on updating values of constant parameters
- Model outputs are most sensitive to the constant parameters
- We were able to achieve good calibration metrics with this strategy
- Downside: only updating constants leads to overfitting.
- In this study: analyse sensitivity of transit-related behavioral parameters
2023-10-10
Overview
18-Vriable Analysis
- We started with an 18-variable sensitivity analysis.
- All of them are transit relatested
- HBW_bInitWait_TransferTime_XitDrv
- HBW_bInitWait_TransferTime_XitWlk
- HBW_bIVTT_AcEgT_XitWlk
- HBW_ASC_XitWlk
- HBW_ASC_RailDrv
- HBW_ASC_Taxi
- HBW_ASC_Bike
- HBW_ASC_XitWlk
- HBW_b_RoadNtwkDensityAtDestBG_Drive
- HBW_bCost_tncXit
- HBW_b_HHVEH_XitWlk
- HBW_b_HHSize1_GetRide
- HBW_bCost_Generic
- HBW_b_HHVEH_RailWlk
- HBW_b_Edu_AssocPlus_XitWlk
- HBW_ASC_Walk
- HBW_b_NoDL_GetRide
- HBW_ASC_RailWlk
- HBW_b_Edu_LtHS_GetRide
- HBW_b_NoDL_RailWlk
- HBW_NEST_BIKE_AND_PNR
- HBW_b_HHSize_XitWlk
- HBW_b_Edu_GradProf_Walk
Morris Sensetivity Methodology
- Design experiment: OAT (One At Time); i.e. only one parameter vary at a time.
- Global: over the entire input variables’ bounding box
- Main goal: varibale selection (exclude unimportant variables)
- Two main summary statistics: \(\mu_i\) average change in output when input \(i\) is changed, \(\mu_i^*\) absolute value of change.
- \(\mu^*\) is not sensitive to +/- cancellations and better measure effect even when it is non-monotone and non-linear.
Morris Analysis
BART Sensetivity Methodology
- Bayesian Additive Regression Trees is a sum-of-trees model for approximating a univariate function \[y = f(x) + \epsilon\] using a sum of trees \[f(x)\approx h(x)\equiv \sum_{j=1}^{m}g_j(x)\]
- Advantages of this approach for sensitivity are the effective stochastic search of the BART algorithm and the simple variable usage counts.
- We use variable count summary statistic, which provides information about which variables are used in the trees our BART fit
- We have samples of this summary statistic for each of the variables and plot the means and blue dashed line which is [0.05,0.95] quantile.
BART Analysis
Results for 18-variable analysis
- Both Morris and BART approaches agree that the following varibales are important:
- HBW_bIVTT_AcEgT_XitWlk
- HBW_ASC_XitWlk
- HBW_ASC_Bike
- HBW_b_RoadNtwkDensityAtDestBG_Drive
- HBW_ASC_Walk
- HBW_NEST_BIKE_AND_PNR
- HBW_b_Edu_GradProf_Walk
Four varibale analysis
- We pick four variables with the highest level of sensetivity identified bot by Morris and BART
- HBW_b_HHSize_XitWlk, HBW_bIVTT_AcEgT_XitWlk, HBW_b_RoadNtwkDensityAtDestBG_Drive, HBW_NEST_BIKE_AND_PNR
| 1 | 2 | 3 | 4 |
|---|---|---|---|
| -0.7342822 | -0.0133673 | -6.638646 | 1.2472212 |
| -0.8616598 | -0.1950287 | 1.635876 | 0.3978892 |
| 2.1099235 | -0.1281620 | -5.332324 | 0.7499557 |
- We ran latin hypercube design with 40 designs
Four varibale analysis: Design of Experiment
Four varibale analysis
Average number of trips by mode across designs 
Four varibale analysis
Deviations from the mean for top ix modes across designs 
Mode 2 is volatile, otherwise low sensitivity from design to design
Four varibale analysis
Another output metric we use is deviations of total number of “in-network” vehicles \[Y_{net,i} = \dfrac{1}{720}\sum_{t=1}^{720}|y_t - \hat y_{t,i}|,\]
- \(y_t\) in the number of “in-network” at time \(t\) for the baseline scenario
- \(\hat y_{t,i}\) in the number of “in-network” at time \(t\) for the design \(i\)
More variance compafed to number of trips by mode!
Four varibale analysis
.
Four varibale analysis
.
Four varibale analysis: Network
.
Four varibale analysis: Conclusion
- Most of the behavioral variables (non-constants) appear to be not important (output is insensitive to those)
- When 4 most sensitive variables were analyzed separately, only two are important:
HBW_b_RoadNtwkDensityAtDestBG_Drive,HBW_NEST_BIKE_AND_PNR - However overall sensetivity of both network flows (in-network counts) and mode splits is low.
Next step: sequential design of experiment. Attempt to perform optimization