The Lund Tram Project - an application of the successive method

Per Broberg
August 5, 2014

Introduction

In March 2011 a number of people gathered to assess the cost of building a tramway in Lund, Sweden. The work on this presentation:

  • Start
  • Outcome
    • validation of the calculations presented
    • a critical appraisal of the method
    • a critical view of the relevance to decision making

Procedure of the successive method as used by Lund

  • The participants discussed the project iteratively
  • The outcome was estimates of minimum, likely and maximum costs for each part of the project

Data from the report

  • Data from the report were entered manually
  • An R data.frame with the columns below
    • Name, represents to cost category
    • Min, Likely, Max represent the Minimum, Likely and Maximum cost
[1] "Namn" "Min"  "Trol" "Max"

Some technical detail

  • For each cost category
    • the mean is estimated through \( \frac{Min+2.9Norm+Max}{4.9} \)
    • the standard deviation (STD) through \( \frac{Max - Min}{5} \)
  • The total cost is estimated as the sum over the categories
  • The variance is obtained by summing variances over the categories

Results in million SEK

      Names Costs STD
1 Generella   115  NA
2      Best   190  NA
3      Mark    95  NA
4     Gator    91  NA
5     Total   779 124

Graphical presentation

  • The report claims that the following distribution captures the uncertainty of the cost estimate

plot of chunk unnamed-chunk-3

Empirical outcomes

“…a study of 44 urban rail projects _ in North America, Europe, and developing nations, including London’s Tube and the metros in Washington, D.C., and Mexico City _ found that the average construction cost overrun in constant prices was 45 percent”

Flyvbjerg et al (2009)

It is not clear how many of these used the successive method. But it is quite common.

Empirical cost overrun distribution

  • A quarter of the project had over 60% cost overrun
  • Seems to have fat tails
  • The official report gives an impression of a low risk project
    • with a bell-shaped narrow distribution of cost
    • as likely to have a cost overrun as not
  • However, this is in stark contrast to actual data

Shape of distribution

  • Considering the fact that
    • very few project are much faster than planned
    • and, a notable fraction are much longer
    • a fat tail distribution seems more relevant, such as the following

plot of chunk unnamed-chunk-4

some more problems ( Part 1)

  • Is the successive method frequentist or Bayesian ?
  • Suppose first it is frequentist
  • Suppose then you have
    • a model with a certain uncertainty attached to it
  • Suppose you use the model for prediction
    • then the uncertainty of the estimated model
    • is compounded by the uncertainty coming from drawing a new observation under the model
  • The successive method does not account for this uncertainty

Some more problems ( Part 2 )

  • Suppose next the method is rooted in the Bayesian tradition
  • Then there is no prior presented that will yield the normal distribution ultimately used for the prediction

Some more problems ( Part 3)

  • The estimation of standard deviation does not agree with sound statistical principle
    • the estimate tends to increase as the sample size grows
    • the range / 5 has no the theoretical foundation
  • The estimate of the mean is ad hoc
    • the weights do not represent anything