Institute for Transport Studies FACULTY OF ENVIRONMENT Statistical Benchmarking: What it can add to CQC Phill Wheat Senior Research Fellow 15th October 2013 What is Good? • Consider trying to determine if an authority is maintaining it’s highway network efficiently Different answers from different measures Cost per highway km Cost per head Which to use? • Some authorities will perform well against some criteria and badly against others • Lack of a single measure which gives a handle on potential cost savings that could be made What is Good? • Even if there is only one measure, it maybe the case that larger authorities have an implicit (scale) advantage over smaller authorities • Also other factors which influence (characterise) cost: Customer perception and output quality An authority has high cost, high quality Another has low cost, low quality The question we really want to know is whether either can reduce cost and still maintain quality What is Good? Cost B A O C Cost frontier (drawn for a given level of quality) Notice not a straight line… Economies of Scale (unit costs fall with the size of the authority) Highway Length What is Good? Cost C B D A O E Cost frontier (quality = high) Cost frontier (quality = low) The bottom line here is that authorities A, B, C, D and E are all achieving the minimum cost possible for their size and quality, even though their unit costs are quite different Highway Length What is Good? • Authorities X and Y are above the frontier and so are inefficient • By adopting best practice Authorities X and Y can reduce costs without sacrificing output X Cost Y A X’ C B Cost frontier (drawn for a given level of quality) Cost efficiency (TE) = Minimum cost/ Actual cost TEA=OX’/OX 0[...].. .What we DO get from this analysis • An estimate of the minimum cost for each authority tailored to its own characteristics, quality and citizen satisfaction • A tool to conduct what if analysis e.g How do (minimum) costs change if: – Authorities merged highway functions and increased network size for a given operation, – an Authority were to change quality (e.g to improve average condition by... large authority can be expected to have unit costs 15% cheaper than a small authority even when both are efficient (producing at minimum possible cost) The cost of incrementing quality: highway maintenance model What we DO get from this analysis • An estimate of the minimum cost for each authority tailored to its own characteristics, quality and citizen satisfaction • A tool to conduct what if analysis... Authorities merged highway functions and increased network size for a given operation, – an Authority were to change quality (e.g to improve average condition by 1%) – Authority was prepared to allow public satisfaction to reduce • A potential cost saving for each LA (if they closed the gap) • Identification of the best performing authorities which is useful to direct more process oriented analysis (to. .. (more years of data, more cost drivers, more cost categories) • But needs to be taken forward along side other analyses – Process benchmarking is important too – Authorities need the why as well as the what! Institute for Transport Studies Phill Wheat Senior Research Fellow Institute for Transport Studies University of Leeds p.e.wheat@its.leeds.ac.uk We offer taught courses, bespoke training and research... establish why there is a gap) What we DO NOT get from this analysis • An understanding of WHY there is a gap between actual observed cost and minimum cost – Analysis is useful to identify potential saving magnitude and direct to which LAs do look like they are performing close to minimum cost – But supplementary work is required to establish and disseminate best practice • The ‘gap’ is what is left over once... improve average condition by 1%) – Authority was prepared to allow public satisfaction to reduce • A potential cost saving for each LA (if they closed the gap) • Identification of the best performing authorities which is useful to direct more process oriented analysis (to establish why there is a gap) The estimated cost frontier: highway maintenance model Unit Cost at: 5000km of highway length = £2350... we control for the effects of cost drivers – We can only control for the cost drivers (variables) that we can collect – Thus there maybe other reasons (outside of an LA’s control) which my explain the gap • The gap is likely to be a maximum possible saving rather than an absolute Going forward • We have made a good start in the pilot study • Clear ways to take this forward (more years of data, more ... output quality An authority has high cost, high quality Another has low cost, low quality The question we really want to know is whether either can reduce cost and still maintain quality What is... maintenance model What we DO get from this analysis • An estimate of the minimum cost for each authority tailored to its own characteristics, quality and citizen satisfaction • A tool to conduct what if... for a given operation, – an Authority were to change quality (e.g to improve average condition by 1%) – Authority was prepared to allow public satisfaction to reduce • A potential cost saving