23 November 2001

Lies, Damned Lies And Software Metrics

Professor Barbara Kitchenham, Professor of Quantitive Software Engineering in the Department of Computer Science, gave the latest lecture in the University's programme of Inaugural Lectures for 2001/2002.The topic of the lecture, in the Westminster Theatre, was "Lies, damned lies and software metrics?".

Software metrics is the term given to measuring software development processes and software products and the specific measurements that are used. Software metrics are intended to help managers control and monitor software production projects and software maintenance tasks. Many "software metrics" are standard measures that would be applied to any manufacturing or engineering business. These include measures of labour (staff effort), time to market, defect rates. However, some measures are specific to software. These are measures of the software itself. For example, there is often the need to measure the "size" of a software application. One popular measure of size is lines of code. Lines of code is a count of the number lines of computer code in a program. The limitations of this measure can be appreciated by noting that it is analogous to measuring the "size" of a knitting pattern by counting the number of individual instructions.

Many large companies have reported good results using software metrics to assist project management, quality control and time to market. These include companies such as Nortel, Phillips, NASA Goddard. However, Professor Kitchenham believes that when we assume quantitative methods will assist software management, we are overlooking some major problems. Software is an intangible commodity so the properties we want to measure (e.g. size, quality, complexity) and our proposed measures cannot be judged against physical reality. This makes them difficult to calibrate and interpret. Software production process are human-intensive not machine intensive, and so models used to control development processes are inaccurate, context dependent and quickly get out of date. Defining measures is not the only problem, software engineers are often unable or unwilling to use data analyses methods correctly. This can lead to biased models and invalid analyses.

Her goal as a researcher is to enable the software industry to benefit from the use of metrics while minimising the chance of being managers being actively misled by metrics. Thus, her lecture described the limitations and problems associated with software metrics as well as the potential benefits. In addition, she suggested some approaches that could be adopted to avoid measurement and analysis errors. Finally she pointed out that the problems affecting software metrics apply equally to measuring any social phenomena. Thus, as members of society, we should be very cautious about accepting without question simplistic league tables for schools, hospitals, and other services.

Other lectures in the series are: Wednesday 12 December: "The Politics of Sin in the United States" by Professor Chris Bailey, School of American Studies; Thursday 17 January: "Emotions, Law and Crime" by Professor Susanne Karstedt, Department of Criminology; Thursday 14 February: "Constructing a Continent: European Union in the Palaeozoic Era" by Professor John Winchester, School of Earth Sciences and Geography; Wednesday 13 March: "Lessons from the Common Cold - can science conquer it in the future?" by Professor Monica Spiteri, School of Postgraduate Medicine; Wednesday 1 May: "Commitments and Ties: Solidarities in Personal Relationships" by Professor Graham Allan, School of Social Relations; Wednesday 22 May: "War and Poetry in the Romantic Period" by Professor Simon Bainbridge, School of English and Philosophy.

This week:

Vice-Chancellor's Committee - 19 November, 2001
(Keele access only)

Meeting Of Senate

Vice-Chancellor Speaks At National Audit Office Conference

Lies, Damned Lies And Software Metrics

Recent Research Grants