'Soft metrics': development and application of a framework for the measurement of human and organisational factors in projects

The 'soft metrics' project was a defence industry sponsored research activity undertaken to develop project performance analysis and control capabilities for systems engineering operations within BAE SYSTEMS. The 'soft' focus of this work addresses the less 'tangible' human and organisational factors that influence project effectiveness as measured by key contractuallydetermined cost, quality and time parameters related to delivery of the product. The key research objectives were: 1) to investigate the nature and influence of soft performance issues in projects, 2) to explore the extent of existing research and development knowledge for control of soft factors, 3) to develop an appropriate practical approach to the analysis and measurement of soft issues with tools to support project performance management efforts, and 4) to make a feasibility case for the research product through application and validation in operational case projects. The research approach involved in-depth, qualitative study of four relevant case projects undertaken to define the industrial context for application, support an iterative development process and validate the results of development efforts towards the specification of an integrated soft metrics tool and approach. An industry scoping study and detailed review of relevant research and operations management literature revealed a gap in current project management metrication practices regarding soft factors analysis and measurement capability. Building upon existing sociotechnical performance factors models an applied Human and Organisational Performance (HOP) modelling framework was developed for the representation of dependencies between upstream process indicators and 'outcome' effectiveness criteria in a systems engineering project context. Review of human sciences research literature and subsequent refinement through casebased investigation led to the identification of over 100 potential human and organisational performance variables with which to populate the HOP model, representing a variety of 'soft issues' known to influence performance in industry project based operations. Implicated factors were associated with issues relating to: team composition, human knowledge and experience, work group climate and cohesion, functional autonomy, task and goal characteristics, human workload, motivation, stakeholder communication and project management decision-making processes, amongst others. A variety of soft metrics and techniques for quantification of these factors were developed through a multi-facet approach to measurement that involved decomposition of broad, higher-level variables into specific sub-factors. Appropriate subjective judgement-based items and objective criteria were defined to numerically quantify variance in specific sub-factors. To provide a practical, integrated solution an application process with detailed sub-activities was developed to allow project management teams to identify and analyse 'soft' performance problem issues and select appropriate soft metrics for proactive monitoring and control within the project's lifecycle. This process was subsequently successfully implemented in three systems engineering case projects. Through implementation of a structured approach in focus groups, project managers reported they were able to identify and reason about complex human and organisational factors that had previously been managed intuitively, and relate them to specific effects upon project performance outcomes to support risk assessment. A variety of performance-critical 'system preconditions' were identified and linked to key outcome objectives within the HOP modelling framework, through their impact upon specific human activities or 'behavioural' variables that represented human performance in the project work environment. In terms of feasibility, project work groups reported that the soft metrics approach was of potentially high utility in supporting performance control through project planning, work process improvement efforts and project performance review activities, providing that practical issues associated with the level of effort currently involved in the implementation of the prototype tool were resolved. This work highlights metrication of human and organisational factors in projects as an important and viable area for future research work to support enhancement in operations performance management capabilities.