Final report on POD test scenarios & performance requirements. Deliverable 4.1.4 of the Capri project

The aim of the Capri project was to deliver a pilot scheme for the use of automated and connected passenger transport ‘pods on demand’ (PODs) as a mobility service in ‘campus’ locations such as airports, hospitals, business parks, shopping and tourist centres. These areas may be entirely privately owned or comprise elements of both off-road and public highway usage. Four trials have taken place during the Capri project; the first in an off road site closed to the public, two further on private off-road land that are open to the public, and the fourth trial operating as dual mode with the pod demonstrating use in public areas both off-road and on a privately owned road. A key part of the project was to build user and regulatory trust in these vehicles. A major concern of end users is whether or not autonomous vehicles (AVs) will be safe, which is why Work Package 4 of the Capri project was tasked with developing a robust verification and validation (V&V) process to assess the safety of the Capri POD. Primarily this V&V was done in simulation in a virtual world, and the role of Loughborough University was to design test scenarios for use in these simulations. Additionally, throughout the project it became clear that real-world physical safety testing was also required to reassure any concerns over carrying out the public trials. Although the PODs are thoroughly tested in-house by the manufacturer, Westfield Technology Group, Loughborough was asked to carry out further pre-trial testing to provide an independent evaluation of the safety performance. Detailed test plans for physical testing were created by combining the scenario generation methodology we had already developed for virtual testing, alongside our expertise in carrying out road accident investigation, real-world vehicle trials, and studies of driver behaviour. The scope of this report is to give insight into the scenario design methodologies that were developed by Loughborough throughout the Capri project, and how this methodology was utilised to generate scenarios for both virtual and physical validation of the POD. The AV sector has at times been criticised for not being transparent, leading the public to question if safety concerns are being hidden from them. It is hoped that by making this information available for Capri and demonstrating the robust safety assurance procedures that were followed during the project, that end users will gain more confidence in the safety of the technology. Section 2 of this report gives the reader some background information on AV development as well as some of the current procedures for testing and safety assurance for other AV types. Section 3 goes into detail on the process for designing scenarios, including the challenges, the methods developed within Capri to overcome these, and the data sources that can be used in scenario development. Finally, the specific testing protocols utilised for assessing POD performance in the virtual and physical worlds are respectively given in Sections 4 and 5. It is noted here that the testing protocols for the physical testing included in this report are significantly more detailed than those for the virtual testing. This is because the physical testing was carried out by Loughborough, so the protocols cover the implementation as well as the scenario design. For the virtual world, implementation was carried out by other partners in WP4, so only the scenarios and test parameters are described. Further information on how the simulations were carried out can be found in reports from T&VS, the University of Bristol, and the University of the West of England. Furthermore, safety concerns and testing procedures relating to cyber security were not within the scope of this report. This element was covered in detail by the University of Warwick and Nexor.