Breast screening: understanding case difficulty and the nature of errors

In the UK all screeners undertake the PERFORMS scheme where they read annual sets of challenging cases. During this assessment, they give each case a confidence rating on whether it should be recalled. If they decide to recall a case, they also indicate the center of any key mammographic features on a display of the relevant mammographic case view. Expert radiological opinion defines what the key abnormalities (targets) are in any case. Data can then be analyzed using ROC and JAFROC approaches, and particularly for the latter, assessing whether a user has correctly located a feature or not is important. Using image pixel information alone it is possible to delineate correct localization of an abnormality from an incorrect location by defining an area of interest. To explore such location information in more detail, data from the last year of the PERFORMS scheme were reanalyzed and the location responses for each of the 675 participants on 120 screening cases examined. Additionally, expert radiological opinions had been garnered for various reasons, including accurately delineating any abnormalities. An algorithmic approach is developed which assesses whether users’ indications should be included as correct abnormality identification or not, based on the feedback location information of all participants’ indicated locations and the relative position of an indicated location to the abnormality. This approach is proposed to be superior to simple pixel distance approaches which measure a fixed distance from the centre of a target to the user’s indicated location. The approach adds to the experimenter’s repertoire of tools when examining user errors and case difficulty in medical imaging research.