A hybrid case-based reasoning clinical decision support system for treatment-planning of bone tissue engineering and regenerative medicine approaches

Longstanding limitations of traditional bone grafting treatment methods have led to the development of alternatives known as bone graft substitute methods (BSMs). These range in complexity with newer approaches at pre-clinical, clinical trial and approved development phases typically employing cutting-edge materials and techniques from a wide variety of disciplines. This inter-disciplinary complexity combined with an expansion in the number and quality of clinical trials for emerging therapies each year means that there are increasing demands placed on decision-makers to make informed treatment planning choices. The case based reasoning methodology (CBR) is conceptually similar to medical-decision-making. It is a process by which a computer system can offer an instance from experience when faced with a new problem, foregoing the need to build a solution from first principles. This study documents the design of the first clinical decision support tool (CDSS) built around case based reasoning principles for treatment planning with BSMs including bone tissue engineering and regenerative medicine (BTERM) approaches.

CBR is often combined with other techniques to improve its functionality and tailor it to specific requirements of the domain. In the current work, the CBR method was hybridised with the following: a novel preset feature weighting algorithm (PFWA), which assigns weights locally to each user query based on the expected magnitude, number of matches and the likelihood of matches in the co-feature space; and a case set filtering element, reducing the case set used by the PFWA to a subset of more relevant cases to the user query. These elements were aimed at providing an initial treatment recommendation (ITR), representing the closest match to the user query. An additional system output, namely, the co-citation adjusted treatment recommendation (coTR), was produced by a novel intra-cluster co-citation retrieval (ICCCR) technique which uses citation network relationships to highlight emerging treatment methods within the query’s specialty.

Evaluation of the hybrid-CBR retrieval system’s ITR output took place using 23 unique user queries. A Euclidian distance based metric called ‘retrieval performance’ was used to evaluate each configuration’s capability of finding the most similar treatment to the user query. The full hybrid functionality performed an average of 8.8 percentage points higher than the evenly-weighted benchmark system (p=0.000027, α=0.05). Disabling case set filtering led to a 0.3% drop in the average retrieval performance, although this result was not statistically significant (p=0.46). The practical significance of using the full hybrid configuration over an evenly-weighted tool was shown by five user queries where changes in the top case coincided with increases in retrieval performance. Furthermore, the preset functionality at the heart of the full hybrid system was found to be more readily generalisable than other weight setting approaches, making it more likely to perform well in the event of new queries or if the landscape of the BSM domain changes. In testing this, PFWA avoided undesirable weighting and retrieval performance changes in five out of eight examples of the most unusual (limiting) circumstances.

The coTRs produced by ICCCR were assessed using semi-quantitative relevancy checks, which ensured that at least 50% of the ‘treatment space’ values in the ITR are within range of the corresponding cluster’s values. 84% passed the check and resulted in successful coTR. Also, intrinsic cluster evaluations using modularity and silhouette coefficient measures (0.844 and 0.951 respectively) suggest that reliable insight can be derived from the clustering. Furthermore, tests using 9 real user queries revealed that the ITR and the next best case tended to be found within the same cluster and were 3-12% more similar (depending on the CBR configuration utilised) than the best case from elsewhere, supporting the notion that the clustering is meaningful to the domain.

These findings support the use of a full hybrid CBR-CDSS including PFWA, CF and ICCCR for clinical treatment planning within the BSM domain. Other configurations of this functionality were found to be less effective at suggesting the best ITR. ICCCR was effective at producing relevant coTRs and the clustering which they are based on was found to be of high quality and generally consistent with the treatment similarity discernible by the CBR process. This marks the first time, to the author’s knowledge, that clinical bone tissue engineered treatments have been compiled and represented in a decision support tool.