one-versus-all.pdf (622.67 kB)

All versus one: An empirical comparison on retrained and incremental machine learning for modeling performance of adaptable software

Download (622.67 kB)
conference contribution
posted on 19.09.2019, 15:05 by Tao Chen
Given the ever-increasing complexity of adaptable software systems and their commonly hidden internal information (e.g., software runs in the public cloud), machine learning based performance modeling has gained momentum for evaluating, understanding and predicting software performance, which facilitates better informed self-adaptations. As performance data accumulates during the run of the software, updating the performance models becomes necessary. To this end, there are two conventional modeling methods: the retrained modeling that always discard the old model and retrain a new one using all available data; or the incremental modeling that retains the existing model and tunes it using one newly arrival data sample. Generally, literature on machine learning based performance modeling for adaptable software chooses either of those methods according to a general belief, but they provide insufficient evidences or references to justify their choice. This paper is the first to report on a comprehensive empirical study that examines both modeling methods under distinct domains of adaptable software, 5 performance indicators, 8 learning algorithms and settings, covering a total of 1,360 different conditions. Our findings challenge the general belief, which is shown to be only partially correct, and reveal some of the important, statistically significant factors that are often overlooked in existing work, providing evidence-based insights on the choice.

History

School

  • Science

Department

  • Computer Science

Published in

2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)

Pages

157 - 168

Source

2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)

Publisher

IEEE

Version

AM (Accepted Manuscript)

Rights holder

© IEEE

Publisher statement

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Publication date

2019-08-05

Copyright date

2019

ISBN

9781728133683

eISSN

2157-2321

Language

en

Location

Montreal, QC, Canada

Event dates

25th May 2019 - 25th May 2019

Depositor

Dr Tao Chen

Exports

Logo branding

Categories

Exports