Automated UI testing suffers from fragility due to contin- uous – although minor – changes in the UI of applications. Such fragility has been shown especially for the web domain, though no clear evidence is available for mobile applications. Our goal is to perform an exploratory assessment of the extent and causes of the fragiliy of UI automated tests for mobile applications. For this purpose, we analyzed a small test suite -that we developed using five different testing frameworks- for an Android application (K-9 Mail) and observed the changes induced in the tests by the evolution of the UI. We found that up to 75% of code-based tests, and up to 100% of image recognition tests, had to be adapted because of the changes induced by the evolution of the application between two different versions. In addition we identified the main causes of such fragility: changes of identifiers, text or graphics, removal or relocation of elements, activity flow variation, execution time variation, and usage of physical buttons. The preliminary assessment showed that the fragility of UI tests can be a relevant issue also for mobile applications. A few common causes were found that can be used as the basis for providing guidelines for fragility avoidance and repair.

Automated Mobile UI Test Fragility: An Exploratory Assessment Study on Android / Coppola, Riccardo; Raffero, Emanuele; Torchiano, Marco. - STAMPA. - (2016), pp. 11-20. (Intervento presentato al convegno 2nd International Workshop on User Interface Test Automation - INTUITEST 2016 tenutosi a Saarbrücken, Germany nel July 18–20, 2016) [10.1145/2945404.2945406].

Automated Mobile UI Test Fragility: An Exploratory Assessment Study on Android

COPPOLA, RICCARDO;TORCHIANO, MARCO
2016

Abstract

Automated UI testing suffers from fragility due to contin- uous – although minor – changes in the UI of applications. Such fragility has been shown especially for the web domain, though no clear evidence is available for mobile applications. Our goal is to perform an exploratory assessment of the extent and causes of the fragiliy of UI automated tests for mobile applications. For this purpose, we analyzed a small test suite -that we developed using five different testing frameworks- for an Android application (K-9 Mail) and observed the changes induced in the tests by the evolution of the UI. We found that up to 75% of code-based tests, and up to 100% of image recognition tests, had to be adapted because of the changes induced by the evolution of the application between two different versions. In addition we identified the main causes of such fragility: changes of identifiers, text or graphics, removal or relocation of elements, activity flow variation, execution time variation, and usage of physical buttons. The preliminary assessment showed that the fragility of UI tests can be a relevant issue also for mobile applications. A few common causes were found that can be used as the basis for providing guidelines for fragility avoidance and repair.
2016
978-1-4503-4412-8
File in questo prodotto:
File Dimensione Formato  
isstaws16intuitestmain-intuitestmain2-p-6330f22-27021-preprint.pdf

accesso aperto

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 13.26 MB
Formato Adobe PDF
13.26 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2644368
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo