An Adjustable Sample Size Estimation Model for Usability Assessment

Abstract

Usability assessment plays a fundamental role in discovering usability problems and the determination of the level of usability for a given software product. One crucial aspect in every usability assessment is the estimation of the sample size desired for a software product. Once we start estimating the sample size needed for a usability assessment, a baffling question comes in mind: "how many users are enough for a given software product?". To the best of our knowledge, the majority of existing models estimates the sample size needed for a usability assessment, based on historical data. A better estimation should be based on both historical data, which provide an initial idea from previous software products and based on present data, which provide a practical idea for a given software product. Therefore, in this paper we have proposed an adjustable sample size estimation model for usability assessment, which enhances the estimation process by using two factors: Alpha factor (α), which estimates the problem discovery rate (λ_α) from historical data and the Beta factor (β), which estimates the problem discovery rate (λ_β) by using the complexity of the software product. λ_β is taken as vertical and domain wise for a given software product, to adjust the alpha factor appropriately to give the desired confidence level in the results. An illustrative case study has been provided at the end of the paper.