MBST Description

Improvement of software quality requires a testing approach that can produce statistically meaningful metrics. One such approach is Model-Based Statistical Testing (MBST).

The first step in MBST is construction of a usage model as a finite state Markov chain to represent the population of all possible system uses. Test cases are generated sampling the set of paths from the model source to its sink.  Using the Markov properties of the model, a number of statistical parameters can be computed for each of the states and arcs in the model. These parameters provide a quantitative basis for model validation by domain experts.Analysis of test results along with model properties provides a quantitative basis for system certification and release. This tutorial provides a detailed description of the process.

JUMBL

JUMBL provides a library and command line interface to facilitate practical application of MBST. The library and  command line tools are written in Java™ for maximum portability. JUMBL accepts usage model definitions in a variety of formats, including text, graphics, and spread sheets. Model complexity is managed by support for hierarchical models.

For newly created or modified models, JUMBL commands are provided to check graphical integrity and to compute Markov properties to be used in model validation. Once a model has been validated, JUMBL commands are used to generate test cases in a variety of ways that can be generally categorized as random, graph theoretic, and crafted. JUMBL also has a facility for recording and managing test results. Pass/fail information is maintained at the arc (event) level. Finally computations, based on the model, test results, and a reliability model, produce quality metrics such as model coverage, system reliability, and sampling effectiveness.

Additional details are provided here and in the JUMBL User Guide, available from the download site.

ESTA'

ESTA' provides an Eclipse-based GUI to perform all phases of the MBST process. Many of the underlying functions are performed by the JUMBL. Usage models can be created and modified using a tree editor, and can be viewed as a directed graph. Usage models  are stored in an XML file format. However other common usage model file formats can be imported transparently.

Menu selections are provided to check model graphical integrity, perform model analysis, and generate test cases. Generated test cases are packaged in test record objects displayed in a tree editor and serialized as XML files. Once generated, test cases can be exported as text files suitable for manual or automated test execution. Test results can be recorded manually by editing test cases or imported in one of three file formats. Test records (with or without results recorded) can be analyzed using the JUMBL computational engines.