Using the JUMBL

This document describes how to use the JUMBL command line utilities. The utilities are discussed in the order they would typically be used.

More extensive documentation is available in the user's guide.

In this document, the command line prompt is indicated by a dollar sign (``$''). Text following the dollar sign is input which should be typed at the command line, and is shown in bold. Output generated by the JUMBL in response to a command is given in red code font.

JUMBL command names are not case-sensitive, and can be abbreviated by typing just the beginning characters needed for uniqueness.

$ jumbl WriteModel mymodel.tml
$ jumbl w mymodel.tml

These two command lines are considered identical by JUMBL.

Every JUMBL command provides online help by default. To get a list of JUMBL commands, enter jumbl by itself.

$ jumbl
JUMBL: Specify command.
The following commands are understood:
     About
     Analyze
     Check
     CraftTest
     Flatten
     GenTest
     ManageTest
     Options
     Prune
     RecordResults
     Write

This software is Copyright (c) 2002 by the Software Quality Research
Laboratory, all rights reserved. Visit http://sqrl.eecs.utk.edu/

The information available in the web command line guide is also available from the command line. Enter a command with no arguments or switches, or with the --help or -h switch to obtain the command line guide entry for that command.

$ jumbl Write
Usage: Write [files...]
Write the specified files to the given type, or to the default type
if none is specified. This can be used to convert from one
representation to another.

  -e [ext] or
  --extension=[ext] ......Specify a file extension to override the
                          default extension for the output type,
                          including the period.
  -h or
  --help .................Describe this command.
  --list .................List the types which can be used with the
                          --type option. If this option is specified,
                          no other action will be taken.
  -O [path] or
  --object_path=[path] ...Specify the object search path.
  -s [suffix] or
  --suffix=[suffix] ......Set the file name suffix to use for the
                          generated files. This is to prevent
                          overwriting the original with the new file.
                          The default suffix is the empty string. The
                          new file will be named
                          [base][suffix][extension], where the
                          extension is dependent on the type
                          selected.
  -t [type] or
  --type=[type] ..........Specify the type of the output. To find out
                          what types are available, use --list.

This software is Copyright (c) 2002 by the Software Quality Research
Laboratory, all rights reserved. Visit http://sqrl.eecs.utk.edu/

Some JUMBL commands write a model to a file. Whenever a JUMBL command writes a model to a file, it uses the SM format by default (see Converting Models).

More Information

Boxes like this add additional information, technical details, or additional usage examples.

Note that, even with these notes, this guide is not exhaustive. The JUMBL routines provide many additional capabilities not documented here, and the JUMBL itself provides capabilities not yet available from the command line.

Creating Models

You may develop your model in any file format understood by the JUMBL. The suggested format is The Model Language (TML). TML is a textual format designed specifically for representing Markov chain usage models, and supports hierarchical modeling, constraints, and automated testing information.

Other formats can be used as well. For example, GML is a language for expressing graphs, and is supported by several graphical tools, including the following:

Graphlet, a graph layout and editing tool. This is the utility used at SQRL and thus is the most supported. It runs on Windows, UNIX, and Linux.
The yed editor of yWorks, a commercial graph layout and editing tool. It is written in the Java language, and runs on any 1.2+ Java platform.
The Angela! graph editor, a GPL'd graph layout and editing tool. At the time of writing, Angela! does not provide for naming of edges and encodes color names in a manner not understood by the JUMBL. Angela! runs only on UNIX and Linux.

Model Markup Language (MML) is an XML extension for describing models.

Converting Models

You can convert from one file type understood by JUMBL to another file type using Write. By default, Write produces the saved model (SM) format.

$ jumbl Write *.gml *.tml

This would convert all your TML and GML files to SM files (with the extension .sm). You can specify the particular format you want with -t.

$ jumbl Write -t TML *.gml

This would convert all your GML files directly to TML (so you could add constraints and labels, for instance).

Avoiding Overwrite

$ jumbl Write mymodel.sm -t TML

This would normally overwrite mymodel.tml. To avoid this, you can specify a suffix to add to the name with -s.

$ jumbl Write mymodel.sm -s foo -t TML

This would re-name the model and write the TML to a file called ``mymodelfoo.tml''.

Checking Models

To check the structure of a model, use Check.

$ jumbl Check *.tml

This will check the structure of all your TML models.

What Check Looks For

Check verifies that there is a path (a sequence of arcs) from the model source to every node of the model. If a node is not reachable from the source, it is listed as ``unreachable.'' Check then verifies that there is a path from every node of the model to the sink. If the sink cannot be reached from a node, the node is listed as a ``trap'' node.

Check reports the number of nodes and arcs in the model.

Check verifies input trajectories and selectors.

Dealing with Referenced Models

If you have built a model out of component models (``referenced models'') then you may wish to ``flatten'' the model (reduce it to just states and arcs, with all model references either removed or instantiated).

To flatten a model by collapsing the submodels to single states, use Flatten --collapse.

$ jumbl Flatten --collapse TopLevel.tml

This will produce a SM file in which each model reference has been replaced with a single state. If you do not specify --collapse, each referenced model is instantiated. In order for this to work, SM files must exist for all referenced models. Create these SM files with Write (see Converting Models, above).

$ jumbl Flatten TopLevel.tml

This will flatten the model by replacing every reference to a model with the states and arcs of the referenced model. The flattenned model is placed in TopLevel.sm. This will remove all unreachable and trap nodes from the model, and then write the pruned model to TopLevel.sm. You can use Prune on any model, but its primary use is with Flatten.

How Flatten Names New Nodes

When you flatten by instantiating (the default), nodes are added to the model. These nodes are given a name of the form referenced_model.[state]. Name conflicts (such as when the same model is included more than once) are broken by suffixing an underscore and a number to the state name to make it unique.

Analyzing Models

The Analyze command (note the spelling) produces an HTML report with the suffix ``_an.html''. This report describes the statistical properties of the Markov chain. Analyze resolves constraints on the model, if any are present.

$ jumbl Analyze *.tml

This will produce one analysis report for each TML file in the directory.

See the JUMBL Statistics Guide for more information.

By default the unnamed distribution is used. If you want to specify a different probability distribution, use -k and specify the key. Remember that you specify a distribution key in TML by prefixing a constraint with the key and a colon, as follows.

($ 1/2 $)
fido:($ 1/3 $)

This gives two distributions; the default distribution which will use 1/2, and the fido distribution which will use 1/3.

$ jumbl Analyze mymodel.tml -k fido

This would analyze mymodel.tml with respect to the distribution with key ``fido''.

Generating Tests

Tests are generated by the GenTest command.

GenTest by default generates random walks of the model, conditioned by the specified probabilities (if any).
GenTest --min generates a non-random sample of tests which cover all arcs of the model with the lowest cost (if no cost is specified, 1.0 is assumed for every arc and thus the model is covered in the fewest steps possible).
GenTest --weight generates non-random tests in decreasing order by probability mass, with the highest first.

Specify the number of tests for GenTest (except when using --min) with either --num or -n.

$ jumbl GenWeighted -n 10 mymodel.tml

This will generate the ten highest-probability tests from mymodel.tml.

In order to facilitate later operations, every test generator causes the model to be written to a SM file.

Keys, Costs, and Weights

When using GenTest, you may want to select one of many probability distributions. By default, the unnamed distribution (``default'' distribution) is used. To select another, use -k.

$ jumbl GenTest -n 10 -k fido

This will generate ten tests from the model using the distribution with the key ``fido.'' See Analyzing Models for a short discussion of multiple distributions in TML.

For GenTest --weight, -k specifies the key of the weights to use and, again, the default distribution is assumed if none is specified. GenTest --weight tries to maximize the product of these weights.

For GenTest --min, the key for costs is specified with -k, and 1.0 is assumed if no key is explicitly given (note that, in this case, the default distribution is not used). GenTest --min tries to minimize the total cost, which is the sum of the arc costs.

Writing Tests

Tests are saved in Saved Test Records (STR files) with the extension .str, which can be manipulated using the ManageTest command. To export these to another format, use ManageTest Export. By default, Export tries to create a nice, human-readable text file (with the extension .txt) for each test. If you have used labels to specify automated test information, you can instead generate the labels by specifying the key to use with -k and the file extension with -e.

$ jumbl ManageTest Export MyModel.str -k dos -e .bat

This will write all tests in the MyModel test record to DOS batch files using the labels keyed with ``dos'' (if any). Remember that in TML you give a label a key by prefixing the label with the key and a colon, as follows.

dos:|$ echo Working

Test Results

If you have recorded results of a test (see Recording Results) then these will also be written out. Use --no-fail to suppress writing these results in the output.

Recording Results

Testing results are recorded in the test cases (in STR files) themselves using RecordResults. By default, this records that all events in every test were completed successfully. To select particular tests, give the test or range of tests after an at (@) symbol (use no spaces). To record failures, give a comma-separated list of failed steps (use no spaces). If testing stopped after the last failure, suffix ``,S'' to the list of failures.

Subsequent calls to RecordResults replace prior information, so the following process is recommended.

Record complete success. $ jumbl RecordResults MyModel.str
Record failure information for each test which failed. $ jumbl RecordResults MyModel.str@7,5,19 MyModel.str@12,17,21,S

This records that all tests passed except the seventh and 12th tests. Failures were observed during the seventh test on steps 5 and 19, and during the 12th test on steps 17 and 21. The 12th test stopped at step 21. Step numbers can be obtained from the text file created by ManageTest Export (see Writing Tests).

Results Files

If you have a lot of results, or if you are using automated testing, you may want to create a results file. The format of this file is essentially the same as the command line. Give a test name followed by a list of failures (if any). If testing stopped after a particular failure, use ``,S''. The following file contents are equivalent to the example above.

MyModel.str@7,5,19
MyModel.str@12,17,21,S

You place each test record on a separate line. You must not have any whitespace in the record itself (the same is true for the command line).

Indicate that RecordResults should use this file with --file=name.

$ jumbl RecordResults --file=results.txt

This uses the contents of the file results.txt as the failure information.

Getting a Test Analysis

If you have recorded the results of the testing experience (see Recording Results) then you can use Analyze to get a detailed analysis of the testing experience, including expected field reliability.

$ AnalyzeTest MyModel.str

This will write an HTML report with the suffix ``_ta.html'' containing information about the testing experience.

See the JUMBL Test Statistics Guide for more information.

By default the unnamed (``default'') distribution is used. If you want to specify a different probability distribution, use -k and specify the key. See Analyzing Models for a short discussion of multiple distributions in TML.

$ jumbl AnalyzeTest MyModel.str -k fido

This would analyze MyModel.str with respect to the distribution with key ``fido''.