[Home] [Essays] [Book_Reviews] [Tools] [Links] [News] [Blog] [About]

Page 5

4     Using Graphs in Your Testing. 15

5     Experiments in visualisation. 18

4         Using Graphs in Your Testing

In previous sections I have covered how I used graphs in my testing process. Now I want to explain how you can start to use graphs in your own testing processes.

I have already mentioned that the tool support for using graphs in testing is not particularly extensive. I have written a number of programs to help, but you need to be able to understand how to do this for yourself. You also need to understand how the various tools available to you can be integrated into your test process.

We know that graphs, when represented as diagrams can still help us with our testing. We can use them to communicate and understand. And we can manually identify paths through the graphs and interpret the paths during testing. There are two basic levels for doing this:

1.        Work with Diagrams   

• communicate and understand
• identify paths for test case derivation

2.        Work with Diagrams with supporting node and edge annotation

• as above
• represent scripts as sequences of edges for repeatable testing

At level 1, we are constructing graphs as diagrams, and using the diagrams as a basis for communication. We use them as a basis for test case derivation by writing tests to cover all the paths that are appropriate to traverse through the graph, this typically involves red penning the graph as we write our tests.

At level 2, we are going in  to more detail and our graphs have to be more detailed. The graphs have to be detailed enough so that by following a path through the graph we can actually follow a script. But because human testers are involved, the descriptions on the nodes and edges don’t have to be to the same level of detail that would be required for automated execution. Giving us a certain degree of ambiguity in the abstraction we can use, and this works to our advantage, provided that there are not too many ways of interpreting the path.

It is essential to make a note of the path that the test is covering so each node and edge are given a unique identifier. The path is described as a list of the unique identifiers of the edges e.g. {1, 5, 3, 6}. When executing the script, the tester follows the path and reads the annotation against the node and edge to determine what to do during the script.

Experiment with the various tools in Appendix 4: Tools List, to see which one suits you best. You might well discover that at levels 1 and 2, normal vector graphic tools like Visio and Smartdraw will suit your needs, although you will miss out on the time saving automatic layout functionality.

We can actually do a lot of good testing work at levels 1 and 2, and you will note that in all the case studies I have presented I have been working at level 2. One of the reasons I wrote the Compendium-TA tool, was to move into experimenting with level 3 where we work with the actual graph rather than the diagram.

3.        Work with Graph Representation

• diagrams become an output
• automatic parsing
• automatic script generation

Graph tools focus on the visualisation of graphs. Testers are interested in that only as a side effect, our goal is not to draw graphs, but to use graphs to help conduct testing. We can conduct testing with graphs in a more automated fashion when graphs are represented in a format that we can easily handle. But be warned, in order to do any automated processing, such as generating test scripts from paths, or assessing coverage metrics, or any kind of advanced visualisation, we need to start doing some programming.

The graph tools (Appendix 4: Tools List) typically use a Graph Markup Language to represent the graphs (Appendix 5: Graph Languages). One of the difficulties with interfacing with the existing tools using programming languages is the number of graph representation languages which are in use, and the difficulty in getting some of the tools to work. Many of the tools are supplied in source format or are written in Java, some require extra libraries. During the body of the paper I have mentioned some of the easier tools to get working, and I have highlighed in Appendix 4: Tools List the tools which I recommend you start with.

There is no “Standard” graph markup language so when choosing one, we really need to focus on the tools that support the markup language we want to use, and the ease of integration into our testing process.

All the markup languages that we will be considering are simple text files. Examples are provided in Appendix 5: Graph Languages.

 The language formats

Ideally the markup language chosen will be simple to construct without tools, simple to parse and extensible. By extensible I mean the ability to add extra fields or notation i.e. adding descriptions to nodes, adding cross references fields.

GraphViz and GML are the oldest languages and are a structured text rather than XML. I find that this actually makes them simpler to manage and maintain without formal tool support.

The GraphViz format is used by the AT&T GraphViz toolset. It is a very simple representation and is my favourite when it comes to hand crafting the graph representations. GML was created as the markup language for the Graphlet tool and is also the text representation used by the VGJ tool. GML is a very simple structured  text language, and while it is easy to write by hand, I prefer the GraphViz representation.

All the XML variants are very similar but I find them more difficult to work with. If you are a better programmer than I, then perhaps XML is the ideal format for you, but be warned, the XML languages actually have poorer tool support.

The use of graph languages has two main considerations:

• use the graph mark up languages as our graph representation, in which case it has to handle extra fields (in which case we are also interested in tool support of the extra fields).
• use it as the output from our extended representation in which case it is the tool visualisation support that we are interested in.

One approach to graph construction that I have seen mentioned on the net is to write a small program with very little processing in it and pass it through doxygen (www.doxygen.org). Doxygen is a code parser which visualises the structure of the source code using GraphViz. I would recommend learning the very simple GraphViz representation rather than this more error prone and complicated process.

I believe that the tester is best served by using the graph representation languages as outputs of their own representations of graphs. This is the approach used in the Compendium-TA tool, but it is easy to take the same approach in Excel, or any spreadsheet or database that supports macros. One reason for choosing this option is that although the mark up languages can handle our extensions, the tools which support the markup languages generally do not, and often will not, persist your extensions when saving the graph.

I am now going to suggest that if you want to use graphs more effectively in your own testing processes then you, or a friendly programmer of your acquaintance, will have to do some programming. I have added a set of links in Appendix 4: Tools List (code libraries) should you wish to use standard graph libraries.

I will assume that you are using a spreadsheet would suggest the tabs in the sheet be:

-          Links

o        columns: linkID, fromNodeID, toNodeID, Name, Description

-          Nodes

o        columns: nodeID, Name, Description

-          Paths

o        columns: pathID, Description, linkID, linkID*

As an abstract example I will use the following graph:

It is a simple matter to write macros which will output a graph markup format from the above information for feeding into the tools for automatic layout.

From this information it is also a simple matter to construct the various matrix representations of graphs presented in [Beizer90] should you wish to experiment with the various algorithms presented there.

Another extension possible with the above representation is to expand the path representation into a textual script that the tester can follow. This is done by writing a text file for each link in the path and outputting the description of the fromNode, then the link description, then the toNode description. The fromNode description is only written for the first node link in the path.

It is undeniably useful to have a graph drawing tool for constructing graphs quickly, and if it is one of the tools which supports Dot or GML then the textual output from the tool is easy to manually convert into the above spreadsheet form.

For the more adventurous among you, it is surely possible to parse the output files automatically.

5         Experiments in visualisation

This is really the future vision, and possibility section of this paper, and is your chance to experiment, as I have only dabbled with this on client sites.

Given that graphs can be used as a communication tool so that everyone can be made aware of the testing that will be carried out on a system. It seems reasonable to assume that the same graphs can be used to document various project status information.

Information such as:

Defect Density

Progress

Coverage (measures of coverage 1, 2, 3 etc.)

Outstanding Defects

Good Bits (system elements with few faults)

Using a tool like GraphViz, it is easy to add extra information into the GraphViz file to change the colour of a node, or the style of an edge.

So for Progress, we might show the graph with areas which have been tested, as coloured blue, and areas which are currently under test as green.

For Defect Density we might have different representations for our defect density scale: 1 – 10 Outstanding defects (green circle), 11- 20 outstanding defects (blue circle),  21 – 30 outstanding defects (red circle), above 30 outstanding defects (red square).

In order to implement the above, we have to either use cross referencing information or extra attributes on the nodes and links and then a customisation to our export routine.

The defect density could be calculated by counting the number of cross referenced defects to a node on a graph or by using a defect density attribute on the node.

This extends the use of graphs as a communication mechanism throughout the testing lifecycle and may help communicate testing status to those managers who never seem to read the numbers in the progress reports.

Page 5