Difference between revisions of "Digital Test Manager"
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
'''Digital Test Manager (DTM)''' is a [[layered product]] distributed by [[VMS Software]], part of [[DECset]], that can be used to run, review, and store software regression tests and test results. DTM organizes and automates the software regression testing process. | '''Digital Test Manager (DTM)''' is a [[layered product]] distributed by [[VMS Software]], part of [[DECset]], that can be used to run, review, and store software regression tests and test results. DTM organizes and automates the software regression testing process. | ||
| + | |||
| + | =Environments= | ||
DTM supports three testing environments: | DTM supports three testing environments: | ||
| Line 6: | Line 8: | ||
* Interactive DECwindows | * Interactive DECwindows | ||
| + | =Components= | ||
The three basic components of DTM are: | The three basic components of DTM are: | ||
* A '''DTM library''' - a data structure that DTM uses to store the information it needs to manage a test system in an OpenVMS directory. | * A '''DTM library''' - a data structure that DTM uses to store the information it needs to manage a test system in an OpenVMS directory. | ||
| − | * A '''DTM test''' - a collection of fields that contains the information that VSI Digital Test Manager needs to run a particular test. A test description requires a template file and can have other optionally specified test-related entities. A '''template file''' is an OpenVMS command procedure that executes a noninteractive test, or a session file containing a recorded interactive terminal or DECwindows session | + | * A '''DTM test''' - a collection of fields that contains the information that VSI Digital Test Manager needs to run a particular test. A test description requires a template file and can have other optionally specified test-related entities. A '''template file''' is an OpenVMS command procedure that executes a noninteractive test, or a session file containing a recorded interactive terminal or DECwindows session. A '''benchmark file''' contains the expected output for the test's execution. |
| − | * A '''collection''' - a set of tests selected for execution. You can execute a test only in the context of a collection. You can select tests for inclusion in a collection by test name or groups | + | * A '''collection''' - a set of tests selected for execution. You can execute a test only in the context of a collection. You can select tests for inclusion in a collection by test name or groups. |
| − | + | ==Tests== | |
DTM tests are characterized by test descriptions. A '''DTM test description''' consists of the following elements: | DTM tests are characterized by test descriptions. A '''DTM test description''' consists of the following elements: | ||
| Line 26: | Line 29: | ||
* '''Filters''': identifies one or more filters to remove run-timedependent information from the result file. | * '''Filters''': identifies one or more filters to remove run-timedependent information from the result file. | ||
* '''Remark''': identifies a comment that you add to the history. | * '''Remark''': identifies a comment that you add to the history. | ||
| + | |||
| + | ==Collections== | ||
A '''collection''' is a snapshot of specified test descriptions and the VSI Digital Test Manager library as they exist at the time you create the collection. You must organize tests into collections before you can execute them to produce result files for comparison. | A '''collection''' is a snapshot of specified test descriptions and the VSI Digital Test Manager library as they exist at the time you create the collection. You must organize tests into collections before you can execute them to produce result files for comparison. | ||
| Line 33: | Line 38: | ||
When you execute a collection, VSI Digital Test Manager compares the test results with the benchmark file for each test that has been run. If the comparison is unsuccessful (differences are detected), VSI Digital Test Manager creates a difference file. If the comparison is successful (no differences are detected), the result file is deleted and no difference file is created. | When you execute a collection, VSI Digital Test Manager compares the test results with the benchmark file for each test that has been run. If the comparison is unsuccessful (differences are detected), VSI Digital Test Manager creates a difference file. If the comparison is successful (no differences are detected), the result file is deleted and no difference file is created. | ||
| − | You can then review the differences using the Review subsystem, which gives you access to the results obtained by executing the collection, as well as other information about the collection. It also enables you to invoke the Performance and Coverage Analyzer (PCA) to gather performance and coverage data for the test | + | =Instructions= |
| + | Before any DTM tests can be created, a DTM library needs to be created first. To do that, you create an empty directory and then run the CREATE LIBRARY command on this directory. If you already have a library, you can execute SET LIBRARY. When the library is set, it will be pointed to by the DTM$LIB logical. | ||
| + | |||
| + | Tests are created with a CREATE TEST_DESCRIPTION command. You can specify an existing template (.COM file) and benchmark (.BMK) file or RECORD an interactive test. To execute the tests, add them to a collection with CREATE COLLECTION, and then RUN that collection. | ||
| + | |||
| + | You can then review the differences using the Review subsystem, which gives you access to the results obtained by executing the collection, as well as other information about the collection. It also enables you to invoke the Performance and Coverage Analyzer (PCA) to gather performance and coverage data for the test. | ||
You must execute and compare a collection before it can be reviewed. | You must execute and compare a collection before it can be reviewed. | ||
| − | + | You can specify an initialization file to be executed when you run DTM either by defining the logical name DTM$INIT to it or with the /INIT qualifier when you run DTM. | |
| − | + | ||
| + | You can execute DTM commands from a file using @filespec inside of DTM. | ||
| + | |||
==See also== | ==See also== | ||
* [https://vmssoftware.com/docs/VSI_DTM_GD.PDF VSI DECset for OpenVMS Guide to VSI Digital Test Manager] | * [https://vmssoftware.com/docs/VSI_DTM_GD.PDF VSI DECset for OpenVMS Guide to VSI Digital Test Manager] | ||
| + | |||
| + | [[Category:VSI Products]] | ||
Latest revision as of 16:08, 13 November 2023
Digital Test Manager (DTM) is a layered product distributed by VMS Software, part of DECset, that can be used to run, review, and store software regression tests and test results. DTM organizes and automates the software regression testing process.
Environments
DTM supports three testing environments:
- Noninteractive (command procedure)
- Interactive terminal (recorded input)
- Interactive DECwindows
Components
The three basic components of DTM are:
- A DTM library - a data structure that DTM uses to store the information it needs to manage a test system in an OpenVMS directory.
- A DTM test - a collection of fields that contains the information that VSI Digital Test Manager needs to run a particular test. A test description requires a template file and can have other optionally specified test-related entities. A template file is an OpenVMS command procedure that executes a noninteractive test, or a session file containing a recorded interactive terminal or DECwindows session. A benchmark file contains the expected output for the test's execution.
- A collection - a set of tests selected for execution. You can execute a test only in the context of a collection. You can select tests for inclusion in a collection by test name or groups.
Tests
DTM tests are characterized by test descriptions. A DTM test description consists of the following elements:
- Test name: identifies the test description.
- Test prologue: identifies a DCL command file that runs immediately before the template file. You use a prologue file to set up any special environment that the test requires. Output from a prologue file does not appear in the test results.
- Test epilogue: identifies a DCL command file that runs immediately after the template file. You use an epilogue file to clean up operations, or to apply user-created filters to the result file. Unlike the prologue file, the epilogue file can directly alter the test results.
- Template: identifies a DCL command file for an on interactive test, or the session file for an interactive terminal or DECwindows test. This field defaults to test-name.SESSION for an interactive terminal or DECwindows test and test-name.COM for noninteractive tests.
- Benchmark: identifies a file that contains the expected test output. It is the standard against which VSI Digital Test Manager compares the results of a test run. This field defaults to test-name.BMK.
- Variables: identifies the variables and associated values used with the template, prologue, or epilogue files for this test.
- Groups: identifies the groups to which the test description belongs.
- Test type: identifies the test as either an interactive terminal, DECwindows, or noninteractive test.
- Command: Identifies a DCL command to be spawned when a DECwindows test is recorded or executed. This command can be used to invoke applications for inclusion in the test.
- Comparison type: identifies the comparison type: screen, record, or character.
- Filters: identifies one or more filters to remove run-timedependent information from the result file.
- Remark: identifies a comment that you add to the history.
Collections
A collection is a snapshot of specified test descriptions and the VSI Digital Test Manager library as they exist at the time you create the collection. You must organize tests into collections before you can execute them to produce result files for comparison.
When you execute a collection, VSI Digital Test Manager sets up the test environment and executes all the tests in the collection (this can be done in batch mode). Each test in a collection generates a separate result file. The result file contains the output generated by the template file. The result file is used for comparison against a test's benchmark file.
When you execute a collection, VSI Digital Test Manager compares the test results with the benchmark file for each test that has been run. If the comparison is unsuccessful (differences are detected), VSI Digital Test Manager creates a difference file. If the comparison is successful (no differences are detected), the result file is deleted and no difference file is created.
Instructions
Before any DTM tests can be created, a DTM library needs to be created first. To do that, you create an empty directory and then run the CREATE LIBRARY command on this directory. If you already have a library, you can execute SET LIBRARY. When the library is set, it will be pointed to by the DTM$LIB logical.
Tests are created with a CREATE TEST_DESCRIPTION command. You can specify an existing template (.COM file) and benchmark (.BMK) file or RECORD an interactive test. To execute the tests, add them to a collection with CREATE COLLECTION, and then RUN that collection.
You can then review the differences using the Review subsystem, which gives you access to the results obtained by executing the collection, as well as other information about the collection. It also enables you to invoke the Performance and Coverage Analyzer (PCA) to gather performance and coverage data for the test. You must execute and compare a collection before it can be reviewed.
You can specify an initialization file to be executed when you run DTM either by defining the logical name DTM$INIT to it or with the /INIT qualifier when you run DTM.
You can execute DTM commands from a file using @filespec inside of DTM.
