Test Anything Protocol
From Wikipedia, the free encyclopedia
The Test Anything Protocol (TAP) is a general purpose format for transmitting the result of test programs to a thing which interprets and takes action on those results. Though it is language agnostic, it is primarily used by Perl modules.
TAP's general format is:
1..N ok 1 Description # Directive # Diagnostic .... ok 47 Description ok 48 Description more tests....
For example, a test file's output might look like:
1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet
TAP separates the test program from the thing which interprets the results using simple text pipelining. This allows one to run several tests isolated in different processes together in one harness. It allows one to write tests in different languages and run them together with unified results. It also allows the runner of the tests, rather than the author of the tests, to decide how to display the results. The results can be transformed by the runner into other formats (for example, XML) or even cause actions to happen (a smoke test server can send email on failure).
TAP is designed to be human readable. The simple "ok", "not ok" output and descriptions do not require a parser or even extensive knowledge of TAP for the basic intention of each line to be understood by the user.
TAP is designed to be portable. Its plain text format means it will work with any language or operating system.
TAP is designed to be streamable. The results can be interpreted a line at a time without having to wait for the test to complete.
Contents |
[edit] Specification
Despite its being almost 20 years old and in very wide use, it is only now being formally specified. This is to allow for adoption by other languages, the writing of additional parsers, and extension of TAP. The current spec is the TAP module on CPAN. This spec is under development and incomplete. Where the spec is lacking, the behavior of Test::Harness with the existing Perl core tests and, to a lesser extent, CPAN module tests, should be used as a guide.
A formal grammar is in development.
[edit] History
TAP came into being with the first version of Perl (1987) although it wasn't called anything then. For most of its existence the only TAP parser was Perl's core test harness (t/TEST) written by Larry Wall. When Perl 5 introduced modules, Test::Harness was written by Tim Bunce and Andreas König allowing Perl module authors to take advantage of TAP. Testing has become very popular in Perl in the last few years and Perl authors working with other languages wishing to take their tools with them have begun to write TAP parsers and TAP-based testing libraries in other languages.
[edit] TAP Parsers
These are libraries which parse TAP and display the results.
- Test::Harness is the oldest and most complete TAP parser. It is limited in how it displays TAP. Though it most often runs tests written in Perl, it can launch any process which generates TAP. Most of the TAP spec is taken from the behavior of Test::Harness.
- The t/TEST parser contained in the Perl source code.
- TAP::Parser is a new and more flexible parser being written by Curtis "Ovid" Poe, Andy Armstrong and other people. It was formerly called TAPx::Parser.
- Test::Run is a fork of Test::Harness being written by Shlomi Fish.
- TAP::Harness was supposed to be a rewrite of Test::Harness by Michael G. Schwern. It has not been materialized and TAP::Harness eventually became the name for a harness based on TAP::Parser.
[edit] TAP Producers
These are libraries for writing tests which output TAP.
- Test::More is the most popular testing module for Perl.
- libtap is a TAP producer written in C.
- Test.Simple is a port of the Perl Test::Simple and Test::More modules to JavaScript by David Wheeler.
- test-harness.php A TAP parser for PHP.
- PyTAP A beginning TAP implementation for Python.
[edit] Other Resources
- testanything.org is a site dedicated to the discussion, development and promotion of TAP.
