Writing Testsuites
Pike includes a set of scripts to generate and run testsuites. While the primary purpose of these scripts is to test the Pike interpreter itself, you can easily use them to test your own code.This document describes the tools, as well as the syntax for writing tests.1 Testing toolsThe two tools that are included with Pike are called mktestsuite and test_pike.mktestsuiteThe mktestsuite script is a simple shell script that uses M4 to convert a testsuite input file into a testsuite that can be run by test_pike.pike. This script is found in $PIKE/include/pike.Usage:
mktestsuite testsuite.in > testsuite
The input file is simply a series of test invocations; any code outside of a test invocation will be ignored and will not appear in the final testsuite file.test_pikeThe most important part of the regression testing infrastructure is test_pike, which is the pike script that performs the actual testing. Included in $PIKE/include/pike for Pike releases before 7.7, and as the builtin tool pike -x test_pike in releases 7.7 and higher, test_pike includes a large number of options for performing testing.Usage:
pike test_pike testuitefile
test_pike also accepts a number of arguments, which you can review by passing the "--help"" argument to the script.1 Writing testsuitesThe easiest way to generate a testsuite that can be run by test_pike is to use the mktestsuite script. mktestsuite uses M4 to perform macro substitution on your input test file. By using this preprocessor, you have access to a number of test types, which are listed below.The format of a testsuite that uses this approach simply consists of a set of tests, listed in the order they are to be run.A Simple Example:
test_true(1)
test_false(0)
// this next test will fail
test_false(-1)
Running our testsuite:
/usr/local/pike/7.6.50/include/pike/mktestsuite tests.in > tests
/usr/local/pike/7.6.50/include/pike/test_pike.pike tests
Doing tests in tests (3 tests)
tests.in:4: Test 3 (shift 0) (CRNL) failed.
1: mixed a() { return -1; }
2:o->a(): -1Failed tests: 1.
Total tests: 3 (0 tests skipped)
Many options are available from within the test harness, be sure to check them out by using the "--help" option.1 Available test typesIn general, there are two forms of each type of test; one takes as its argument a set of statements that when run, will be placed within a function body and run. The other, denoted with a _any suffix, accepts as its argument(s) any block of Pike code, such as class definitions and multiple function definitions.test_anyUsage:
parameter a: a block of code (classes and functions) that return a valueparameter b: a value or expression does the return value of the block "a" equal the value b, as in ==?Example:
test_any([[
int f (int i) {i = 0; return i;};
return f (1);
]],0)
test_any_equalUsage:
parameter a: a block of code (classes and functions) that return a valueparameter b: a value or expressiondoes the return value of the block "a" equal the value b, as in equal()? useful for checking reference types when you want to check for the sameness of valuesExample:
test_any_equal([[
class Foo {int foo();};
return indices(Foo());
]], ({"foo"}))
test_eqUsage:
parameter a: a value or expressionparameter b: a value or expressionare a and b the same item, as in ==?Example:
test_eq('u117f', 0x117f)
test_equalUsage:
parameter a: a value or expressionparameter b: a value or expressionare a and b equal, as in equal()? useful for checking reference types when you want to check for the sameness of valuesExample:
test_equal(mtest_m,copy_value(mtest_m))
test_doUsage:
parameter a: a function bodydoes a run?Example:
test_do([[
void foo (int i) {
multiset res = (<>);
if (i) res = res;
};
foo (1);
]])
test_trueUsage:
parameter a: a value or expressiondoes a evaluate to true (ie not integer zero (0) )Example:
test_true("A" == upper_case("a"))
test_falseUsage:
parameter a: a value or expressiondoes a evaluate to false (ie an integer zero (0))Example:test_compileUsage:
parameter a: one or more statements to compile, these statements will be placed inside a function body.does the code compile?Example:
test_compile([[ Stdio.File foo=Stdio.File(); ]])
test_compile_anyUsage:
parameter a: a code block (functions and classes) to compiledoes the code compile?Example:
test_compile_any([[
void foo()
{
Stdio.File bar(int x, int y)
{
return 0;
};
}
]])
test_compile_errorUsage:
test_compile_error([[a]])
parameter a: one or more statements to compile (these statements will be placed inside a function body).does the code fail to compile?Example:
test_compile_error([[ string a="x"; int b; b="x"*17; ]])
test_compile_error_anyUsage:
test_compile_error_any([[a]])
parameter a: a code block (functions and classes) to compiledoes the code fail to compile?Example:
test_compile_error_any([[
mixed foo;
mapping query_variables() { return ([]); };
mixed foo(mixed bar) { return 1/foo; }
]])
test_compile_warningUsage:
test_compile_warning([[a]])
parameter a: one or more statements to compile (these statements will be placed inside a function body).does the code compile, generating a warning?Example:
test_compile_warning([[ int *a ]])
test_compile_warning_anyUsage:
test_compile_warning_any([[a]])
parameter a: a code block (functions and classes) to compiledoes the code compile, but generate warnings?Example:
test_compile_warning_any([[
#pragma strict_typesclass foo(string|int a)
{
void bar()
{
int b = a;
}
}object a = foo(1);]])
test_eval_errorUsage:
parameter a: one or more statements to compile (these statements will be placed inside a function body).does the code throw a runtime error when evaluated?Example:
test_eval_error([[ class Z { int destroy() { return 1/y; } }(); ]])
test_define_programUsage:
test_define_program(a, [[b]])
parameter a: a name for the program to be defined as
parameter b: a class bodydefine a class for use in later testsExample:
test_define_program(bar,[[ int a(){ return 1; } ]])
test_programUsage:
parameter a: a class definition which has an member function a() which returns an int. this method should return a "true" value for the test to succeed.Example:
// we use the program "bar", which was defined
//previously through the use of test_define_program
test_program([[ inherit bar; ]])
test_program_eqUsage:
test_program_eq([[a]], b)
parameter a: a program definition which has a member function "a" which returns a valueparameter b: a value or expressiondoes the evaluation of function "a()" in parameter a equal the value "b", as in ==?Example:
test_program_eq([[
constant x = X;
class X {constant c = "right";} constant y = Y;
class Y {constant c = "wrong";} string a()
{
return ::`[]("x")->c;
}
]], "right")
test_program_equal
test_program_equal([[a]], b)
parameter a: a program definition which has a member function "a" which returns a valueparameter b: a value or expressiondoes the evaluation of function "a()" in parameter a equal the value "b", as in equal()?Example:
test_program_eq([[
constant x = X;
class X {constant c = ({"right"});} constant y = Y;
class Y {constant c = ({"wrong"});} string a()
{
return ::`[]("x")->c;
}
]], ({"right"}))
other testsuite functions:test_testsUsage:
parameter a: a class definition which has a member function a, which returns an array with 2 integer elements: a[create 0] is the number of passed tests, and a[create 1] is the number of failures.perform a series of tests and return the resulting number of successes and failures.test_doUsage:
parameter a: an expression which will be evaluated.this "test" is useful for performing operations; as the test is merely run, but no result is checked.1 Other resources- http://pike.ida.liu.se/generated/manual/ref/chapter_19.html
Powered by PikeWiki2
|
|