This document describes the tools, as well as the syntax for writing tests.

1 Testing tools

The two tools that are included with Pike are called mktestsuite and test_pike.

mktestsuite

The mktestsuite script is a simple shell script that uses M4 to convert a testsuite input file into a testsuite that can be run by test_pike.pike. This script is found in $PIKE/include/pike.

Usage:

mktestsuite testsuite.in > testsuite

The input file is simply a series of test invocations; any code outside of a test invocation will be ignored and will not appear in the final testsuite file.

test_pike

The most important part of the regression testing infrastructure is test_pike, which is the pike script that performs the actual testing. Included in $PIKE/include/pike for Pike releases before 7.7, and as the builtin tool pike -x test_pike in releases 7.7 and higher, test_pike includes a large number of options for performing testing.

Usage:

  pike test_pike testuitefile

test_pike also accepts a number of arguments, which you can review by passing the "--help"" argument to the script.

1 Writing testsuites

The easiest way to generate a testsuite that can be run by test_pike is to use the mktestsuite script. mktestsuite uses M4 to perform macro substitution on your input test file. By using this preprocessor, you have access to a number of test types, which are listed below.

The format of a testsuite that uses this approach simply consists of a set of tests, listed in the order they are to be run.

A Simple Example:

test_true(1)
test_false(0)
// this next test will fail
test_false(-1)

Running our testsuite:

/usr/local/pike/7.6.50/include/pike/mktestsuite tests.in > tests
/usr/local/pike/7.6.50/include/pike/test_pike.pike tests
Doing tests in tests (3 tests)
tests.in:4: Test 3 (shift 0) (CRNL) failed.
  1: mixed a() { return -1; }
  2:
o->a(): -1
Failed tests: 1.
Total tests: 3  (0 tests skipped)

Many options are available from within the test harness, be sure to check them out by using the "--help" option.

1 Available test types

In general, there are two forms of each type of test; one takes as its argument a set of statements that when run, will be placed within a function body and run. The other, denoted with a _any suffix, accepts as its argument(s) any block of Pike code, such as class definitions and multiple function definitions.

test_any

Usage:

test_any([[a]], b)

parameter a: a block of code (classes and functions) that return a value

parameter b: a value or expression

does the return value of the block "a" equal the value b, as in ==?

Example:

test_any([[
  int f (int i) {i = 0; return i;};
  return f (1);
]],0)

test_any_equal

Usage:

test_any_equal([[a]], b)

parameter a: a block of code (classes and functions) that return a value

parameter b: a value or expression

does the return value of the block "a" equal the value b, as in equal()? useful for checking reference types when you want to check for the sameness of values

Example:

test_any_equal([[
  class Foo {int foo();};
  return indices(Foo());
]], ({"foo"}))

test_eq

Usage:

test_eq(a, b)

parameter a: a value or expression

parameter b: a value or expression

are a and b the same item, as in ==?

Example:

test_eq('u117f', 0x117f)

test_equal

Usage:

test_equal(a, b)

parameter a: a value or expression

parameter b: a value or expression

are a and b equal, as in equal()? useful for checking reference types when you want to check for the sameness of values

Example:

test_equal(mtest_m,copy_value(mtest_m))

test_do

Usage:

test_do(a)

parameter a: a function body

does a run?

Example:

test_do([[
  void foo (int i) {
    multiset res = (<>);
    if (i) res = res;
  };
  foo (1);
]])

test_true

Usage:

test_true(a)

parameter a: a value or expression

does a evaluate to true (ie not integer zero (0) )

Example:

test_true("A" == upper_case("a"))

test_false

Usage:

test_false(a)

parameter a: a value or expression

does a evaluate to false (ie an integer zero (0))

Example:

test_false("A" == "a")

test_compile

Usage:

test_compile([[a]])

parameter a: one or more statements to compile, these statements will be placed inside a function body.

does the code compile?

Example:

test_compile([[ Stdio.File foo=Stdio.File(); ]])

test_compile_any

Usage:

test_compile_any([[a]])

parameter a: a code block (functions and classes) to compile

does the code compile?

Example:

test_compile_any([[
  void foo()
  {
    Stdio.File bar(int x, int y)
    {
      return 0;
    };
  }
]])

test_compile_error

Usage:

test_compile_error([[a]])

parameter a: one or more statements to compile (these statements will be placed inside a function body).

does the code fail to compile?

Example:

test_compile_error([[ string a="x"; int b; b="x"*17; ]])

test_compile_error_any

Usage:

test_compile_error_any([[a]])

parameter a: a code block (functions and classes) to compile

does the code fail to compile?

Example:

test_compile_error_any([[
 mixed foo;
 mapping query_variables() { return ([]); };
 mixed foo(mixed bar) { return 1/foo; }
]])

test_compile_warning

Usage:

test_compile_warning([[a]])

parameter a: one or more statements to compile (these statements will be placed inside a function body).

does the code compile, generating a warning?

Example:

test_compile_warning([[ int *a ]])

test_compile_warning_any

Usage:

test_compile_warning_any([[a]])

parameter a: a code block (functions and classes) to compile

does the code compile, but generate warnings?

Example:

test_compile_warning_any([[
#pragma strict_types
class foo(string|int a)
{
  void bar()
  {
    int b = a; 
  }
}
object a = foo(1);
]])

test_eval_error

Usage:

test_eval_error([[a]])

parameter a: one or more statements to compile (these statements will be placed inside a function body).

does the code throw a runtime error when evaluated?

Example:

test_eval_error([[ class Z { int destroy() { return 1/y; } }(); ]])

test_define_program

Usage:

test_define_program(a, [[b]])

parameter a: a name for the program to be defined as parameter b: a class body

define a class for use in later tests

Example:

test_define_program(bar,[[ int a(){ return 1; } ]])

test_program

Usage:

test_program([[a]])

parameter a: a class definition which has an member function a() which returns an int. this method should return a "true" value for the test to succeed.

Example:

// we use the program "bar", which was defined 
//previously through the use of test_define_program
test_program([[ inherit bar; ]])

test_program_eq

Usage:

test_program_eq([[a]], b)

parameter a: a program definition which has a member function "a" which returns a value

parameter b: a value or expression

does the evaluation of function "a()" in parameter a equal the value "b", as in ==?

Example:

test_program_eq([[
  constant x = X;
  class X {constant c = "right";}
  constant y = Y;
  class Y {constant c = "wrong";}
  string a()
  {
    return ::`[]("x")->c;
  }
]], "right")

test_program_equal

test_program_equal([[a]], b)

parameter a: a program definition which has a member function "a" which returns a value

parameter b: a value or expression

does the evaluation of function "a()" in parameter a equal the value "b", as in equal()?

Example:

test_program_eq([[
  constant x = X;
  class X {constant c = ({"right"});}
  constant y = Y;
  class Y {constant c = ({"wrong"});}
  string a()
  {
    return ::`[]("x")->c;
  }
]], ({"right"}))

other testsuite functions:

test_tests

Usage:

test_tests([[a]])

parameter a: a class definition which has a member function a, which returns an array with 2 integer elements: a[create 0] is the number of passed tests, and a[create 1] is the number of failures.

perform a series of tests and return the resulting number of successes and failures.

test_do

Usage:

test_do(a)

parameter a: an expression which will be evaluated.

this "test" is useful for performing operations; as the test is merely run, but no result is checked.

1 Other resources

- Pike Test Suite, from the Pike Reference Manual

Powered by PikeWiki2