Hash :
c96b881f
Author :
Date :
2011-06-03T22:50:56
parallel-tests: allow each test to have multiple results With this change, we improve the code creating the `test-suite.log' global log and the console testsuite summary to make it able to grasp multiple results per test script. This is required in order to introduce the planned support for test protocols, like TAP and SubUnit, which can indeed run multiple tests per test script, each with its individual result. The implementation makes use of a custom reStructuredText field `:test-result:'. Note that no new documentation is added by this change; that is be left for follow-up changes. * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files, recognize a report of a test's result only if it is declared with the custom `:test-result:' reStructuredText field placed at the beginning of a line. Extend and add explanatory comments. (recheck, recheck-html): Add explanatory comments. * lib/test-driver: Write an appropriate reStructuredText field `:test-result:' in the generated log file. Use a reStructuredText transition to better separate the test outcome report from the test script's registered output. Improve comments. * tests/test-driver-custom-xfail-tests.test: Adjust. * tests/parallel-tests7.test: Adjust. * tests/parallel-tests-empty-testlogs.test: New test. * tests/parallel-tests-recheck-override.test: Likewise. * tests/parallel-tests2.test: Extend and keep more in-sync with ... * tests/test-driver-custom-html.test: ... this new related test. * tests/test-driver-custom-no-html.test: New test. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise. * tests/trivial-test-driver: New file, used by the last four tests above. * tests/Makefile.am (TESTS): Update. (EXTRA_DIST): Distribute `trivial-test-driver'. (test-driver-custom-multitest.log): Depend on `trivial-test-driver'. (test-driver-custom-multitest-recheck.log): Likewise. (test-driver-custom-multitest-recheck2.log): Likewise. (test-driver-custom-html.log): Likewise.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
#! /bin/sh
# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that auxiliary script 'test-driver' doesn't get needlessly
# installed or referenced if it's not used, i.e., if the user has
# defined his own `*LOG_DRIVER' variables.
parallel_tests=yes
. ./defs || Exit 1
cat >> configure.in <<'END'
AC_PROG_CC
AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
AC_SUBST([X_LOG_DRIVER], [none])
AC_OUTPUT
END
mkdir sub1 sub2
cat > Makefile.am <<'END'
SUBDIRS = sub1 sub2
LOG_DRIVER =
TEST_LOG_DRIVER = :
TESTS = foo bar.test
END
cat > sub1/Makefile.am <<'END'
TEST_EXTENSIONS = .x .sh .pl
SH_LOG_DRIVER = dummy1
PL_LOG_DRIVER = dummy2
TESTS = a.pl b.sh c.x
END
cat > sub2/Makefile.am <<'END'
TEST_EXTENSIONS = .bar
LOG_DRIVER = x
BAR_LOG_DRIVER = y
TESTS = 1 2.bar 3.test 4.t 5.tt $(check_PROGRAMS)
check_PROGRAMS = p1 p2$(EXEEXT) p3.bar p4.suf
END
$ACLOCAL
for opts in '' '--add-missing' '-a -c'; do
$AUTOMAKE $opts
$FGREP test-driver Makefile.in sub[12]/Makefile.in && Exit 1
find . | $FGREP test-driver && Exit 1
done
: