kmx git

Commit	Date	Message
db973eaa	2012-02-21T19:47:36	tests: prefer 'configure.ac' over 'configure.in' With this change, our testsuite now uses 'configure.ac' as the name for the typical autoconf input, instead of the obsolescent 'configure.in' (which has been deprecated for several years, at least since autoconf 2.50). Most of the test cases changed by this commit have been automatically modified with this sed command (using GNU sed): sed -i 's/\<configure\(\\\?\)\.in\>/configure\1.ac/g' * syntax-checks.mk (sc_tests_no_configure_in): New syntax check. (syntax_check_rules): Add it. * tests/defs: Create stub for autoconf input in 'configure.ac', not in 'configure.in'. Adjust comments. ($AUTOCONF, $AUTOHEADER, $AUTORECONF): Remove workaround for the infamous debian autoconf wrapper: we don't need such workaround anymore now that we name our autoconf input as 'configure.ac'. For more information, see commit v1.11-564-g63da492 "test defs: hack to support autoconf-wrapper programs" of 16-12-2011. * tests/README: Use 'configure.ac', not 'configure.in'. * Many many tests: Likewise. * tests/backcompat3.test: Adjust to avoid spurious failures. * tests/backcompat5.test: Likewise. * tests/missing6.test: Likewise. * tests/backcompat6.test: Likewise, and extend a bit since we are at it.
641a5a4b	2012-02-16T10:46:23	maint: run "make update-copyright"
e00d6c98	2011-09-10T21:59:09	tests defs: more namespace-safeness ($using_tap): Renamed ... ($am_using_tap): ... to this. ($parallel_tests): Renamed ... ($am_parallel_tests): ... to this. ($test_prefer_config_shell): Renamed ... ($am_test_prefer_config_shell): ... to this. ($original_AUTOMAKE): Renamed ... ($am_original_AUTOMAKE): ... to this. ($original_ACLOCAL): Renamed ... ($am_original_ACLOCAL): ... to this. * tests/defs, tests/defs-static.in, tests/gen-testsuite-part, tests/README, and all callers: Adjusted. * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Adjust, and rewrite to make future adjustments easier. * tests/defs (funcs_file_): Renamed ... (am_funcs_file): ... to this. * Makefile.am (sc_tests_obsolete_variables): New maintainer check. (syntax_check_rules): Add it.
dffe171f	2011-08-03T15:46:02	Merge branch 'test-protocols' into tap-testsuite-work
0c81b43f	2011-07-28T12:25:22	test harness: use new `.trs' files to hold test metadata With this change, the test harness will keep test metadata in dedicated `.trs' files, instead of having them embedded into the `.log' files. This allows for easier forward-compatibility and extension of test metadata, and for more flexibility in the format of the `.log' files. Note that this change makes the `:end-metadata:' field obsolete. * doc/automake.texi (Parallel Test Harness, Log files generation and test results recording): Document the new APIs and behaviour; some related minor rewordings and fixlets. * NEWS: Update. * automake.in (handle_tests): When bringing in the content of `check2.am', substitute %BASE% with the basename of the `.log' file being created by a rule. Add the generated `.trs' files to the list of files to be cleaned by "make mostlyclean". * lib/am/check.am (am__test_driver_flags): Rename ... (am__common_driver_flags): ... to this, and remove the flags `--test-name' and `--log-file' from it: they are now define in the proper rules in `check2.am'. (am__TEST_BASES): New internal variable, holding the names of the tests, with any registered extension removed. (am__stealth_MAKE): New internal variable, can be used instead of $(MAKE) in recipes requiring a recursive call to make, but which are not intended to be executed by "make -n". (.log.trs): New suffix rule, to recover from deletion of `.trs' files. ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the new API of "test logs in `.log' files, test metadata in `.trs' files". It goes to some length to work correctly in face of unreadable or missing `.log' and `.trs' files, and to error out with proper error messages when this is not possible. [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale" `.trs' files (in addition to `.log files) before remaking the $(TEST_SUITE_LOG). (recheck, recheck-html): Look for the `:recheck:' field in the `.trs' files, not in the `.log' files. * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust the call to the test driver, in particularly passing the new option `--trs-file'. [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise. * lib/tap-driver ($USAGE): Adjust the help screen. (Getopt::Long::GetOptions): Handle the `--trs-file' option, through the use of ... ($trs_file): ... this new global variable. (finish): Write metadata for the test run to `$trs_file' rather then to `$log_file', through the use of ... (write_test_results): ... this new function. * lib/test-driver (print_usage): Update the help screen. (Option parsing): Handle the `--trs-file' option, through the use of ... ($resfile): ... this new global variable. (Main code): Write metadata for the test run to `$trsfile' rather than to `$logfile'. Minor related adjustments to comments. * tests/.gitignore: Ignore `.trs' files. tests/parallel-tests-unreadable-log.test: Moved ... * tests/parallel-tests-unreadable.test: ... to this, and extended to also check the semantics for unreadable `.trs' files. * tests/test-driver-end-metadata.test: Deleted as obsolete. * tests/test-driver-metadata-no-leading-space.test: Likewise. * tests/test-driver-global-log.test: Renamed ... * tests/test-metadata-global-log.test: ... to this, and modified as to verify the new APIs and semantics. * tests/test-driver-recheck: Renamed ... * tests/test-metadata-recheck.test: ... to this, and modified likewise. * tests/parallel-tests-once.test: New test. * tests/parallel-tests-make-n.test: Likewise. * test-metadata-results.test: Likewise. * test-missing.test: Likewise. * test-missing2.test: Likewise. * test-trs-basic.test: Likewise. * test-trs-recover.test: Likewise. * test-trs-recover2.test: Likewise. * tests/Makefile.am (TESTS): Update.
1bb5974d	2011-07-04T18:12:07	test defs: new auxiliary function 'count_test_results' * tests/defs (count_test_results): New function. * tests/check11.test: Use it. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise, and slightly improve debugging output. * tests/parallel-tests.test: Make use of `count_test_results'. Also, make grepping of "make check" output slightly stricter * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-log-override-2.test: Likewise, and throw in a small optimization.
eaac33fe	2011-07-01T11:12:31	parallel-tests: simplify testsuite summary Prefer a more deterministic, "tabular" format for the testsuite summary, always listing the numbers of passed, failed, xfailed, xpassed, skipped and errored tests, even when these numbers are zero. This simplify the logic of testsuite summary creation, makes it more easily machine-parseable, and will probably allow for easier addition of new kinds of test results in the future. * lib/am/check.am (am__tty_colors_dummy): New make variable, to reduce code duplication. Extracted from previous versions of $(am__tty_colors), and extended by defining two new variables `$mgn' and `$brg'. [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable. (am__text_box): Delete, is not needed anymore. ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the new testsuite summary format. * NEWS: Update. * tests/check10.test: Don't run with the parallel-tests harness too, that makes no sense anymore. * tests/color.test: Update and adjust. * tests/color2.test: Likewise. * tests/parallel-tests.test: Likewise. * tests/parallel-tests3.test: Likewise. * tests/parallel-tests6.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-unreadable-log.test: Likewise. * tests/parallel-tests-empty-testlogs.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-end-test-results.test: Likewise. * tests/parallel-tests-no-color-in-log.test: New test. * tests/testsuite-summary-color.test: Likewise. * tests/testsuite-summary-count.test: Likewise. * tests/testsuite-summary-count-many.test: Likewise. * tests/testsuite-summary-reference-log.test: Likewise. * tests/testsuite-summary-checks.sh: New auxiliary script, used by the new tests above. * tests/extract-testsuite-summary: Likewise. * tests/trivial-test-driver: Optimize for speed when there are lots of of tests. * tests/Makefile.am (EXTRA_DIST): Distribute them. (testsuite-summary-color.log, testsuite-summary-count.log): Depend on them. (testsuite-summary-count-many.log): Depend on the auxiliary scripts 'trivial-test-driver' and 'extract-testsuite-summary'. (TESTS): Update.
ae610ad3	2011-05-23T14:26:58	tests: tweak and improve tests on "Simple Tests" driver * tests/check.test: Make grepping of generated Makefile.in slightly stricter. Add trailing `:' command. * tests/check2.test: Ensure verbose printing of captured make output. Make grepping of generated Makefile.in and of make output stricter. Add trailing `:' command. * tests/check3.test: Likewise. Also, fix minor error in a comment. * tests/check4.test: Stricter grepping of make output. Use `using_gmake' subroutine instead of copy&paste from defs.in to determine whether $MAKE is GNU make. Add a trailing `:' command. * tests/check6.test: Add trailing `:' command. * tests/check10.test: Prefer redirection `>' over `>>' when creating new files. Ensure verbose printing of captured make output. * tests/check11.test: Likewise. * tests/check7.test: Likewise. Make grepping of make output stricter. Prefer "make distcheck" over a simple "make check" in a VPATH build. Add trailing `:' command. (a.c): Format following GNU coding standards. (Makefile.am): Explicitly declare target `print-xfail-tests' as PHONY. Distribute $(check_SCRIPTS), for distcheck. * tests/parallel-tests.test: Prefer redirection `>' over `>>' when creating new files. * tests/parallel-tests2.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests8.test: Likewise. Make grepping of automake stderr stricter. * tests/check8.test: Likewise. Make formatting of created .c files more consistent with GNU Coding Standards. tests/check-subst-prog.test (configure.in): Break overly long lines. (pass-prog.c, xfail-prog.c): Format them more consistently with GNU Coding Standards. * tests/parallel-tests3.test: Use a shorter and clearer skip message. Use the `unindent' subroutine to improve readability of indented loop bodies. Simplify the hack used to please maintainer-check. * tests/parallel-tests5.test: Make formatting of created .c files more consistent with GNU Coding Standards. Use the `unindent' subroutine to improve readability of indented loop bodies. tests/parallel-tests10.test: Add blank line, for clarity. Prefer trailing `:' over trailing `Exit 0', for consistency with other tests.
79103003	2011-01-08T19:40:04	tests: enable 'errexit' shell flag by default. * tests/defs: Enable `errexit' shell flag (near the end). Removed redundant comment about the enabling of shell traces. * tests/README (Writing test cases): Update, and use nicer formatting in a couple of places. * All tests: Adjusted by removing now-redundant calls to 'set -e'.
03449117	2010-03-04T20:22:17	Avoid generation of `tests/defs-p' file * tests/defs.in: In the generated `configure.in' snippet: call `AM_INIT_AUTOMAKE' with the `parallel-tests' option if the shell variable `parallel_tests' is set to `yes'. * tests/Makefile.am (defs-p): Target removed. (check_SCRIPTS): Removed `defs-p'. (clean-local-check): Do not unlink `defs-p' anymore. ($(parallel_tests)): Transformation rules for the test scripts adjusted. * tests/gen-parallel-tests: Selection rules for the test scripts adjusted. * tests/parallel-tests.test: Set `$parallel_tests' to `yes' then include `./defs' (rather than simply including `./defs-p'). * tests/parallel-tests2.test: Likewise. * tests/parallel-tests3.test: Likewise. * tests/parallel-tests4.test: Likewise. * tests/parallel-tests5.test: Likewise. * tests/parallel-tests6.test: Likewise. * tests/parallel-tests7.test: Likewise. * tests/parallel-tests8.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests10.test: Likewise. * tests/README (Section "Writing Test Cases", subsection "Do"): Adjusted the parts referring to tests checking `parallel-tests' behaviour. Some other minor related improvements. * tests/.gitignore (defs-p): Removed.
fcf2f560	2009-05-17T15:32:40	Revert Automake license to GPLv2+. Automake will move to GPLv3+ once the Exception statement has been rewritten to use the new GPLv3 exception language. This change does not impact the COPYING file that may be installed by `automake --add-missing'. * COPYING: Revert to GPLv2. All uses changed. * NEWS: Update. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
68ce3ae9	2009-04-25T10:43:09	parallel-tests: fix recheck* targets for non-GNU make. * lib/am/check.am (recheck recheck-html): Override AM_MAKEFLAGS to portably transport the TEST_LOGS settings through the recursive `make' invocations. Fixes parallel-tests9.test failure with Solaris make. * tests/parallel-tests9.test: Also ensure that the test summary is correct for `recheck'. Suggestion by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
4e9c77ba	2009-04-14T23:42:14	testsuite: do not fail on systems without read permissions. * tests/instfail-info.test: Do not use the `non-root' requirement for testing whether files may be made unreadable; instead use `test -r' and skip the test if that still works. * tests/instfail-java.test: Likewise. * tests/instfail-libtool.test: Likewise. * tests/instfail.test: Likewise. * tests/instmany-mans.test: Likewise. * tests/instmany-python.test: Likewise. * tests/instmany.test: Likewise. * tests/parallel-tests9.test: Likewise. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
79f2c3e2	2009-04-10T13:26:16	parallel-tests: `clean recheck' should not rerun all tests. * lib/am/check.am (recheck, recheck-html): Do not rerun all tests if `$(TEST_SUITE_LOG)' does not exist. * tests/parallel-tests2.test: Adjust recheck test. * tests/parallel-tests9.test: Adjust recheck-html test. * NEWS: Update. Suggestion by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
68b7dc1d	2009-04-06T07:03:16	parallel-tests: redo check-html, recheck, recheck-html. * lib/am/check.am (recheck, recheck-am): Remove. (recheck-TESTS): Rename to ... (recheck): ... this and rewrite, factored ... (recheck-html): ... with this rule. Pass TEST_LOGS rather than RECHECK_LOGS to `check' and `check-html', respectively, to avoid running outdated tests. Invoking the public macros ensures `check_SCRIPTS' etc. are created in time. Do not output errors for tests that were not run yet. If the testsuite has not run at all, run all tests. (check-html): Run `check' target, not `check-TESTS', to ensure `check_SCRIPTS' etc. are created in time. (.PHONY, .MAKE, AM_RECURSIVE_TARGETS): Adjust contents. * tests/parallel-tests2.test: Expose the check-html and recheck-html issues. * tests/parallel-tests9.test: Expose the recheck issues. Bugs in previous version pointed out by Akim, who already had them fixed in his original version. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
8f33eec4	2009-03-28T22:58:34	parallel-tests: redo lazy checking: recheck and RECHECK_LOGS. Replace the LAZY_TEST_SUITE API with a simpler yet more powerful one: RECHECK_LOGS specifies those tests which are to be removed in any case before testing. Provide a `recheck' convenience target to set RECHECK_LOGS to all failed and unexpectedly passed tests. Document several ways to limit the set of tests run. * lib/am/check.am [PARALLEL_TESTS] (RECHECK_LOGS): New variable, default to $(TESTS_LOGS). (check-TESTS): Remove $(RECHECK_LOGS) not $(TEST_LOGS). Drop use of LAZY_TEST_SUITE. ($(TEST_SUITE_LOG)): Do not output note about lazy rerun, as LAZY_TEST_SUITE is gone. (recheck): New target. (recheck-am, recheck-TESTS): New internal targets. * doc/automake.texi (Tests): Update @vindex for TESTS and TEST_LOGS. Replace description of LAZY_TEST_SUITE with a list of ways the set of tests to be run can be modified. Document RECHECK_LOGS and the recheck target. * tests/defs.in: Unset RECHECK_LOGS not LAZY_TEST_SUITE. * tests/parallel-tests.test: Adjust, replacing LAZY_TEST_SUITE with corresponding RECHECK_LOGS settings, and add another RECHECK_LOGS test. * tests/parallel-tests9.test: New test, test `recheck'. * tests/Makefile.am: Update. Suggestion and different implementation by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

db973eaa

2012-02-21T19:47:36

tests: prefer 'configure.ac' over 'configure.in' With this change, our testsuite now uses 'configure.ac' as the name for the typical autoconf input, instead of the obsolescent 'configure.in' (which has been deprecated for several years, at least since autoconf 2.50). Most of the test cases changed by this commit have been automatically modified with this sed command (using GNU sed): sed -i 's/\<configure$\\\?$\.in\>/configure\1.ac/g' * syntax-checks.mk (sc_tests_no_configure_in): New syntax check. (syntax_check_rules): Add it. * tests/defs: Create stub for autoconf input in 'configure.ac', not in 'configure.in'. Adjust comments. ($AUTOCONF, $AUTOHEADER, $AUTORECONF): Remove workaround for the infamous debian autoconf wrapper: we don't need such workaround anymore now that we name our autoconf input as 'configure.ac'. For more information, see commit v1.11-564-g63da492 "test defs: hack to support autoconf-wrapper programs" of 16-12-2011. * tests/README: Use 'configure.ac', not 'configure.in'. * Many many tests: Likewise. * tests/backcompat3.test: Adjust to avoid spurious failures. * tests/backcompat5.test: Likewise. * tests/missing6.test: Likewise. * tests/backcompat6.test: Likewise, and extend a bit since we are at it.

641a5a4b

2012-02-16T10:46:23

maint: run "make update-copyright"

e00d6c98

2011-09-10T21:59:09

tests defs: more namespace-safeness ($using_tap): Renamed ... ($am_using_tap): ... to this. ($parallel_tests): Renamed ... ($am_parallel_tests): ... to this. ($test_prefer_config_shell): Renamed ... ($am_test_prefer_config_shell): ... to this. ($original_AUTOMAKE): Renamed ... ($am_original_AUTOMAKE): ... to this. ($original_ACLOCAL): Renamed ... ($am_original_ACLOCAL): ... to this. * tests/defs, tests/defs-static.in, tests/gen-testsuite-part, tests/README, and all callers: Adjusted. * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Adjust, and rewrite to make future adjustments easier. * tests/defs (funcs_file_): Renamed ... (am_funcs_file): ... to this. * Makefile.am (sc_tests_obsolete_variables): New maintainer check. (syntax_check_rules): Add it.

dffe171f

2011-08-03T15:46:02

Merge branch 'test-protocols' into tap-testsuite-work

0c81b43f

2011-07-28T12:25:22

test harness: use new `.trs' files to hold test metadata With this change, the test harness will keep test metadata in dedicated `.trs' files, instead of having them embedded into the `.log' files. This allows for easier forward-compatibility and extension of test metadata, and for more flexibility in the format of the `.log' files. Note that this change makes the `:end-metadata:' field obsolete. * doc/automake.texi (Parallel Test Harness, Log files generation and test results recording): Document the new APIs and behaviour; some related minor rewordings and fixlets. * NEWS: Update. * automake.in (handle_tests): When bringing in the content of `check2.am', substitute %BASE% with the basename of the `.log' file being created by a rule. Add the generated `.trs' files to the list of files to be cleaned by "make mostlyclean". * lib/am/check.am (am__test_driver_flags): Rename ... (am__common_driver_flags): ... to this, and remove the flags `--test-name' and `--log-file' from it: they are now define in the proper rules in `check2.am'. (am__TEST_BASES): New internal variable, holding the names of the tests, with any registered extension removed. (am__stealth_MAKE): New internal variable, can be used instead of $(MAKE) in recipes requiring a recursive call to make, but which are not intended to be executed by "make -n". (.log.trs): New suffix rule, to recover from deletion of `.trs' files. ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the new API of "test logs in `.log' files, test metadata in `.trs' files". It goes to some length to work correctly in face of unreadable or missing `.log' and `.trs' files, and to error out with proper error messages when this is not possible. [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale" `.trs' files (in addition to `.log files) before remaking the $(TEST_SUITE_LOG). (recheck, recheck-html): Look for the `:recheck:' field in the `.trs' files, not in the `.log' files. * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust the call to the test driver, in particularly passing the new option `--trs-file'. [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise. * lib/tap-driver ($USAGE): Adjust the help screen. (Getopt::Long::GetOptions): Handle the `--trs-file' option, through the use of ... ($trs_file): ... this new global variable. (finish): Write metadata for the test run to `$trs_file' rather then to `$log_file', through the use of ... (write_test_results): ... this new function. * lib/test-driver (print_usage): Update the help screen. (Option parsing): Handle the `--trs-file' option, through the use of ... ($resfile): ... this new global variable. (Main code): Write metadata for the test run to `$trsfile' rather than to `$logfile'. Minor related adjustments to comments. * tests/.gitignore: Ignore `*.trs' files. * tests/parallel-tests-unreadable-log.test: Moved ... * tests/parallel-tests-unreadable.test: ... to this, and extended to also check the semantics for unreadable `.trs' files. * tests/test-driver-end-metadata.test: Deleted as obsolete. * tests/test-driver-metadata-no-leading-space.test: Likewise. * tests/test-driver-global-log.test: Renamed ... * tests/test-metadata-global-log.test: ... to this, and modified as to verify the new APIs and semantics. * tests/test-driver-recheck: Renamed ... * tests/test-metadata-recheck.test: ... to this, and modified likewise. * tests/parallel-tests-once.test: New test. * tests/parallel-tests-make-n.test: Likewise. * test-metadata-results.test: Likewise. * test-missing.test: Likewise. * test-missing2.test: Likewise. * test-trs-basic.test: Likewise. * test-trs-recover.test: Likewise. * test-trs-recover2.test: Likewise. * tests/Makefile.am (TESTS): Update.

1bb5974d

2011-07-04T18:12:07

test defs: new auxiliary function 'count_test_results' * tests/defs (count_test_results): New function. * tests/check11.test: Use it. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-custom-multitest-recheck.test: Likewise. * tests/test-driver-custom-multitest-recheck2.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise, and slightly improve debugging output. * tests/parallel-tests.test: Make use of `count_test_results'. Also, make grepping of "make check" output slightly stricter * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-log-override-2.test: Likewise, and throw in a small optimization.

eaac33fe

2011-07-01T11:12:31

parallel-tests: simplify testsuite summary Prefer a more deterministic, "tabular" format for the testsuite summary, always listing the numbers of passed, failed, xfailed, xpassed, skipped and errored tests, even when these numbers are zero. This simplify the logic of testsuite summary creation, makes it more easily machine-parseable, and will probably allow for easier addition of new kinds of test results in the future. * lib/am/check.am (am__tty_colors_dummy): New make variable, to reduce code duplication. Extracted from previous versions of $(am__tty_colors), and extended by defining two new variables `$mgn' and `$brg'. [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable. (am__text_box): Delete, is not needed anymore. ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the new testsuite summary format. * NEWS: Update. * tests/check10.test: Don't run with the parallel-tests harness too, that makes no sense anymore. * tests/color.test: Update and adjust. * tests/color2.test: Likewise. * tests/parallel-tests.test: Likewise. * tests/parallel-tests3.test: Likewise. * tests/parallel-tests6.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests-unreadable-log.test: Likewise. * tests/parallel-tests-empty-testlogs.test: Likewise. * tests/parallel-tests-log-override-recheck.test: Likewise. * tests/parallel-tests-no-spurious-summary.test: Likewise. * tests/test-driver-custom-multitest.test: Likewise. * tests/test-driver-end-test-results.test: Likewise. * tests/parallel-tests-no-color-in-log.test: New test. * tests/testsuite-summary-color.test: Likewise. * tests/testsuite-summary-count.test: Likewise. * tests/testsuite-summary-count-many.test: Likewise. * tests/testsuite-summary-reference-log.test: Likewise. * tests/testsuite-summary-checks.sh: New auxiliary script, used by the new tests above. * tests/extract-testsuite-summary: Likewise. * tests/trivial-test-driver: Optimize for speed when there are lots of of tests. * tests/Makefile.am (EXTRA_DIST): Distribute them. (testsuite-summary-color.log, testsuite-summary-count.log): Depend on them. (testsuite-summary-count-many.log): Depend on the auxiliary scripts 'trivial-test-driver' and 'extract-testsuite-summary'. (TESTS): Update.

ae610ad3

2011-05-23T14:26:58

tests: tweak and improve tests on "Simple Tests" driver * tests/check.test: Make grepping of generated Makefile.in slightly stricter. Add trailing `:' command. * tests/check2.test: Ensure verbose printing of captured make output. Make grepping of generated Makefile.in and of make output stricter. Add trailing `:' command. * tests/check3.test: Likewise. Also, fix minor error in a comment. * tests/check4.test: Stricter grepping of make output. Use `using_gmake' subroutine instead of copy&paste from defs.in to determine whether $MAKE is GNU make. Add a trailing `:' command. * tests/check6.test: Add trailing `:' command. * tests/check10.test: Prefer redirection `>' over `>>' when creating new files. Ensure verbose printing of captured make output. * tests/check11.test: Likewise. * tests/check7.test: Likewise. Make grepping of make output stricter. Prefer "make distcheck" over a simple "make check" in a VPATH build. Add trailing `:' command. (a.c): Format following GNU coding standards. (Makefile.am): Explicitly declare target `print-xfail-tests' as PHONY. Distribute $(check_SCRIPTS), for distcheck. * tests/parallel-tests.test: Prefer redirection `>' over `>>' when creating new files. * tests/parallel-tests2.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests8.test: Likewise. Make grepping of automake stderr stricter. * tests/check8.test: Likewise. Make formatting of created *.c files more consistent with GNU Coding Standards. * tests/check-subst-prog.test (configure.in): Break overly long lines. (pass-prog.c, xfail-prog.c): Format them more consistently with GNU Coding Standards. * tests/parallel-tests3.test: Use a shorter and clearer skip message. Use the `unindent' subroutine to improve readability of indented loop bodies. Simplify the hack used to please maintainer-check. * tests/parallel-tests5.test: Make formatting of created *.c files more consistent with GNU Coding Standards. Use the `unindent' subroutine to improve readability of indented loop bodies. * tests/parallel-tests10.test: Add blank line, for clarity. Prefer trailing `:' over trailing `Exit 0', for consistency with other tests.

79103003

2011-01-08T19:40:04

tests: enable 'errexit' shell flag by default. * tests/defs: Enable `errexit' shell flag (near the end). Removed redundant comment about the enabling of shell traces. * tests/README (Writing test cases): Update, and use nicer formatting in a couple of places. * All tests: Adjusted by removing now-redundant calls to 'set -e'.

03449117

2010-03-04T20:22:17

Avoid generation of `tests/defs-p' file * tests/defs.in: In the generated `configure.in' snippet: call `AM_INIT_AUTOMAKE' with the `parallel-tests' option if the shell variable `parallel_tests' is set to `yes'. * tests/Makefile.am (defs-p): Target removed. (check_SCRIPTS): Removed `defs-p'. (clean-local-check): Do not unlink `defs-p' anymore. ($(parallel_tests)): Transformation rules for the test scripts adjusted. * tests/gen-parallel-tests: Selection rules for the test scripts adjusted. * tests/parallel-tests.test: Set `$parallel_tests' to `yes' then include `./defs' (rather than simply including `./defs-p'). * tests/parallel-tests2.test: Likewise. * tests/parallel-tests3.test: Likewise. * tests/parallel-tests4.test: Likewise. * tests/parallel-tests5.test: Likewise. * tests/parallel-tests6.test: Likewise. * tests/parallel-tests7.test: Likewise. * tests/parallel-tests8.test: Likewise. * tests/parallel-tests9.test: Likewise. * tests/parallel-tests10.test: Likewise. * tests/README (Section "Writing Test Cases", subsection "Do"): Adjusted the parts referring to tests checking `parallel-tests' behaviour. Some other minor related improvements. * tests/.gitignore (defs-p): Removed.

fcf2f560

2009-05-17T15:32:40

Revert Automake license to GPLv2+. Automake will move to GPLv3+ once the Exception statement has been rewritten to use the new GPLv3 exception language. This change does not impact the COPYING file that may be installed by `automake --add-missing'. * COPYING: Revert to GPLv2. All uses changed. * NEWS: Update. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

68ce3ae9

2009-04-25T10:43:09

parallel-tests: fix recheck* targets for non-GNU make. * lib/am/check.am (recheck recheck-html): Override AM_MAKEFLAGS to portably transport the TEST_LOGS settings through the recursive `make' invocations. Fixes parallel-tests9.test failure with Solaris make. * tests/parallel-tests9.test: Also ensure that the test summary is correct for `recheck'. Suggestion by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

4e9c77ba

2009-04-14T23:42:14

testsuite: do not fail on systems without read permissions. * tests/instfail-info.test: Do not use the `non-root' requirement for testing whether files may be made unreadable; instead use `test -r' and skip the test if that still works. * tests/instfail-java.test: Likewise. * tests/instfail-libtool.test: Likewise. * tests/instfail.test: Likewise. * tests/instmany-mans.test: Likewise. * tests/instmany-python.test: Likewise. * tests/instmany.test: Likewise. * tests/parallel-tests9.test: Likewise. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

79f2c3e2

2009-04-10T13:26:16

parallel-tests: `clean recheck' should not rerun all tests. * lib/am/check.am (recheck, recheck-html): Do not rerun all tests if `$(TEST_SUITE_LOG)' does not exist. * tests/parallel-tests2.test: Adjust recheck test. * tests/parallel-tests9.test: Adjust recheck-html test. * NEWS: Update. Suggestion by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

68b7dc1d

2009-04-06T07:03:16

parallel-tests: redo check-html, recheck, recheck-html. * lib/am/check.am (recheck, recheck-am): Remove. (recheck-TESTS): Rename to ... (recheck): ... this and rewrite, factored ... (recheck-html): ... with this rule. Pass TEST_LOGS rather than RECHECK_LOGS to `check' and `check-html', respectively, to avoid running outdated tests. Invoking the public macros ensures `check_SCRIPTS' etc. are created in time. Do not output errors for tests that were not run yet. If the testsuite has not run at all, run all tests. (check-html): Run `check' target, not `check-TESTS', to ensure `check_SCRIPTS' etc. are created in time. (.PHONY, .MAKE, AM_RECURSIVE_TARGETS): Adjust contents. * tests/parallel-tests2.test: Expose the check-html and recheck-html issues. * tests/parallel-tests9.test: Expose the recheck issues. Bugs in previous version pointed out by Akim, who already had them fixed in his original version. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

8f33eec4

2009-03-28T22:58:34

parallel-tests: redo lazy checking: recheck and RECHECK_LOGS. Replace the LAZY_TEST_SUITE API with a simpler yet more powerful one: RECHECK_LOGS specifies those tests which are to be removed in any case before testing. Provide a `recheck' convenience target to set RECHECK_LOGS to all failed and unexpectedly passed tests. Document several ways to limit the set of tests run. * lib/am/check.am [PARALLEL_TESTS] (RECHECK_LOGS): New variable, default to $(TESTS_LOGS). (check-TESTS): Remove $(RECHECK_LOGS) not $(TEST_LOGS). Drop use of LAZY_TEST_SUITE. ($(TEST_SUITE_LOG)): Do not output note about lazy rerun, as LAZY_TEST_SUITE is gone. (recheck): New target. (recheck-am, recheck-TESTS): New internal targets. * doc/automake.texi (Tests): Update @vindex for TESTS and TEST_LOGS. Replace description of LAZY_TEST_SUITE with a list of ways the set of tests to be run can be modified. Document RECHECK_LOGS and the recheck target. * tests/defs.in: Unset RECHECK_LOGS not LAZY_TEST_SUITE. * tests/parallel-tests.test: Adjust, replacing LAZY_TEST_SUITE with corresponding RECHECK_LOGS settings, and add another RECHECK_LOGS test. * tests/parallel-tests9.test: New test, test `recheck'. * tests/Makefile.am: Update. Suggestion and different implementation by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>

kc3-lang/automake/tests/parallel-tests9.test

tests/parallel-tests9.test

Log