lib/am/check2.am


Log

Author Commit Date CI Message
Stefano Lattarini e1606b32 2012-02-18T13:59:26 parallel-tests: fix another BSD parallel make issue When BSD make is run in parallel mode, it apparently strips any leading directory component from the automatic variable '$*' (of course, against what POSIX mandates). This was causing FreeBSD 9.0 make and NetBSD 5.1 make to spuriously fail with automake-generated test harnesses if subdir tests were present *and* make was being run in parallel mode. This issue affected also the Automake own testsuite. * lib/am/check2.am (am__set_b): New internal variable. (%OBJ%, %EXT%.log, %EXT%$(EXEEXT).log): Use it to work around the described BSD make issue. * tests/parallel-tests3.test: Enhanced to expose the bug. * tests/parallel-tests-subdir.test: Enhance a little, since we are at it. * NEWS: Update.
Stefano Lattarini 641a5a4b 2012-02-16T10:46:23 maint: run "make update-copyright"
Stefano Lattarini 254227b6 2012-01-05T13:41:13 parallel-tests: avoid trailing backslashes in make recipes The new testsuite-harness could generate recipes with a trailing backslash character (possibly followed by blank characters only), in the very common case where the user hadn't defined the special $(AM_TESTS_FD_REDIRECT) variable. This caused spurious syntax errors with at least older bash versions (e.g., bash 2.05b), and could be potentially unportable to other weaker shells. See automake bug#10436: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10436> and coreutils bug#10427: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10427#8> * lib/am/check2.am: Rework line breaks so that no backslash can be at the end of a line. * tests/parallel-tests-trailing-bslash.test: New test. * tests/list-of-tests.mk: Add it. Report and diagnosis by Paul Eggert.
Stefano Lattarini 0c81b43f 2011-07-28T12:25:22 test harness: use new `.trs' files to hold test metadata With this change, the test harness will keep test metadata in dedicated `.trs' files, instead of having them embedded into the `.log' files. This allows for easier forward-compatibility and extension of test metadata, and for more flexibility in the format of the `.log' files. Note that this change makes the `:end-metadata:' field obsolete. * doc/automake.texi (Parallel Test Harness, Log files generation and test results recording): Document the new APIs and behaviour; some related minor rewordings and fixlets. * NEWS: Update. * automake.in (handle_tests): When bringing in the content of `check2.am', substitute %BASE% with the basename of the `.log' file being created by a rule. Add the generated `.trs' files to the list of files to be cleaned by "make mostlyclean". * lib/am/check.am (am__test_driver_flags): Rename ... (am__common_driver_flags): ... to this, and remove the flags `--test-name' and `--log-file' from it: they are now define in the proper rules in `check2.am'. (am__TEST_BASES): New internal variable, holding the names of the tests, with any registered extension removed. (am__stealth_MAKE): New internal variable, can be used instead of $(MAKE) in recipes requiring a recursive call to make, but which are not intended to be executed by "make -n". (.log.trs): New suffix rule, to recover from deletion of `.trs' files. ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the new API of "test logs in `.log' files, test metadata in `.trs' files". It goes to some length to work correctly in face of unreadable or missing `.log' and `.trs' files, and to error out with proper error messages when this is not possible. [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale" `.trs' files (in addition to `.log files) before remaking the $(TEST_SUITE_LOG). (recheck, recheck-html): Look for the `:recheck:' field in the `.trs' files, not in the `.log' files. * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust the call to the test driver, in particularly passing the new option `--trs-file'. [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise. * lib/tap-driver ($USAGE): Adjust the help screen. (Getopt::Long::GetOptions): Handle the `--trs-file' option, through the use of ... ($trs_file): ... this new global variable. (finish): Write metadata for the test run to `$trs_file' rather then to `$log_file', through the use of ... (write_test_results): ... this new function. * lib/test-driver (print_usage): Update the help screen. (Option parsing): Handle the `--trs-file' option, through the use of ... ($resfile): ... this new global variable. (Main code): Write metadata for the test run to `$trsfile' rather than to `$logfile'. Minor related adjustments to comments. * tests/.gitignore: Ignore `*.trs' files. * tests/parallel-tests-unreadable-log.test: Moved ... * tests/parallel-tests-unreadable.test: ... to this, and extended to also check the semantics for unreadable `.trs' files. * tests/test-driver-end-metadata.test: Deleted as obsolete. * tests/test-driver-metadata-no-leading-space.test: Likewise. * tests/test-driver-global-log.test: Renamed ... * tests/test-metadata-global-log.test: ... to this, and modified as to verify the new APIs and semantics. * tests/test-driver-recheck: Renamed ... * tests/test-metadata-recheck.test: ... to this, and modified likewise. * tests/parallel-tests-once.test: New test. * tests/parallel-tests-make-n.test: Likewise. * test-metadata-results.test: Likewise. * test-missing.test: Likewise. * test-missing2.test: Likewise. * test-trs-basic.test: Likewise. * test-trs-recover.test: Likewise. * test-trs-recover2.test: Likewise. * tests/Makefile.am (TESTS): Update.
Stefano Lattarini b9e9d546 2011-07-24T16:47:24 simple tests: support developer-defined fd redirections Motivated by coreutils bug#8846, and related discussions: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846> <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html> In those threads it has been shown how problematic it can be to do portable file descriptor redirections in the testsuite when using the Automake testsuite harness. This change should remedy to that situation. * lib/am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Append redirection defined in $(AM_TESTS_FD_REDIRECT) to the command-line invocations of the test scripts. * lib/am/check.am [!%?PARALLEL_TESTS%] $(check-TESTS): Likewise. * NEWS: Update. * doc/automake.texi (Script-based Testsuites): Document the new feature. * tests/check-fd-redirect.test: New test. * tests/parallel-tests-fd-redirect.test: Likewise. * tests/parallel-tests-am_tests_environment.test: Remove checks about the use of redirections in AM_TESTS_ENVIRONMENT: they would check deprecated (if not undefined) behaviour now. Strengthen a couple of still valid checks, to keep the test more in sync with the documentation. Improve debugging information. * tests/Makefile.am (TESTS): Update.
Stefano Lattarini 79922343 2011-05-20T21:45:51 parallel-tests: allow custom driver scripts Allow suffix-based definition of custom "driver script" for the test scripts. These driver scripts will be responsible of launching the tests (or their corresponding $(LOG_COMPILER), if they have an associated one), interpreting and displaying the test results, and writing the `.log' files. This new API should allow easy and flexible use of different test protocols in the future; in particular, we plan to use it to implement TAP and SubUnit harnesses. Note that no new documentation is added by this change; that is be left for follow-up changes. * automake.in (handle_tests): Define default for $(LOG_DRIVER), and, for any registered test extension `<ext>', define defaults for $(<ext>_LOG_DRIVER). Substitute %DRIVER% using these new variables, instead of the old internal $(am__test_driver). When processing check2.am, also substitute %DRIVER_FLAGS%. Require auxiliary script `test-driver' only if no driver has been explicitly defined for the test script kinds. * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call. * tests/parallel-tests-no-extra-driver.test: New test. * tests/test-driver-custom.test: Likewise. * tests/test-driver-custom-xfail-tests.test: Likewise. * tests/test-driver-fail.test: Likewise. * tests/Makefile.am: Update. * NEWS: Update.
Stefano Lattarini a27c9c49 2011-05-10T11:30:05 parallel-tests: add auxiliary script 'test-driver', refactor This refactoring should cause no API of functionality change, and is meant only to simplify the future implementation of TAP and SubUnit testsuite drivers. More precisely, our roadmap is to move most of the "testsuite driving" features out of the Automake-generated Makefiles, and into external scripts with well-defined interfaces. This will allow the user to define its own personalized testsuite drivers, and will also offer us a framework upon which to implement our new TAP and SubUnit drivers, all in a very unobtrusive way and retaining an high degree of code reuse and backward-compatibility. * lib/test-driver: New auxiliary script. * lib/Makefile.am (dist_SCRIPT_DATA): Add it. * automake.in (handle_tests): Require the new auxiliary script `test-driver', and define a new internal makefile variable `$(am__test_driver)', used to call it. Perform new substitution on `DRIVER' when processing the `check2.am' file. * lib/check.am (am__tty_colors): Define new shell variable `$am__color_tests'. (am__rst_section): Removed, its role taken over by the new `test-driver' script. (am__test_driver_flags): New variable, contains the command line options passed to `test-driver'. (am__check_pre): Do not deal with temporary files and exit traps anymore, as the `test-driver' script takes care of that now. Define shell variable `$am__enable_hard_errors', used by `$(am__test_driver_flags)'. Reorder so that we don't need to save and restore the value of the `TERM' environment variable anymore. Other related adjustments. (am__check_post): Remove, as its role has been completely taken over by the `test-driver' script. * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Call the test script through the Automake substituted `%DRIVER%', and honor the command-line options in `$(am__test_driver_flags)'. Do not call the obsoleted `$(am__check_post)' anymore. * doc/automake.texi (Auxiliary Programs): Mention the new `test-driver' script. (Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR. Since we are at it, break the list of auxiliary scripts by placing one per line, to simplify potential future additions of new scripts. * tests/check.test: Adjust. * tests/check2.test : Likewise. * tests/check3.test : Likewise. * tests/check4.test : Likewise. * tests/check10.test: Likewise. * tests/color.test: Likewise. * tests/color2.test: Likewise. * tests/comment9.test: Likewise. * tests/dejagnu.test: Likewise. * tests/exeext4.test: Likewise. * tests/maken3.test: Likewise. * tests/maken4.test: Likewise. * tests/parallel-tests-interrupt.test: Likewise. * tests/posixsubst-tests.test: Likewise. * tests/repeated-options.test: Likewise. * tests/check-no-test-driver.test: New test. * tests/parallel-test-driver-install.test: Likewise. * tests/Makefile.am (TESTS): Update. * NEWS: Update.
Ralf Wildenhues fcf2f560 2009-05-17T15:32:40 Revert Automake license to GPLv2+. Automake will move to GPLv3+ once the Exception statement has been rewritten to use the new GPLv3 exception language. This change does not impact the COPYING file that may be installed by `automake --add-missing'. * COPYING: Revert to GPLv2. All uses changed. * NEWS: Update. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
Ralf Wildenhues b6bebfca 2009-05-03T09:59:21 Fix copyright years in *.am files. * lib/am/check.am, lib/am/check2.am, lib/am/distdir.am, lib/am/program.am, lib/am/tags.am: Fix copyright years. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
Ralf Wildenhues 1692795d 2009-04-09T00:16:35 parallel-tests: Fix driver for nonempty executable extension. * automake.in (handle_tests): New substitution `%am__EXEEXT%', defined as 'FALSE' for non-generic rules, or if no programs are built at all. Otherwise, define it as configure conditional. * lib/am/check2.am [%am__EXEEXT%] (%EXT%$(EXEEXT).log): New conditional generic rule. * m4/init.m4: Hook an m4_provide of `_AM_COMPILER_EXEEXT' onto Autoconf's `_AC_COMPILER_EXEEXT' macro. (AM_INIT_AUTOMAKE): If `_AM_COMPILER_EXEEXT' has been provided at `AC_CONFIG_COMMANDS_PRE' time, then introduce a conditional `am__EXEEXT', defined to true iff `$EXEEXT' is nonempty. * tests/check5.test: Only match `_EXEEXT_[1-9]' here, to avoid false positives stemming from `@am__EXEEXT_TRUE@'. * NEWS: Update. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
Ralf Wildenhues 66c2dcaa 2009-03-11T23:55:26 parallel-tests: per-extension test driver: <EXT>_LOG_COMPILER. For test files with extension <ext>, introduce the internal variable <EXT>_LOG_COMPILE, which expands to $(<EXT>_LOG_COMPILER) $(AM_<EXT>_LOG_FLAGS) $(<EXT>_LOG_FLAGS). Turn also the lib/Automake/tests testsuite over to the new test driver. * doc/automake.texi (Tests): Document `EXT_LOG_COMPILER' and `EXT_LOG_FLAGS'. * lib/am/check2.am: Insert `%COMPILE%' right before test. * automake.in (handle_tests): Substitute `COMPILE' for check2, empty for tests without extension, and `$(ext_LOG_COMPILE)' for extension `ext'. In the latter case, define it from the public components. * configure.ac (AM_INIT_AUTOMAKE): Use `parallel-test' globally. * tests/Makefile.am (AUTOMAKE_OPTIONS): Remove, not needed here any more. * lib/Automake/tests/Makefile.am (TESTS_ENVIRONMENT): Split ... (PL_LOG_COMPILER, PL_LOG_FLAGS): ... into these new variables. (TESTS_EXTENSIONS): New variable, initialize to `.pl'. * tests/parallel-tests7.test: New test. * tests/Makefile.am: Update. Suggestion by Akim Demaille. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
Akim Demaille 9aa1e525 2009-03-08T11:29:03 Parallel test execution: new option `parallel-tests'. * automake.in (handle_tests): Set new conditional PARALLEL_TESTS when reading check.am. Handle option `parallel-tests' and all its semantics. Define macros TEST_EXTENSIONS, TEST_SUITE_LOG, TEST_SUITE_HTML, TEST_LOGS, TEST_LOGS_TMP, suffix rules if applicable, and per-target rules for other tests. Add all log files to %clean_files at the `MOSTLY_CLEAN' level. * lib/Automake/Options.pm (_process_option_list): Accept `parallel-tests'. * lib/am/check.am [!PARALLEL_TESTS] (check-TESTS): Move existing testsuite driver under this new conditional. [PARALLEL_TESTS] (am__rst_title, am__rst_section, am__text_box am__sh_e_setup) (am__check_pre, am__check_post): New internal macros. ($(TEST_SUITE_LOG), check-TESTS, .log.html, check-html): New rules. * lib/am/check2.am: New file. * lib/am/Makefile.am (dist_am_DATA): Add check2.am. * tests/Makefile.am (AUTOMAKE_OPTIONS): Use `parallel-tests'. (clean-local): Renamed from distclean-local. * tests/defs.in: Drop VERBOSE handling, not needed here any more. Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>