kmx git

Commit	Date	Message
0c81b43f	2011-07-28T12:25:22	test harness: use new `.trs' files to hold test metadata With this change, the test harness will keep test metadata in dedicated `.trs' files, instead of having them embedded into the `.log' files. This allows for easier forward-compatibility and extension of test metadata, and for more flexibility in the format of the `.log' files. Note that this change makes the `:end-metadata:' field obsolete. * doc/automake.texi (Parallel Test Harness, Log files generation and test results recording): Document the new APIs and behaviour; some related minor rewordings and fixlets. * NEWS: Update. * automake.in (handle_tests): When bringing in the content of `check2.am', substitute %BASE% with the basename of the `.log' file being created by a rule. Add the generated `.trs' files to the list of files to be cleaned by "make mostlyclean". * lib/am/check.am (am__test_driver_flags): Rename ... (am__common_driver_flags): ... to this, and remove the flags `--test-name' and `--log-file' from it: they are now define in the proper rules in `check2.am'. (am__TEST_BASES): New internal variable, holding the names of the tests, with any registered extension removed. (am__stealth_MAKE): New internal variable, can be used instead of $(MAKE) in recipes requiring a recursive call to make, but which are not intended to be executed by "make -n". (.log.trs): New suffix rule, to recover from deletion of `.trs' files. ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the new API of "test logs in `.log' files, test metadata in `.trs' files". It goes to some length to work correctly in face of unreadable or missing `.log' and `.trs' files, and to error out with proper error messages when this is not possible. [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale" `.trs' files (in addition to `.log files) before remaking the $(TEST_SUITE_LOG). (recheck, recheck-html): Look for the `:recheck:' field in the `.trs' files, not in the `.log' files. * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust the call to the test driver, in particularly passing the new option `--trs-file'. [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise. * lib/tap-driver ($USAGE): Adjust the help screen. (Getopt::Long::GetOptions): Handle the `--trs-file' option, through the use of ... ($trs_file): ... this new global variable. (finish): Write metadata for the test run to `$trs_file' rather then to `$log_file', through the use of ... (write_test_results): ... this new function. * lib/test-driver (print_usage): Update the help screen. (Option parsing): Handle the `--trs-file' option, through the use of ... ($resfile): ... this new global variable. (Main code): Write metadata for the test run to `$trsfile' rather than to `$logfile'. Minor related adjustments to comments. * tests/.gitignore: Ignore `.trs' files. tests/parallel-tests-unreadable-log.test: Moved ... * tests/parallel-tests-unreadable.test: ... to this, and extended to also check the semantics for unreadable `.trs' files. * tests/test-driver-end-metadata.test: Deleted as obsolete. * tests/test-driver-metadata-no-leading-space.test: Likewise. * tests/test-driver-global-log.test: Renamed ... * tests/test-metadata-global-log.test: ... to this, and modified as to verify the new APIs and semantics. * tests/test-driver-recheck: Renamed ... * tests/test-metadata-recheck.test: ... to this, and modified likewise. * tests/parallel-tests-once.test: New test. * tests/parallel-tests-make-n.test: Likewise. * test-metadata-results.test: Likewise. * test-missing.test: Likewise. * test-missing2.test: Likewise. * test-trs-basic.test: Likewise. * test-trs-recover.test: Likewise. * test-trs-recover2.test: Likewise. * tests/Makefile.am (TESTS): Update.
4014b502	2011-07-26T14:51:20	test harness: allow more metadata in log files This change reworks and improves the parallel test harness to use more specialized reStructuredText fields in the log files (instead of relying on specially-placed of "magic lines" and more indirect semantical formatting); the new fields are the following: - ":recheck:": tell whether the associated test will have to be re-run by "make recheck"; - ":copy-in-global-log:": tell whether the content of the log file should be copied in the "global log" `test-suite.log'; - ":end-metadata:", which inhibits the scanning of the rest of the log file (for what concerns test metadata). Also, the special `:test-result:' value "END" has been removed, superseded by the new `:end-metadata:' field. * doc/automake.texi (Log files generation and test results recording): Document the new API and semantics. Remove or fix some obsolete comments. * lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html): Adjust comments and code. * lib/tap-driver (finish): Adjust, with the help of ... (must_recheck, copy_in_global_log): ... these new functions. * lib/test-driver (Main code): Adjust, with the help of ... ($recheck, $gcopy): ... these new variables. * tests/trivial-test-driver: Update to obey the new APIs. * tests/test-driver-recheck.test: Likewise. * tests/test-driver-global-log.test: Likewise. * tests/tap-passthrough.test: Relax the test, avoiding to check what is written in `test-suite.log'; such check has been moved ... * tests/tap-global-log.test: ... in this new test, and extended. * tests/test-driver-metadata-no-leading-space.test: New test. * tests/test-driver-end-test-results.test: Removed, it checked the old APIs; superseded by ... * tests/test-driver-end-metadata.test: ... this new test. * tests/tap-log.test: Improve syncing with ... * tests/test-log.test: ... this new test. * tests/parallel-tests.test: Remove some duplication w.r.t. this last new test. Updated heading comments. * tests/Makefile.am (TESTS): Update.
7858f2bb	2011-07-17T16:04:06	tap: some preparatory refactoring (2) This is a follow-up simplification. * lib/tap-driver (console_output): Renamed ... (report): ... to this, and extended to appropriately register the test results when needed. (testsuite_error, handle_tap_comment, handle_tap_test, handle_tap_plan): Adjusted accordingly.
2fdeb6bb	2011-07-17T16:46:41	tap: support colorization of testsuite progress output * lib/tap-driver (%COLORS): New variable (definition extracted from `lib/am/check.am:$(am__tty_colors)', with some obvious adjustments. (report): Adjust to colorize console output when required, using ... (decorate_result): ... this new function. (colored): New function, used by the one above. * tests/tap-summary.test: Also run the checks when `color-tests' is in use. * tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.
ba5fce66	2011-07-17T15:18:51	tap: some preparatory refactoring (1) This refactoring is aimed at simplifying the introduction of colored console output for the TAP driver. * lib/tap-driver (console_output): Now accept two arguments, the first one indicating which kind of thing is to be displayed (for now only a test result or a diagnostic comment), and the second one (if present) the message associated to it. (handle_tap_test, handle_tap_comment, handle_tap_plan, testsuite_error): Adapt to the new `console_output' interface.
58c182ae	2011-07-03T19:20:22	tap: add experimental TAP-aware driver * doc/automake.texi (Using the TAP test protocol): New section. (Overview of Custom Test Drivers Support): Minor updates. * lib/tap-driver: New script, TAP-aware test driver for Automake; implemented in perl and based on TAP::Parser. * lib/Makefile.am (dist_script_DATA): Add it. * tests/tap-autonumber.test: New test. * tests/tap-bailout.test: Likewise. * tests/tap-basic.test: Likewise. * tests/tap-deps.test: Likewise. * tests/tap-diagnostic.test: Likewise. * tests/tap-empty.test: Likewise. * tests/tap-empty-diagnostic.test: Likewise. * tests/tap-escape-directive.test: Likewise. * tests/tap-exit.test: Likewise. * tests/tap-fancy.test: Likewise. * tests/tap-fancy2.test: Likewise. * tests/tap-global-result.test: Likewise. * tests/tap-html.test: Likewise. * tests/tap-log.test: Likewise. * tests/tap-merge-stdout-stderr.test: Likewise. * tests/tap-more.test: Likewise. * tests/tap-more2.test: Likewise. * tests/tap-no-merge-stdout-stderr.test: Likewise. * tests/tap-no-spurious-summary.test: Likewise. * tests/tap-no-spurious.test: Likewise. * tests/tap-not-ok-skip.test: Likewise. * tests/tap-numeric-description.test: Likewise. * tests/tap-out-of-order.test: Likewise. * tests/tap-passthrough.test: Likewise. * tests/tap-plan.test: Likewise. * tests/tap-plan-errors.test: Likewise. * tests/tap-plan-corner.test: Likewise. * tests/tap-realtime.test: Likewise. * tests/tap-recheck-logs.test: Likewise. * tests/tap-recheck.test: Likewise. * tests/tap-skip-whole.test: Likewise. * tests/tap-summary.test: Likewise. * tests/tap-todo-skip.test: Likewise. * tests/tap-todo-skip-together.test: Likewise. * tests/tap-todo-skip-whitespace.test: Likewise. * tests/tap-skipall-whitespace.test: Likewise. * tests/tap-unplanned.test: Likewise. * tests/tap-whitespace-normalization.test: Likewise. * tests/tap-with-and-without-number.test: Likewise. * tests/tap-xfail-tests.test: Likewise. * tests/tap-bad-prog.test: New xfailing test. * tests/tap-color.test: Likewise. * tests/tap-plan-corner2.test: Likewise. * tests/tap-message-0.test: Likewise. * tests/tap-signal.test: Likewise. * tests/Makefile.am (TESTS, XFAIL_TESTS): Update.

0c81b43f

2011-07-28T12:25:22

test harness: use new `.trs' files to hold test metadata With this change, the test harness will keep test metadata in dedicated `.trs' files, instead of having them embedded into the `.log' files. This allows for easier forward-compatibility and extension of test metadata, and for more flexibility in the format of the `.log' files. Note that this change makes the `:end-metadata:' field obsolete. * doc/automake.texi (Parallel Test Harness, Log files generation and test results recording): Document the new APIs and behaviour; some related minor rewordings and fixlets. * NEWS: Update. * automake.in (handle_tests): When bringing in the content of `check2.am', substitute %BASE% with the basename of the `.log' file being created by a rule. Add the generated `.trs' files to the list of files to be cleaned by "make mostlyclean". * lib/am/check.am (am__test_driver_flags): Rename ... (am__common_driver_flags): ... to this, and remove the flags `--test-name' and `--log-file' from it: they are now define in the proper rules in `check2.am'. (am__TEST_BASES): New internal variable, holding the names of the tests, with any registered extension removed. (am__stealth_MAKE): New internal variable, can be used instead of $(MAKE) in recipes requiring a recursive call to make, but which are not intended to be executed by "make -n". (.log.trs): New suffix rule, to recover from deletion of `.trs' files. ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the new API of "test logs in `.log' files, test metadata in `.trs' files". It goes to some length to work correctly in face of unreadable or missing `.log' and `.trs' files, and to error out with proper error messages when this is not possible. [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale" `.trs' files (in addition to `.log files) before remaking the $(TEST_SUITE_LOG). (recheck, recheck-html): Look for the `:recheck:' field in the `.trs' files, not in the `.log' files. * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust the call to the test driver, in particularly passing the new option `--trs-file'. [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise. * lib/tap-driver ($USAGE): Adjust the help screen. (Getopt::Long::GetOptions): Handle the `--trs-file' option, through the use of ... ($trs_file): ... this new global variable. (finish): Write metadata for the test run to `$trs_file' rather then to `$log_file', through the use of ... (write_test_results): ... this new function. * lib/test-driver (print_usage): Update the help screen. (Option parsing): Handle the `--trs-file' option, through the use of ... ($resfile): ... this new global variable. (Main code): Write metadata for the test run to `$trsfile' rather than to `$logfile'. Minor related adjustments to comments. * tests/.gitignore: Ignore `*.trs' files. * tests/parallel-tests-unreadable-log.test: Moved ... * tests/parallel-tests-unreadable.test: ... to this, and extended to also check the semantics for unreadable `.trs' files. * tests/test-driver-end-metadata.test: Deleted as obsolete. * tests/test-driver-metadata-no-leading-space.test: Likewise. * tests/test-driver-global-log.test: Renamed ... * tests/test-metadata-global-log.test: ... to this, and modified as to verify the new APIs and semantics. * tests/test-driver-recheck: Renamed ... * tests/test-metadata-recheck.test: ... to this, and modified likewise. * tests/parallel-tests-once.test: New test. * tests/parallel-tests-make-n.test: Likewise. * test-metadata-results.test: Likewise. * test-missing.test: Likewise. * test-missing2.test: Likewise. * test-trs-basic.test: Likewise. * test-trs-recover.test: Likewise. * test-trs-recover2.test: Likewise. * tests/Makefile.am (TESTS): Update.

4014b502

2011-07-26T14:51:20

test harness: allow more metadata in log files This change reworks and improves the parallel test harness to use more specialized reStructuredText fields in the log files (instead of relying on specially-placed of "magic lines" and more indirect semantical formatting); the new fields are the following: - ":recheck:": tell whether the associated test will have to be re-run by "make recheck"; - ":copy-in-global-log:": tell whether the content of the log file should be copied in the "global log" `test-suite.log'; - ":end-metadata:", which inhibits the scanning of the rest of the log file (for what concerns test metadata). Also, the special `:test-result:' value "END" has been removed, superseded by the new `:end-metadata:' field. * doc/automake.texi (Log files generation and test results recording): Document the new API and semantics. Remove or fix some obsolete comments. * lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html): Adjust comments and code. * lib/tap-driver (finish): Adjust, with the help of ... (must_recheck, copy_in_global_log): ... these new functions. * lib/test-driver (Main code): Adjust, with the help of ... ($recheck, $gcopy): ... these new variables. * tests/trivial-test-driver: Update to obey the new APIs. * tests/test-driver-recheck.test: Likewise. * tests/test-driver-global-log.test: Likewise. * tests/tap-passthrough.test: Relax the test, avoiding to check what is written in `test-suite.log'; such check has been moved ... * tests/tap-global-log.test: ... in this new test, and extended. * tests/test-driver-metadata-no-leading-space.test: New test. * tests/test-driver-end-test-results.test: Removed, it checked the old APIs; superseded by ... * tests/test-driver-end-metadata.test: ... this new test. * tests/tap-log.test: Improve syncing with ... * tests/test-log.test: ... this new test. * tests/parallel-tests.test: Remove some duplication w.r.t. this last new test. Updated heading comments. * tests/Makefile.am (TESTS): Update.

7858f2bb

2011-07-17T16:04:06

tap: some preparatory refactoring (2) This is a follow-up simplification. * lib/tap-driver (console_output): Renamed ... (report): ... to this, and extended to appropriately register the test results when needed. (testsuite_error, handle_tap_comment, handle_tap_test, handle_tap_plan): Adjusted accordingly.

2fdeb6bb

2011-07-17T16:46:41

tap: support colorization of testsuite progress output * lib/tap-driver (%COLORS): New variable (definition extracted from `lib/am/check.am:$(am__tty_colors)', with some obvious adjustments. (report): Adjust to colorize console output when required, using ... (decorate_result): ... this new function. (colored): New function, used by the one above. * tests/tap-summary.test: Also run the checks when `color-tests' is in use. * tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.

ba5fce66

2011-07-17T15:18:51

tap: some preparatory refactoring (1) This refactoring is aimed at simplifying the introduction of colored console output for the TAP driver. * lib/tap-driver (console_output): Now accept two arguments, the first one indicating which kind of thing is to be displayed (for now only a test result or a diagnostic comment), and the second one (if present) the message associated to it. (handle_tap_test, handle_tap_comment, handle_tap_plan, testsuite_error): Adapt to the new `console_output' interface.

58c182ae

2011-07-03T19:20:22

tap: add experimental TAP-aware driver * doc/automake.texi (Using the TAP test protocol): New section. (Overview of Custom Test Drivers Support): Minor updates. * lib/tap-driver: New script, TAP-aware test driver for Automake; implemented in perl and based on TAP::Parser. * lib/Makefile.am (dist_script_DATA): Add it. * tests/tap-autonumber.test: New test. * tests/tap-bailout.test: Likewise. * tests/tap-basic.test: Likewise. * tests/tap-deps.test: Likewise. * tests/tap-diagnostic.test: Likewise. * tests/tap-empty.test: Likewise. * tests/tap-empty-diagnostic.test: Likewise. * tests/tap-escape-directive.test: Likewise. * tests/tap-exit.test: Likewise. * tests/tap-fancy.test: Likewise. * tests/tap-fancy2.test: Likewise. * tests/tap-global-result.test: Likewise. * tests/tap-html.test: Likewise. * tests/tap-log.test: Likewise. * tests/tap-merge-stdout-stderr.test: Likewise. * tests/tap-more.test: Likewise. * tests/tap-more2.test: Likewise. * tests/tap-no-merge-stdout-stderr.test: Likewise. * tests/tap-no-spurious-summary.test: Likewise. * tests/tap-no-spurious.test: Likewise. * tests/tap-not-ok-skip.test: Likewise. * tests/tap-numeric-description.test: Likewise. * tests/tap-out-of-order.test: Likewise. * tests/tap-passthrough.test: Likewise. * tests/tap-plan.test: Likewise. * tests/tap-plan-errors.test: Likewise. * tests/tap-plan-corner.test: Likewise. * tests/tap-realtime.test: Likewise. * tests/tap-recheck-logs.test: Likewise. * tests/tap-recheck.test: Likewise. * tests/tap-skip-whole.test: Likewise. * tests/tap-summary.test: Likewise. * tests/tap-todo-skip.test: Likewise. * tests/tap-todo-skip-together.test: Likewise. * tests/tap-todo-skip-whitespace.test: Likewise. * tests/tap-skipall-whitespace.test: Likewise. * tests/tap-unplanned.test: Likewise. * tests/tap-whitespace-normalization.test: Likewise. * tests/tap-with-and-without-number.test: Likewise. * tests/tap-xfail-tests.test: Likewise. * tests/tap-bad-prog.test: New xfailing test. * tests/tap-color.test: Likewise. * tests/tap-plan-corner2.test: Likewise. * tests/tap-message-0.test: Likewise. * tests/tap-signal.test: Likewise. * tests/Makefile.am (TESTS, XFAIL_TESTS): Update.

kc3-lang/automake/lib/tap-driver

lib/tap-driver

Log