t/ax/testsuite-summary-checks.sh

Branch


Log

Author Commit Date CI Message
Paul Eggert 61075eab 2025-01-01T14:31:02 maint: make update-copyright
Paul Eggert 1d35638b 2024-06-07T08:41:45 maint: spelling and whitespace fixes Most of these spelling fixes are just to comments and documentation. However, some affect tests as follows: * t/cond36.sh (tparse.h): Fix misspelling of dependency. * t/disthook.sh: Fix misspelling of file. * t/help3.sh: Fix misspelling of long option. * t/instdir-ltlib.sh: Fix misspellings of macro names. This causes the test to fail, so someone with libtool expertise needs to look into this. * t/tap-no-spurious-numbers.sh (highno): Fix misspelling of shell var.
Paul Eggert b80b5c47 2024-01-01T11:29:06 maint: make update-copyright
Mike Frysinger 34bdde96 2023-01-04T02:00:14 maint: make update-copyright
Jim Meyering 6c8ff6a8 2022-01-12T14:15:12 maint: make update-copyright
Jim Meyering a470a47f 2021-07-11T19:19:42 maint: make update-copyright
Karl Berry 216d1834 2020-06-06T15:42:54 automake: support AM_TESTSUITE_SUMMARY_HEADER override. This change handles https://bugs.gnu.org/11745. * lib/am/check.am (AM_TESTSUITE_SUMMARY_HEADER): new variable. Default value is " for $(PACKAGE_STRING)", including quotes, to keep the default output the same. ($(TEST_SUITE_LOG)): use it, unquoted. * doc/automake.texi (Scripts-based Testsuites): document it. * NEWS: mention it. * t/testsuite-summary-header.sh: new test. * t/list-of-tests.mk (handwritten_tests): add it. * t/ax/testsuite-summary-checks.sh: fix typo.
Jim Meyering cf27a3df 2020-01-01T11:44:41 maint: make update-copyright
Paul Eggert 5ae02cc8 2019-10-14T13:46:55 maint: make update-copyright
Mathieu Lirzin bbaa4cdc 2018-01-04T16:19:30 maint: Update copyright years to 2018 This update has been made with 'make update-copyright'.
Mathieu Lirzin c2757b97 2017-09-19T13:43:07 maint: Reset master
Paul Eggert 3562e384 2017-09-16T13:03:36 Prefer https: URLs In Gnulib, Emacs, etc. we are changing ftp: and http: URLs to use https:, to discourage man-in-the-middle attacks when downloading software. The attached patch propagates these changes upstream to Automake. This patch does not affect files that Automake is downstream of, which I'll patch separately. Althouth the resources are not secret, plain HTTP is vulnerable to malicious routers that tamper with responses from GNU servers, and this sort of thing is all too common when people in some other countries browse US-based websites. See, for example: Aceto G, Botta A, Pescapé A, Awan MF, Ahmad T, Qaisar S. Analyzing internet censorship in Pakistan. RTSI 2016. https://dx.doi.org/10.1109/RTSI.2016.7740626 HTTPS is not a complete solution here, but it can be a significant help. The GNU project regularly serves up code to users, so we should take some care here.
Paul Eggert 199e7a44 2017-09-16T13:03:36 Prefer https: URLs In Gnulib, Emacs, etc. we are changing ftp: and http: URLs to use https:, to discourage man-in-the-middle attacks when downloading software. The attached patch propagates these changes upstream to Automake. This patch does not affect files that Automake is downstream of, which I'll patch separately. Althouth the resources are not secret, plain HTTP is vulnerable to malicious routers that tamper with responses from GNU servers, and this sort of thing is all too common when people in some other countries browse US-based websites. See, for example: Aceto G, Botta A, Pescapé A, Awan MF, Ahmad T, Qaisar S. Analyzing internet censorship in Pakistan. RTSI 2016. https://dx.doi.org/10.1109/RTSI.2016.7740626 HTTPS is not a complete solution here, but it can be a significant help. The GNU project regularly serves up code to users, so we should take some care here.
Mathieu Lirzin d8add592 2017-03-02T18:55:53 maint: Update copyright years to 2017. This update has been made with 'make update-copyright'.
Jim Meyering 1370ce5f 2017-01-01T08:34:49 maint: update copyright dates for 2017 * all files: Run this command, using update-copyright from gnulib: UPDATE_COPYRIGHT_FORCE=1 \ UPDATE_COPYRIGHT_USE_INTERVALS=2 \ UPDATE_COPYRIGHT_MAX_LINE_LENGTH=79 \ update-copyright $(git ls-files)
Stefano Lattarini 5de75f07 2015-01-05T22:48:33 maint: update copyright years to 2015 (branch 'micro') Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini a78f63c5 2014-04-21T15:10:54 maint: update copyright years We've been in 2014 already for few months now... Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 74c1336d 2013-05-24T20:54:41 tests: rename $am_make_rc_got -> $am_make_rc The latter is more natural More importantly, I keep misspelling the former over and over. * t/ax/am-test-lib.sh (run_make): Update. * t/ax/tap-summary-aux.sh: Adjust. * t/ax/testsuite-summary-checks.sh: Likewise. * t/parallel-tests-exit-statuses.sh: Likewise. * t/parallel-tests-extra-programs.sh: Likewise. * t/parallel-tests-fd-redirect-exeext.sh: Likewise. * t/parallel-tests-fd-redirect.sh: Likewise. * t/parallel-tests-fork-bomb.sh: Likewise. * t/parallel-tests-no-spurious-summary.sh: Likewise. * t/parallel-tests-recheck-pr11791.sh: Likewise. * t/parallel-tests-reset-term.sh: Likewise. * t/tap-signal.tap: Likewise. * t/test-driver-acsubst.sh: Likewise. * t/test-driver-cond.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini e2a000cc 2013-05-22T23:29:47 tests: avoid use of redirected 'run_make' invocations Instead, properly use the -E, -O, or -M options of run_make. The occurrences of the 'run_make' invocations using output redirection has been found by the recently-introduced maintainer check 'sc_tests_no_run_make_redirect'. * t/ax/tap-summary-aux.sh: Adjust. * t/ax/testsuite-summary-checks.sh: Likewise. * Several other tests: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini ffd4ecc9 2013-05-20T11:23:01 tests: better idiom to override make macro defs on the cmdline We now do so with a new wrapper shell function 'run_make()', which relies on the use of AM_MAKEFLAGS, avoiding the use of the '-e' make option. The use of that option (that causes the environment variables to take precedence over the macro definitions in the Makefile) has proved to be quite brittle in the past, causing annoying and sometimes problematic spurious failures. This has worsened in some still unpublished work going on in development branches. It's time to solve the issue once and for all. Note that we do not convert all uses of $MAKE in the testsuite right away; we might do so in follow-up changes, with leisure, to avoid a "patch bomb" effect (this commit is already too much of a bomb itself). What we do in this commit is to get rid of all "$MAKE -e" invocations. We admit that the implementation and feature-set of 'run_make()' are far from perfect, but good enough for our current purposes. We'll improve 'run_make()' if and when the need arises. * syntax-checks.mk (sc_tests_no_make_e): New syntax check, guard against the use of "$MAKE -e". (syntax_check_rules): Add it. (sc_tests_overriding_macros_on_cmdline): Adjust. (lint): New, alias for 'maintainer-check', for lazy typists. Idea backported from the 'maint' branch (Automake 1.13a). * t/ax/am-test-lib.sh (run_make): New function. Run $MAKE with the given command-line arguments, handling command-line override of variable definitions in a smart way (using AM_MAKEFLAGS if a non-GNU make implementation is detected to be in use). (useless_vpath_rebuild): Adjust to use 'run_make', to avoid a spurious maintainer check failure. (yl_distcheck): Use 'run_make' rather than bare '$MAKE'. (single_quote, append_single_quoted, is_valid_varname): New auxiliary function, used, directly or indirectly, by it. * Many tests: Adjust to avoid the use of "$MAKE -e", and prefer the use of 'run_make' in few other contexts as well, where it makes sense. Other minor fixlets while at it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7df8b28c 2012-12-31T18:18:37 maint: update copyright year for 2013 (in branch maint) Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 93c2c33e 2012-11-07T21:51:01 Merge branch 'maint' * maint: tests: new variable $am_testaux_builddir tests: rename $am_testauxdir -> $am_testaux_srcdir
Stefano Lattarini 62235835 2012-11-07T17:52:25 tests: rename $am_testauxdir -> $am_testaux_srcdir This is just a preparatory change in view of a future commit. * t/ax/test-defs.in: Here. * t/ax/tap-summary-aux.sh: And here. * t/ax/testsuite-summary-checks.sh: And here. * t/distcheck-missing-m4.sh: And here. * t/distcheck-outdated-m4.sh: And here. * t/self-check-shell-no-trail-bslash.sh: And here. * t/test-driver-acsubst.sh: And here. * t/test-driver-cond.sh: And here. * t/test-driver-custom-multitest.sh: And here. * t/test-driver-custom-multitest-recheck.sh: And here. * t/test-driver-custom-multitest-recheck2.sh: And here. * t/testsuite-summary-count-many.sh: And here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5ddf1007 2012-10-27T16:41:39 tests: prefer including 'test-init.sh' rather than './defs' This is a follow-up to today's commit v1.12.4-22-g0610fc8, "tests: prepare to move ./defs to t/ax/test-init.sh" * All tests: To run the common setup, use the command: . test-init.sh instead of the older, "historical" one: . ./defs || exit 1 Note that the "|| exit 1" wasn't really useful, since the 'errexit' shell flag is in effect in both './defs' and 'test-init.sh', and all the known shells that are good enough to run the automake testsuite do automatically exit with error when a sourced file cannot be found (at least, they do so in non-interactive mode, which is the only mode that concerns us in the testsuite). * t/ax/tap-summary-aux.sh, t/ax/testsuite-summary-checks.sh: Likewise. * gen-testsuite-part: Do the same in the generated tests. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 2d5571e7 2012-07-22T16:38:47 check: support colorized testsuite output by default Fixes automake bug#11855. With this change, we make the 'color-tests' option enabled by default. That option remains still silently accepted as a no-op, for backward compatibility. The developer of a package is still free to disable testsuite coloring on a per-makefile basis (by adding "AM_COLOR_TESTS = no" to the Makefile.am) or on a whole-project basis (by AC_SUBST'ing AM_COLOR_TESTS to "no" in configure.ac). But now the user will be able to request the testsuite output to be colorized, if he really wants to: # With GNU make: make AM_COLOR_TESTS=yes check # With non-GNU make: make AM_COLOR_TESTS=yes AM_MAKEFLAGS=AM_COLOR_TESTS=yes check * NEWS: Update. * doc/automake.texi: Updated, and some related minor reformatting and rewording. * automake.in (handle_tests): No need to pass the transform '%COLOR%' when processing "check.am". * lib/am/parallel-tests.am: Remove use of the '%?COLOR%' transform; just act as if it were unconditionally TRUE. * t/color.sh: No need to specify 'color-tests' in 'AUTOMAKE_OPTIONS' nor in 'AM_INIT_AUTOMAKE'. * t/ax/testsuite-summary-checks.sh: Likewise. * t/ax/tap-summary-aux.sh: Likewise. * t/color2.sh: Likewise. Also, ensure that colorized testsuite output can be disabled by default by calling "AC_SUBST([AM_COLOR_TESTS], [no])". * t/tap-realtime.sh: Define 'AM_COLOR_TESTS' to "no" in Makefile.am, to avoid spurious colorization of the output due to the use of the 'expect' program. * t/color-tests-opt.sh: New test, check that the 'color-tests' option is still recognized as a no-op. * t/list-of-tests.mk: Add the new test. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b21d6869 2012-06-28T13:27:49 Merge branch 'maint' * maint: tests: simpler workaround for shells losing the exit status in exit trap + Extra non-trivial edits: * Several tests: Adjusted to use 'exit' rather than 'Exit'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini c2a52bf0 2012-06-24T10:36:15 tests: simpler workaround for shells losing the exit status in exit trap Now that we can assume our tests are run by a decent POSIX shell, we can simplify our workaround aimed at having the exit status propagated correctly to the code in the exit trap. Unfortunately, we cannot dispense with such a workaround altogether, because it's still required by some shells we need to support (at least Solaris 10 /bin/ksh and /usr/xpg4/bin/sh). For more information about the need of that workaround, see the entry about 'trap' in the section "Limitations of Shell Builtins" in the Autoconf manual: <http://www.gnu.org/software/autoconf/manual/autoconf.html#trap> The new workaround has been tested successfully with the following shells: - Bash 4.1 - Bash 3.2 - Bash 3.0 - Bash 2.05b - dash 0.5.5.1 - dash 0.5.2 - AT&T Ksh 93u (from official Debian package) - MirBSD Korn Shell 40.2 (from official Debian package) - Solaris 9, 10 and 11 /bin/ksh - Solaris 9, 10 and 11 /usr/xpg4/bin/sh - NetBSD 5.1 /bin/sh - NetBSD 5.1 /bin/ksh * t/ax/test-init.sh (Exit): Rename ... (_am_exit): ... like this. (exit): New alias for '_am_exit'. We cannot simply redefine 'exit' as a shell function, because some shells (dash 0.5.5.1, Solaris 10 /bin/ksh and /usr/xpg4/bin/sh) do not allow it. (_am_exit, trap): Add extra escaping for 'exit' calls, to ensure we really invoke the 'exit' builtin and not our alias with the same name. * configure.ac: Check that the shell selected to run our testsuite supports aliases named like shell builtins. * t/REAMDE: Adjust. * All tests: Adjust, by simply using 'exit' instead of 'Exit'. * t/self-check-explicit-skips.sh: Adjust: the first usage of 'exit' after it has been redefined as an alias must be on a new line w.r.t. that where the alias is defined, in order for the redefinition to be honored. * syntax-checks.mk (sc_tests_Exit_not_exit): Delete. (sc_tests_exit_not_Exit): New. (syntax_check_rules): Adjust. (sc_tests_automake_fails): Simplify the recipe a little. * Several tests: Remove now useless spurious quoting once required to placate the 'sc_tests_Exit_not_exit' maintainer check. * gen-testsuite-part: Likewise. Also, avoid uses of 'Exit' in the generated scripts. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5e771b27 2012-05-18T18:00:51 check: the parallel-tests driver is now the default The old serial testsuite driver is still supported, and can be activated by the 'serial-tests' option (introduced by commit 'v1.11-1989-gdeb7773' of 2012-02-20, "automake: new option 'serial-tests'"). * lib/Automake/Options.pm: Now, the 'parallel-tests' option is a no-op (but still explicitly recognized, for backward compatibility), while the 'serial-tests' option is recognized and registered. * automake.in: Adjust to ensure that the parallel test harness is used by default; mostly, this amounts at using "! option('serial-tests')" where "option('parallel-tests')" was used before. * NEWS, doc/automake.texi: Update. * configure.ac (AM_INIT_AUTOMAKE): Drop 'parallel-tests' option, is redundant now. * defs: In the generated AM_INIT_AUTOMAKE call, use the 'serial-tests' when the serial testsuite harness is to be used, and no option otherwise. * lib/am/check.am: Use (static) conditional '%SERIAL_TESTS%' instead of conditional '%PARALLEL_TESTS%'. * lib/test-driver: Adjust heading comments. * t/ax/extract-testsuite-summary.pl: Likewise. * t/check-exported-srcdir.sh: Likewise. * t/check-subst.sh: Likewise. * t/java-compile-run-nested.sh: Likewise. * t/parallel-tests-dry-run-1.sh: Likewise. * t/parallel-tests-dry-run-2.sh: Likewise. * t/parallel-tests-interrupt.sh: Likewise. * t/parallel-tests-many.sh: Likewise. * t/test-metadata-global-log.sh: Likewise. * t/tests-environment-backcompat.sh: Likewise. * m4/init.m4 (AM_INIT_AUTOMAKE): Adjust a comment. * t/check8.sh: Likewise. * t/check-tests-in-builddir.sh: Likewise. * t/java-compile-run-flat.sh: Adjust a botched comment. * t/ax/tap-summary-aux.sh: Remove explicit use of the 'parallel-tests' option. * t/ax/testsuite-summary-checks.sh: Likewise. * t/aclocal-path-precedence.sh: Likewise. * t/dist-aux-many-subdirs.sh: Likewise. * t/parallel-tests-no-color-in-log.sh: Likewise. * t/parallel-tests-reset-term.sh: Likewise. * t/parallel-tests-subdir.sh: Likewise. * t/check-no-test-driver.sh: Fix setup to adjust to the changed semantics. Update heading comments. * t/parallel-tests-driver-install.sh: Adjust to the new semantic. * t/test-driver-is-distributed.sh: Simplified according to the new semantics. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7b142a7b 2012-05-15T16:12:09 tests: use 'parallel-tests' Automake option by default This will help our transition from 'serial-tests' to 'parallel-tests' as the default test suite driver enabled by a TESTS assignment in the input Makefile.am. Note that that change of default will only take place in master, though. * defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE invocation in the created 'configure.ac' stub, unless the variable 'am_serial_tests' is set to "yes". Don't pay attention anymore to the 'am_parallel_tests' variable, that's obsolete now. * defs-static.in: Warn if the 'am_serial_tests' variable is set in the environment; conversely, don't warn anymore about 'am_parallel_tests' being set in the environment. * Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests' variable instead of the now-obsolete 'am_parallel_tests' one. * syntax-checks.mk (sc_tests_obsolete_variables): Also warn against uses of 'am_parallel_tests', which is now deprecated in favor of 'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen, suggest using 'am_serial_tests' instead, not 'am_parallel_tests'. * gen-testsuite-part: Now that we use the 'parallel-tests' by default in our tests, we need to completely change the logic and semantics of generation of sibling tests for those tests that check the Automake generated testsuite harness itself. Do that, and give a complete explanation of the new logic and semantics in the relevant comments. * t/README: Update. * Lots of test cases: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b105d40d 2012-04-30T20:44:50 color-tests: coloring can be forced on non-ANSI terminals as well Before this change, colorization of testsuite output was suppressed whenever the terminal was recognized to be a "dumb" one, incapable of handling ANSI coloring (i.e., when the environment variable TERM had a value of "dumb"). This happened even when the AM_COLOR_TESTS variable was set to a value of "always". Such a behaviour was suboptimal and slightly confusing; in fact, if a user wants to force coloring of testsuite output that is being redirected to a regular file, he should be able to do so even if his terminal is not capable of handling ANSI colors -- in fact, such terminal wouldn't be involved with the testsuite output in any way, so why should it be allowed to influence it? Thus, we now enable coloring of test output whenever AM_COLOR_TESTS is set to "always", irrespective of the value of the TERM environment variable. * NEWS: Update. * lib/am/check.am [%?COLOR%] (am__tty_colors): Activate colorization of testsuite output whenever AM_COLOR_TESTS has the value of "always". * t/ax/tap-summary-aux.sh: Export the TERM environment variable to "dumb" when forcing colorization of the testsuite output; this should *not* prevent such colorization from taking place, and we want to check that this expectation really holds. * t/ax/testsuite-summary-checks.sh: Likewise. * t/color.sh: Likewise, and adjust some grepping checks. * t/tap-color.sh: Likewise. Also, remove redundant "make check" invocation since we are at it. * t/color2.sh: Likewise, and check that exporting TERM=dumb actually prevents testsuite output colorization when AM_COLOR_TESTS is unset. * t/parallel-tests-reset-term.sh: Relax, to prevent it from failing spuriously due to the new semantic. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 8a5096d1 2012-04-06T13:12:25 tests: rename 'tests/' => 't/', '*.test' => '*.sh' When we (soon) convert the Automake testsuite to a non-recursive make setup, we'll have to fix the entries of $(TESTS) to be prepended with the subdirectory they are in; this will increase the length of $(TESTS), and thus increase the possibility of exceeding the command-line length limits on some systems (most notably, MinGW/MSYS). See automake bug#7868 for more information. Thus we rename the 'tests/' subdirectory to 't/', and each 'x.test' script in there to 'x.sh'; this way, the $(TESTS) entry 'foo.test' will become 't/foo.sh', which have the same number of characters. * tests/: Rename ... * t/: ... to this. * t/*.test: Rename ... * t/*.sh: ... to this. * t/.gitignore: Removed as obsolete. * t/defs: Adjust. * t/gen-testsuite-part: Likewise. * t/list-of-tests.mk: Likewise. * t/ccnoco.sh: Likewise. * t/ccnoco3.sh: Likewise. * t/self-check-cleanup.tap: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-me.tap: Likewise. * t/self-check-reexec.tap: Likewise. * README: Likewise. * bootstrap: Likewise * configure.ac: Likewise. * Makefile.am: Likewise. * .gitignore: Likewise. * syntax-check.mk: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>