kc3-lang/automake/t/ax

t/ax

Log

Author Commit Date CI Message
Stefano Lattarini 74c1336d 2013-05-24T20:54:41 tests: rename $am_make_rc_got -> $am_make_rc The latter is more natural More importantly, I keep misspelling the former over and over. * t/ax/am-test-lib.sh (run_make): Update. * t/ax/tap-summary-aux.sh: Adjust. * t/ax/testsuite-summary-checks.sh: Likewise. * t/parallel-tests-exit-statuses.sh: Likewise. * t/parallel-tests-extra-programs.sh: Likewise. * t/parallel-tests-fd-redirect-exeext.sh: Likewise. * t/parallel-tests-fd-redirect.sh: Likewise. * t/parallel-tests-fork-bomb.sh: Likewise. * t/parallel-tests-no-spurious-summary.sh: Likewise. * t/parallel-tests-recheck-pr11791.sh: Likewise. * t/parallel-tests-reset-term.sh: Likewise. * t/tap-signal.tap: Likewise. * t/test-driver-acsubst.sh: Likewise. * t/test-driver-cond.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini f6b9a22e 2013-05-24T12:46:17 tests: ensure $required is not set too late And do so in a safer way, with a runtime check rather than a brittle static maintainer check. * t/ax/test-init.sh: Set the 'required' variable to readonly. * syntax-checks.mk (sc_tests_required_after_defs): Remove. (syntax_check_rules): No longer list it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini d19eb85c 2013-05-24T01:13:40 tests: use append mode to capture parallel make output This will allow us to run the Automake testsuite forcing all the make invocations in the test cases to run in parallel mode [1], but without hitting the spurious failure reported in automake bug#11413. See also: <http://lists.gnu.org/archive/html/bug-make/2013-05/msg00135.html> The make invocations in the test cases can be forced to run in parallel mode by, e.g., invoking "make check" like this: make check AM_TESTSUITE_MAKE="make -j16" The possible spurious failures hinted above are due to the fact that the redirected output of parallel make can racily lose lines. For example, if GNU make (3.82) is run with -j10 on a Makefile like this: all = 0 1 2 3 4 5 6 7 8 9 default: $(all) $(all): @sleep 0.$$(($RANDOM % 10)); echo $@ and has its standard output redirected in overwrite-mode to a regular file, it looses a line of that output every 15 runs or so on a Fedora 17 system with 64 ppc64 cores. Redirection in append mode does not suffer of this problem, as explained here: <http://lists.gnu.org/archive/html/automake-patches/2009-03/msg00073.html> See also previous commits v1.12-63-g45c1fcd of 2012-05-05 (tests: use append mode to capture parallel make output) and Release-1-10-280-g6426999 of 2009-03-10 (Use append mode to capture parallel test output). * t/ax/am-test-lib.sh (run_make): Adjust and enhance. * t/lisp8.sh: Take advantage of the enhancement, nd stop doing output redirection for $MAKE by hand. * t/tap-more.sh: Likewise. * t/parallel-tests-concurrency.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 3ce4015f 2013-05-23T12:10:18 tests: avoid '$MAKE' redirections, use 'run_make' instead The use 'run_make' with the -E, -O and -M option, it is more idiomatic now. Also, this way, centralized fixes and improvements done in 'run_make' will automatically propagate through most of the testsuite. * syntax-checks.mk (sc_tests_no_run_make_redirect): Also check against '$MAKE' invocations that uses output redirections (and not only against 'run_make' invocation that do so). * Several tests: Adjust (and few minor cosmetic changes as well, while at it). Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini e2a000cc 2013-05-22T23:29:47 tests: avoid use of redirected 'run_make' invocations Instead, properly use the -E, -O, or -M options of run_make. The occurrences of the 'run_make' invocations using output redirection has been found by the recently-introduced maintainer check 'sc_tests_no_run_make_redirect'. * t/ax/tap-summary-aux.sh: Adjust. * t/ax/testsuite-summary-checks.sh: Likewise. * Several other tests: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 15996acc 2013-05-22T13:00:48 tests: remove some code duplication * t/ax/am-test-lib (null_install): New function. * t/instdir-java.sh: Use it instead of copied & pasted code. * t/instdir-lisp.sh: Likewise. * t/instdir-ltlib.sh: Likewise. * t/instdir-prog.sh: Likewise. * t/instdir-python.sh: Likewise. * t/instdir-texi.sh: Likewise. * t/instdir.sh: Likewise. * t/instdir2.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini c7cfa093 2013-05-21T20:22:17 tests: run_make: options to do command redirection Let's improve the API of the 'run_make()' helper shell function by adding three new options: -O Save the standard output from make on disk, in a regular file named 'stdout'. -E Save the standard error from make on disk, in a regular file named 'stderr'. -M Save both the standard output and standard error from make on disk, in a regular file named 'output'. This option supersedes both the '-O' and '-E' options. This new API has two main advantages. 1. Its use will allow us to get rid of more cumbersome idioms like, e.g., $MAKE check >stdout && { cat stdout; exit 1; } cat stdout That can now be substituted with a simpler one: run_make -e FAIL -O check 2. More importantly, using the new API we will prevent any extra output from the shell traces of the code in run_make to be redirected along with the make stderr (where that was redirected). This problem was present in usages like, e.g., run_make TESTS=foo.test check 2>stderr && exit 1 grep 'expected error message' stderr Such usages are now to be rewritten as follows: run_make -e FAIL -E TESTS=foo.test check grep 'expected error message' stderr ensuring that 'stderr' won't end up containing unrelated stuff. Note that we do not convert in bulk the old idioms and the use of redirected 'run_make' invocations with this patch. We only convert some occurrences, to ensure that the new implementation of 'run_make' is sound enough. More sweeping conversions will likely be done in follow-up patches. * t/ax/am-test-lib.sh (run_make): Enhance and implement the extended API. * t/tap-xfail-tests.sh: Use the new 'run_make' API. * t/test-driver-cond.sh: Likewise. * t/tests-environment-fd-redirect.sh: Likewise. * t/uninstall-fail.sh: Likewise. * t/yacc-dist-nobuild.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini a422365b 2013-05-21T11:59:21 tests: only activate 'unset' alias if required This makes the test logs easier to read for most shells (which do not actually require that alias). This is especially important now that 'unset' is used in the new 'run_make()' function, and that function is likely going to be used more and more in the future. * t/ax/test-lib.sh (_am_unset, unset): Only define this function and alias if "unset VAR" returns a non-zero exit status when VAR is already unset. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini ffd4ecc9 2013-05-20T11:23:01 tests: better idiom to override make macro defs on the cmdline We now do so with a new wrapper shell function 'run_make()', which relies on the use of AM_MAKEFLAGS, avoiding the use of the '-e' make option. The use of that option (that causes the environment variables to take precedence over the macro definitions in the Makefile) has proved to be quite brittle in the past, causing annoying and sometimes problematic spurious failures. This has worsened in some still unpublished work going on in development branches. It's time to solve the issue once and for all. Note that we do not convert all uses of $MAKE in the testsuite right away; we might do so in follow-up changes, with leisure, to avoid a "patch bomb" effect (this commit is already too much of a bomb itself). What we do in this commit is to get rid of all "$MAKE -e" invocations. We admit that the implementation and feature-set of 'run_make()' are far from perfect, but good enough for our current purposes. We'll improve 'run_make()' if and when the need arises. * syntax-checks.mk (sc_tests_no_make_e): New syntax check, guard against the use of "$MAKE -e". (syntax_check_rules): Add it. (sc_tests_overriding_macros_on_cmdline): Adjust. (lint): New, alias for 'maintainer-check', for lazy typists. Idea backported from the 'maint' branch (Automake 1.13a). * t/ax/am-test-lib.sh (run_make): New function. Run $MAKE with the given command-line arguments, handling command-line override of variable definitions in a smart way (using AM_MAKEFLAGS if a non-GNU make implementation is detected to be in use). (useless_vpath_rebuild): Adjust to use 'run_make', to avoid a spurious maintainer check failure. (yl_distcheck): Use 'run_make' rather than bare '$MAKE'. (single_quote, append_single_quoted, is_valid_varname): New auxiliary function, used, directly or indirectly, by it. * Many tests: Adjust to avoid the use of "$MAKE -e", and prefer the use of 'run_make' in few other contexts as well, where it makes sense. Other minor fixlets while at it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 12cfd38f 2013-05-20T15:17:11 test-lib: minor style changes * t/ax/am-test-lib.sh (require_tool): Here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 8a39c6de 2013-05-20T15:07:48 test-lib: fix botched function name in an error message * t/ax/am-test-lib.sh (useless_vpath_rebuild): Here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 84e12c32 2013-05-17T13:45:44 tests: sanitize 'unset' usages In some shells (e.g., Solaris 10 /bin/ksh, or NetBSD 5.1 /bin/sh), "unset VAR" returns a non-zero exit status in case the VAR variable is already unset. This doesn't interact well with our usage of "set -e" in the testsuite. So far, we've avoided spurious failures by either explicitly ignoring the exit status from unset: unset VAR || : or explicitly ensuring that a variable is set, before trying to unset it: VAR=; unset VAR But we can do better, by aliasing the 'unset' command to a custom function that will take care of these details for us. This will avoid us annoying spurious failures in the future, failures that have already bitten us too much times. For an example, refer to commit 'v1.12.2-88-g5b1dae5' of 2012-08-05 (tests: avoid tons of spurious failures on NetBSD). * t/ax/test-lib.sh (_am_unset): New function. (unset): New alias to it. (_am_exit): Adjust comments. * t/ax/am-test-lib.sh: No need to temporary disable the 'errexit' shell flag when unsetting variables that are potentially already unset. (am_process_requirements): Adjust to remove a now-useless workaround related to unset. * t/aclocal-macrodir.tap: Likewise. * t/aclocal-macrodirs.tap: Likewise. * t/auxdir-autodetect.sh: Likewise. * t/ax/am-test-lib.sh: Likewise. * t/ax/test-lib.sh: Likewise. * t/check-tests-in-builddir.sh: Likewise. * t/dist-formats.tap: Likewise. * t/distcheck-configure-flags-am.sh: Likewise. * t/distcheck-configure-flags.sh: Likewise. * t/java-empty-classpath.sh: Likewise. * t/javaflags.sh: Likewise. * t/lflags.sh: Likewise. * t/lflags2.sh: Likewise. * t/lisp-flags.sh: Likewise. * t/lisp6.sh: Likewise. * t/missing-auxfile-stops-makefiles-creation.sh: Likewise. * t/parallel-am.sh: Likewise. * t/parallel-am2.sh: Likewise. * t/parallel-am3.sh: Likewise. * t/parallel-tests-log-override-recheck.sh: Likewise. * t/pkg-config-macros.sh: Likewise. * t/python-missing.sh: Likewise. * t/python-too-old.sh: Likewise. * t/python11.sh: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-report.sh: Likewise. * t/self-check-seq.tap: Likewise. * t/silent-configsite.sh: Likewise. * t/suffix6c.sh: Likewise. * t/tar-override.sh: Likewise. * t/tests-environment-and-log-compiler.sh: Likewise. * t/vala-configure.sh: Likewise. * t/werror3.sh: Likewise. * t/yflags-cmdline-override.sh: Likewise. * t/yflags.sh: Likewise. * t/yflags2.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 176d229b 2013-05-16T14:49:44 tests: fix some botched/outdated comments * t/ax/test-defs.in: Here. Also, fix a couple of grammaros while at it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 361430c0 2013-05-16T14:18:55 tests: use perl, not find+rm, to remove temporary directories The File::Path::rmtree function from perl, if used right, is more reliable and more portable of our past idiom: find $dirs -type d ! -perm -700 -exec chmod u+rwx {} ';'; rm -rf $$dirs || exit 1 at least of the face of unreadable dirs/files and other similar permission issues (and we have those in our test directories). In fact, this change fixes some spurious failures seen in "make distcheck" on Solaris 10. * t/ax/deltree.pl: New. * Makefile.am (EXTRA_DIST): Add it. (clean-local-check): Use it. * t/ax/test-lib.sh (rm_rf_): Use it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 4348577b 2013-05-15T22:05:21 vala tests: skip in a cross compiler setup The "vala + pkg-config" doesn't interact well with cross-compilation: <https://mail.gnome.org/archives/vala-list/2012-September/msg00094.html> That is causing annoying failures in our testsuite when it's run in cross-compilation mode. It's not worth trying too be overly smart and work around these problems; just skip the affected tests instead. * t/ax/am-test-lib.sh (require_tool): A new requirement 'valac', telling that the test needs to compile Vala-generated C files. In particular, this causes the test to be skipped when using a cross-compiler. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 24dbfd93 2013-01-29T13:24:50 typofix: in comments in 't/ax/test-lib.sh' Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini d3844bbf 2013-02-14T12:32:27 tests on TAP: don't run the driver with perl unconditionally * t/ax/tap-setup.sh: When a 'Makefile.am' was pre-existent in the test directory at the moment the client test script sourced this file, said 'Makefile.am' was tweaked to provide it with a proper definition of TEST_LOG_DRIVER. However, there was an error in this automatic definition, since it caused the TAP test driver to be unconditionally invoked with perl. This wasn't an issue in most situations, since perl is smart enough to re-execute a given script with the proper interpreter if it sees a she-bang line that doesn't seem to point to perl itself. Still, there is no reason to do something blatantly wrong even if our tools correct the dumb mistake for us. So fix the TEST_LOG_DRIVER definition. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 43e25e38 2013-02-14T11:46:58 tests: remove obsolete (and now wrong) comment * t/ax/am-test-lib.sh: Here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7df8b28c 2012-12-31T18:18:37 maint: update copyright year for 2013 (in branch maint) Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini de736a22 2012-12-14T10:43:21 Merge branch 'maint' * maint: docs: fixlet about xz compression levels and command line options maint: more files excluded in 'update-copyright' maint: adapt 'update-copyright' recipe to the new $(FETCHFILES) format HACKING: update instructions to update the manuals at www.gnu.org maint: add an explicative comment in Makefile maint: factor out the name of the dir where web manuals are generated maint: rename rules to build/upload web manuals cosmetic: minor variable and rules reordering in Makefile.am docs: build the web manuals for CVS in the builddir, not in the srcdir release: automate uploading of web manuals in CVS docs: AM_CFLAGS: remove reference to non-existing "more detailed" desc tests: avoid two spurious failures on FreeBSD tests: improve 'pkg-config' requirement
Stefano Lattarini 7a4e1937 2012-12-13T09:46:18 tests: improve 'pkg-config' requirement This improves (and unifies among different tests) the detection of both the 'pkg-config' program and m4 macros provided by it; they are needed in particular by tests on Vala support. Before this, some Vala-related tests were failing spuriously when the pkg-config program was available but the macros from pkg.m4 were not made explicitly available in ACLOCAL_PATH. These tests were: vala-mix vala-mix2 vala-parallel vala-vapi vala-vpath See also similar commit 'v1.11-581-gb7d67d5' of 2011-12-22, "tests: better handling of gettext and libtool requirements". * t/ax/am-test-lib.sh (process_requirements): If a requirement matches the 'pkg-config*' wildcard, try to make available to it the pkg-config m4 macros, and skip it if it's not possible. This is done by sourcing the 'get.sh' fragment generated by ... * t/pkg-config-macros.sh: ... this new test, which looks in advance for the 'pkg-config' program and the m4 macros that comes with it. * t/list-of-tests.mk: Add the new test. * gen-testsuite-part (%deps_extractor): Tests requiring pkg-config will need the m4 macros coming with it; so these tests should be run after the 'pkg-config-macros.sh' test. Automatically generate proper declarations for such dependencies. * t/vala-header.sh: No need to explicitly check that the macro PKG_CHECK_MODULES has been expanded by autoconf. * t/vala-libs.sh: Likewise. * t/vala-non-recursive-setup.sh: Likewise. * t/vala-per-target-flags.sh: Likewise. * t/vala-recursive-setup.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini cb6a19d9 2012-11-21T14:13:02 Merge branch 'maint' * maint: tests: fix a spurious failure when $PYTHON is in the environment python tests: support PEP-3147 installation layout python: uninstall cater to PEP-3147 tests: improve a comment tests: honour $PYTHON override tests: typofix in message news: document fix for bug#8847 (PEP-3147, __pycache__) python: improve support for modern python (CPython 3.2 and PyPy) Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini e1acf4a0 2012-11-19T22:19:21 python tests: support PEP-3147 installation layout This fixes several spurious testsuite failures with python >= 3.2, introduced by recent commit v1.12.4-43-ge0e99ed, "python: improve support for modern python (CPython 3.2 and PyPy)". * t/ax/am-test-lib.sh (python_has_pep3147, pyc_location, py_installed): New functions. ($am_pep3147_tag): New variable. * t/py-compile-basic2.sh: Deleted, too difficult to adapt for the gain it would offer; move the still relevant parts ... * t/py-compile-basic.sh: ... here. Adapt and adjust the rest of the test as well. * t/nobase-python.sh: Adapt and adjust. * t/py-compile-basedir.sh: Likewise. * t/py-compile-destdir.sh: Likewise. * t/py-compile-option-terminate.sh: Likewise. * t/python-pr10995.sh: Likewise. * t/python-virtualenv.sh: Likewise. * t/python10.sh: Likewise. * t/python12.sh: Likewise. * t/python3.sh: Likewise. * t/list-of-tests.mk: Adjust list of tests. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini cae0d860 2012-11-11T12:39:40 tests: improve a comment * t/ax/am-test-lib.sh (require_tool): Here, about python support for '--version' option. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini c3dc9003 2012-11-11T12:03:46 tests: honour $PYTHON override * t/ax/am-test-lib.sh (require_tool): Here. * t/python-too-old.sh: And here. * t/python-vars.sh: And here. * t/python-virtualenv.sh: And here. Also add some sanity checks while at it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 93c2c33e 2012-11-07T21:51:01 Merge branch 'maint' * maint: tests: new variable $am_testaux_builddir tests: rename $am_testauxdir -> $am_testaux_srcdir
Stefano Lattarini bd0d41d7 2012-11-07T17:59:40 tests: new variable $am_testaux_builddir And a related fix that solves a spurious testsuite failures in VPATH builds. * t/ax/test-defs.in ($am_testaux_builddir): New, counterpart of $am_testaux_srcdir, but pointing inside the build directory. * t/self-check-shell-no-trail-bslash.sh: Use $am_testaux_builddir, not $am_testaux_srcdir, when fetching the 'shell-no-trail-bslash' script. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 62235835 2012-11-07T17:52:25 tests: rename $am_testauxdir -> $am_testaux_srcdir This is just a preparatory change in view of a future commit. * t/ax/test-defs.in: Here. * t/ax/tap-summary-aux.sh: And here. * t/ax/testsuite-summary-checks.sh: And here. * t/distcheck-missing-m4.sh: And here. * t/distcheck-outdated-m4.sh: And here. * t/self-check-shell-no-trail-bslash.sh: And here. * t/test-driver-acsubst.sh: And here. * t/test-driver-cond.sh: And here. * t/test-driver-custom-multitest.sh: And here. * t/test-driver-custom-multitest-recheck.sh: And here. * t/test-driver-custom-multitest-recheck2.sh: And here. * t/testsuite-summary-count-many.sh: And here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 153f861b 2012-10-31T12:41:13 Merge branch 'maint' * maint: vala tests: source test-init.sh, not ./defs tests: fix a spurious typo-related failure tests: remove spurious leftover use of 'Exit' tests: can check our recipes avoid trailing backslashes vala: improve comments to AM_PROG_VALAC news: update w.r.t. recent vala changes vala: if no proper compiler found, set $(VALAC) to 'valac' vala: AM_PROG_VALAC should not produce an error for tool-old valac docs: document recent changes to AM_PROG_VALAC tests: enhance tests on AM_PROG_VALAC vala: style fixes in vala.m4 vala: add action arguments, for when no proper vala compiler is found Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini aecd1be6 2012-10-28T14:02:46 tests: can check our recipes avoid trailing backslashes This is related to commit v1.11-1704-g254227b of 2012-05-01, "parallel-tests: avoid trailing backslashes in make recipes", and automake bug#10436. Recipes with a trailing backslash character (possibly followed by blank characters only) can cause spurious syntax errors with at least older bash versions (e.g., bash 2.05b), and can be potentially be unportable to other weaker shells. So provide a target that runs the testsuite looking for this kind of breakage (without requiring a real bugged shell). * t/ax/shell-no-trail-bslash.in: New, a "shell" that chokes on '-c' commands having a trailing '\' (possibly followed by whitespace only). * Makefile (t/ax/shell-no-trail-bslash): Generate this script from it. (noinst_SCRIPTS, CLEANFILES): Add it. (EXTRA_DIST): Add 't/ax/shell-no-trail-bslash.in'. (check-no-trailing-backslash-in-recipes): New target, runs the testsuite with 'shell-no-trail-bslash' as the CONFIG_SHELL, to catch possible recipes having a trailing backslash character (possibly followed by * .gitignore: Update. * t/self-check-shell-no-trail-bslash.sh: New testsuite self-check. * t/parallel-tests-trailing-bslash.sh: Remove as obsolete. * t/list-of-tests.mk: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 57770a2c 2012-10-27T18:31:21 tests: get rid of ./defs, it's no longer used. * defs: Delete. * configure.ac (AC_CONFIG_LINKS): No longer link it. * t/ax/tap-setup.sh: Don't look for a file named 'defs' in the grandparent directory of the current test directory to decide whether we're running in the correct directory. * syntax-check.mk (xdefs): Remove '$(srcdir)/defs'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 55ea5e0f 2012-10-27T17:08:28 tests: remove obsolescent references to './defs' * t/distcom-subdir.sh, t/distcom2.sh, t/ax/depcomp.sh, t/conff2.sh, t/ccnoco3.sh, t/ccnoco.sh, t/self-check-dir.tap, t/self-check-exit.tap, t/self-check-me.tap: Here. Instead, refer to "am-test-lib.sh", "test-init.sh", or simply "the testsuite setup", as appropriate. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5ddf1007 2012-10-27T16:41:39 tests: prefer including 'test-init.sh' rather than './defs' This is a follow-up to today's commit v1.12.4-22-g0610fc8, "tests: prepare to move ./defs to t/ax/test-init.sh" * All tests: To run the common setup, use the command: . test-init.sh instead of the older, "historical" one: . ./defs || exit 1 Note that the "|| exit 1" wasn't really useful, since the 'errexit' shell flag is in effect in both './defs' and 'test-init.sh', and all the known shells that are good enough to run the automake testsuite do automatically exit with error when a sourced file cannot be found (at least, they do so in non-interactive mode, which is the only mode that concerns us in the testsuite). * t/ax/tap-summary-aux.sh, t/ax/testsuite-summary-checks.sh: Likewise. * gen-testsuite-part: Do the same in the generated tests. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b0f45569 2012-10-27T14:23:40 Merge branch 'maint' * maint: tests: prepare to move ./defs to t/ax/test-init.sh
Stefano Lattarini 0610fc8e 2012-10-27T12:59:41 tests: prepare to move ./defs to t/ax/test-init.sh We don't do this in a sweeping passage, because that would cause endless headaches in the synchronization between the maint, master and ng/master branches. Instead, we setup our framework to allow test scripts to work by sourcing either './defs' or 'test-init.sh', so that we'll be able to make the transition gradual and painless. * t/ax/test-init.sh: New, copied from the previous ./defs file. * defs: Simply work by sourcing the new file. * Makefile.am (dist_noinst_DATA): List the new file. * t/README: Adjust to mandate the sourcing of 'test-init.sh' rather than of './defs'. * t/c-demo.sh: Source 'test-init.sh' instead of ./defs. This is done to verify our new setup actually works. * t/ac-output-old.tap: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 97998210 2012-08-26T12:55:38 Merge branch 'maint' * maint: docs: don't suggest to use recursive makefile setup tests: fix a timestamp race in python tests tests: fixup: make distcheck-override-infodir pass again sync: update files from upstream with "make fetch" maint: post-release minor version bump release: stable release 1.12.3 maintcheck: fix spurious warnings docs: fix typo: s/make install-info/make uninstall-info/ tests: fixup: make a couple of tests executable Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini af1ac94a 2012-08-13T18:00:57 maintcheck: fix spurious warnings * t/distcheck-override-infodir.sh: Be sure that valid occurences of the "aclocal" and "automake" strings, which can confuse the 'sc_tests_plain_automake' check, are protected by leading "#" characters. * t/ax/test-lib.sh: Always use '$(...)' for command subtitution, to avoid triggering the 'sc_tests_command_subst' check; there was still once place where `...` was used. While at it, fix a related comment. * t/ax/test-defs.in ($sleep): Use creative quoting to avoid spuriously triggering the 'sc_tests_plain_sleep' check. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 2a7c9aea 2012-08-05T17:45:07 Merge branch 'maint' * maint: tests: avoid spurious failures with older Texinfo tests: avoid tons of spurious failures on NetBSD runtest: avoid spurious failures on NetBSD news: dependency tracking for Portland Group Compilers is now supported
Stefano Lattarini 5b1dae57 2012-08-05T12:14:19 tests: avoid tons of spurious failures on NetBSD * t/ax/am-test-lib.sh (process_requirements): Set the '$am_tool' variable to the empty string before trying to unset it; otherwise, we might be attempting to unset an already-unset variable, which (together with the presence of the 'errexit' shell flag) causes spurious failures at least with the /bin/sh shell from NetBSD 5.1. This was actually causing the great majority of the Automake tests (all those not using a "required=..." declaration) to fail spuriously on that platform! Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 106428fe 2012-07-26T18:16:47 Merge branch 'maint' * maint: (38 commits) maintcheck: fixup list of files in $(xdefs) tests: never source test-defs.sh directly, source test-lib.sh instead runtest: sanitize test environment tests: remove an obsolescent self test tests: "am_using_tap=yes" -> "am_test_protocol=tap" tests: protect test libs against multiple inclusion configure: testsuite shell can return early from "dot-sourced" files tests: move sanitization and "Bournification" in the generic test lib tests: source test defs in the generic test lib test defs: no need to re-add $srcdir/t/ax to $PATH tests: split test libs into "generic" and "automake-specific" test setup: move actual calling of testsuite setup in ./defs test setup: merge definitions of function for simple tests test init: refactor: new function 'am_test_setup' test init: refactor: move displaying of debugging info later test init: refactor: new function 'am_setup_testdir' test init: refactor: new function 'am_set_exit_traps' configure: testsuite shell set exit traps in shell functions test init: refactor: new function 'am_exit_trap' test init: refactor: new function 'process_requirements' ... Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini db188c29 2012-07-26T11:04:31 runtest: sanitize test environment * runtest.in: Here, similarly to what is done by AM_TESTS_ENVIRONMENT in Makefile.am, unset variables that should be under the complete control of the test framework, and that could create havoc if inherited from the environment. This remove the need to check against possible environment "pollution" ... * t/ax/test-defs.in: ... in here. * Makefile.am (AM_TESTS_ENVIRONMENT): Add a comment about the need of synchronization with 'runtest.in'. * t/self-check-env-sanitize.tap: Remove as obsolete. * t/list-of-tests.mk: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini e82c9156 2012-07-26T10:35:09 tests: "am_using_tap=yes" -> "am_test_protocol=tap" And similarly, "am_using_tap=no" -> "am_test_protocol=none". The new '$am_test_protocol' name is clearer, and will allow the easy addition of further test protocols in the future. This is not truly relevant for automake, but we are trying to make some parts of our testsuite framework as general as possible, in view of a future move to a more generic project like Gnulib. * Makefile.am, t/ax/am-test-lib.sh, t/ax/test-defs.in, t/ax/test-lib.sh, t/self-check-env-sanitize.tap, t/self-check-tap.sh: Adjust. * syntax-checks.mk (sc_tests_obsolete_variables): Add 'am_using_tap' to the list of obsolete variables to check against. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini d8b3c5e4 2012-07-26T12:21:40 tests: protect test libs against multiple inclusion * t/ax/test-lib.sh, t/ax/am-test-lib.sh: Return early if already sourced. Use the witness variables '$test_lib_sourced' and '$am_test_lib_sourced', respectively, for this purpose. * runtest.in, Makefile.am (AM_TESTS_ENVIRONMENT): Unset 'test_lib_sourced' and 'am_test_lib_sourced', to avoid interferences from the environment. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 1ab1f546 2012-07-26T10:22:52 tests: move sanitization and "Bournification" in the generic test lib * t/ax/test-defs.sh: From here ... * t/ax/test-lib.sh: ... to here. Also move the initialization of '$argv0' and '$me'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 4fcf43cd 2012-07-26T10:09:04 tests: source test defs in the generic test lib * t/ax/test-lib.sh: That is, here ... * defs: ... rather than here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7a42af01 2012-07-26T10:03:51 test defs: no need to re-add $srcdir/t/ax to $PATH * t/ax/test-defs.sh: Here. That is already done by both 'runtest' and AM_TESTS_ENVIRONMENT, and the presence of '$srcdir/t/ax' in $PATH is anyway required, after the recent changes, for our testsuite framework to work at all. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 55fb7140 2012-07-25T22:49:02 tests: split test libs into "generic" and "automake-specific" This is the first step in the quest to merge the generically useful parts of our test suite framework in a more generic project, like Gnulib. Time will tell if we'll succeed, and whether the success will be worth the extra hassle. * t/ax/test-init.sh: Split out ... * t/ax/am-test-lib.sh, t/ax/test-lib.sh: ... into these two tests. * defs, Makefile.am: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini feab5c26 2012-07-25T21:12:30 test setup: move actual calling of testsuite setup in ./defs * t/ax/test-init.sh: Move call to 'am_test_setup' ... * defs: ... here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 3bb5a901 2012-07-25T21:04:30 test setup: merge definitions of function for simple tests * t/ax/plain-functions.sh: Delete, moving the definitions of ... ($stderr_fileno_): ... this variable ... (warn_, fail_, skip_, skip_all_, fatal_, framework_failure_): ... and these functions ... * t/ax/test-init.sh: ... in here. This allow us to use those functions earlier in this file (instead of having to duplicate their behaviour with inlined code). The TAP-based tests are still able to override these functions later to their TAP-enhanced equivalents when 'tap-functions.sh' is sourced. * Makefile.am (dist_noinst_DATA): Remove 't/ax/plain-functions.sh'. * t/ax/tap-functions.sh: Adjust a comment. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 472e4e47 2012-07-25T20:31:47 test init: refactor: new function 'am_test_setup' * t/ax/test-init.sh (am_test_setup): Here. Call it from the main code. Remove other calls of functions and settings that are now duly called by 'am_test_setup'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 22443fe7 2012-07-25T20:25:55 test init: refactor: move displaying of debugging info later * t/ax/test-init.sh: Here, about the values of $PATH, $am_using_tap and $am_running_installcheck. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 1f4d8c22 2012-07-25T20:20:09 test init: refactor: new function 'am_setup_testdir' * t/ax/test-init.sh (am_setup_testdir): Here. Use it instead of inlining its contents in the main code. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini d9686f93 2012-07-25T20:17:46 test init: refactor: new function 'am_set_exit_traps' * t/ax/test-init.sh (am_set_exit_traps): Here. (trap): Use it instead of inlining the cleanup/finalization code. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7f3690db 2012-07-25T19:45:31 test init: refactor: new function 'am_exit_trap' * t/ax/test-init.sh (am_exit_trap): Here. (trap): Use it instead of inlining the cleanup/finialization code. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 1a915b9a 2012-07-25T19:41:57 test init: refactor: new function 'process_requirements' * t/ax/test-init.sh (process_requirements): Here. Use it in to check that the tools in $required are present. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 42bd480c 2012-07-25T19:23:29 test init: refactor: new function 'require_tool' * t/ax/test-init.sh (require_tool): Here. Use it in the loop in the prerequisites in $required. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 2bb312c8 2012-07-25T18:49:50 test init: remove a couple of paranoid sanity checks * t/ax/test-init.sh: Here, about 'am_top_srcdir' and 'am_top_builddir'; the values of those variables are AC_SUBST'd, so it's nigh impossible for the to be wrong or get "messed up". Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini ecbceb3e 2012-07-25T18:14:23 test defs: move in 't/ax/test-defs.sh' We can do so with minimal churn, now that the early setup of $PATH in both 'runtest' and AM_TESTS_ENVIRONMENT allow the '.' built-in to find to-be-sourced shell scripts in 't/ax' automatically, both in in-tree and VPATH builds. With this change, we take another step forward a more rational and "segregated" organization of our testsuite framework. * defs-static.in: Rename ... * t/ax/test-defs.in: ... like this, and adjust. * Makefile.am, syntax-checks.mk, t/README, gen-testsuite-part, t/tests-init.sh, t/ax/tap-setup.sh, t/tap-summary.sh, .gitignore, t/testsuite-summary-color.sh, t/testsuite-summary-count.sh, t/self-check-tap.sh, defs: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 42d9439d 2012-07-25T19:06:22 perf tests: reorganize * t/list-of-tests.mk (handwritten_TESTS): Don't list ($perf_TESTS) in here anymore. * Makefile.am (EXTRA_DIST): Add them explicitly. (perf): New target, run the performance tests and save the logs of the ones with unexpected results in the file 't/perf/test-suite.log', which is thus ... (PERF_TEST_SUITE_LOG): ... defined in this variable ... (CLEANFILES): ... and added to this. (test_subdirs): Remove 't/perf': the tests in there are no more listed in $(TESTS). * t/ax/test-init.sh: Don't explicitly skip "perf" test here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 2d5571e7 2012-07-22T16:38:47 check: support colorized testsuite output by default Fixes automake bug#11855. With this change, we make the 'color-tests' option enabled by default. That option remains still silently accepted as a no-op, for backward compatibility. The developer of a package is still free to disable testsuite coloring on a per-makefile basis (by adding "AM_COLOR_TESTS = no" to the Makefile.am) or on a whole-project basis (by AC_SUBST'ing AM_COLOR_TESTS to "no" in configure.ac). But now the user will be able to request the testsuite output to be colorized, if he really wants to: # With GNU make: make AM_COLOR_TESTS=yes check # With non-GNU make: make AM_COLOR_TESTS=yes AM_MAKEFLAGS=AM_COLOR_TESTS=yes check * NEWS: Update. * doc/automake.texi: Updated, and some related minor reformatting and rewording. * automake.in (handle_tests): No need to pass the transform '%COLOR%' when processing "check.am". * lib/am/parallel-tests.am: Remove use of the '%?COLOR%' transform; just act as if it were unconditionally TRUE. * t/color.sh: No need to specify 'color-tests' in 'AUTOMAKE_OPTIONS' nor in 'AM_INIT_AUTOMAKE'. * t/ax/testsuite-summary-checks.sh: Likewise. * t/ax/tap-summary-aux.sh: Likewise. * t/color2.sh: Likewise. Also, ensure that colorized testsuite output can be disabled by default by calling "AC_SUBST([AM_COLOR_TESTS], [no])". * t/tap-realtime.sh: Define 'AM_COLOR_TESTS' to "no" in Makefile.am, to avoid spurious colorization of the output due to the use of the 'expect' program. * t/color-tests-opt.sh: New test, check that the 'color-tests' option is still recognized as a no-op. * t/list-of-tests.mk: Add the new test. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 15b8b62a 2012-07-07T12:55:29 Merge branch 'maint' * maint: tests: move 'runtest' into the top-level directory
Stefano Lattarini ea93d8d0 2012-07-07T12:08:05 tests: move 'runtest' into the top-level directory Move our wrapper script 'runtest' (meant to allow the execution of Automake test cases from the command line) from the 't/ax/' directory to the top-level one. This makes the script easier to find and to invoke. Much more importantly, our DejaGNU-checking test cases won't try anymore to use that script instead of the 'runtest' program provided by DejaGNU (that happened because '$(srcdir)/t/ax/' is automatically added early to the $PATH variable in our test cases), which was causing spurious SKIPs. * t/ax/runtest.in: Move ... * runtest.in: ... here. * Makefile.am, t/README, .gitignore: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini cbd1a604 2012-07-07T11:54:28 Merge branch 'maint' * maint: cosmetics: rename t/ax/test-runner => t/ax/runtest fixup: another "make recheck" failure with BSD make test runner: work correctly in VPATH setups compat: automake should define $(mkdir_p), for backward compatibility coverage: test that AM_PROG_MKDIR_P and $(mkdir_p) still works tests init: don't automatically re-execute tests with a POSIX shell yacc tests: fix spurious failure with parallel make tests: ignore minor 'recheck' regression for BSD make tests: don't clutter the top-level dir with temporary test directories tests: avoid spurious failures when @MKDIR_P@ points to "install-sh -d" lisp: better support of VPATH builds news: fixlets and updates + Extra non-trivial edits: * NEWS: State that $(mkdir_p) is still provided as an alias to $(MKDIR_P), for (partial) backward-compatibility. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5197aba1 2012-07-06T23:42:11 cosmetics: rename t/ax/test-runner => t/ax/runtest The latter is shorter and clearer. Better to do the rename early, before other developers or contributors begin to get used to the 'test-runner' name. * t/ax/test-runner.in: Rename ... * t/ax/runtest.in: ... like this. * Makefile.am, t/README, .gitignore: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b6e6409a 2012-07-06T12:24:33 tests init: don't automatically re-execute tests with a POSIX shell I've unwittingly broken support for that feature *again* in some of my recent testsuite tweaking. In this case, the re-execution code works correctly when the tests are executed with a POSIX shells, but breaks when they are invoked by an old-style Bourne shells (e.g., /bin/sh on Solaris). It's time to face it: that feature is too much brittle, and too seldom used (because the Makefile takes care of running the tests with the correct shell anyway, so that a breakage is only experienced when running the tests by hand). It just don't remain working for long, not when we often touch the testsuite setup (which we are going to do again when we'll try to move part of our testsuite framework to Gnulib, or a similar project). So, instead of trying to be extra-smart and automatically re-execute the tests with the correct shell, we now offer a simple wrapper script that the user can employ to run the test scripts with the proper shell. And while we are at it, we write this wrapper to also deal with TAP tests in a better way, running them through the prove(1) utility, so that their results are correctly recognized and reported. * t/ax/test-runner.in: New file; the wrapper script we were talking about. * configure.ac (AC_CONFIG_FILES): Process it into 't/ax/test-runner'. * .gitignore: Add 't/ax/test-runner'. * defs: Remove code for automatic re-execution of the scripts with the correct shell. This file now just a very thin layer around 'defs-static' and 't/ax/test-init.sh'. * t/README: Adjust, and remove or fix some imprecise or outdated text in the process (like "... test scripts are written with portability in mind, so that they should run with any decent Bourne-compatible shell ..." ). * Makefile.am (AM_TESTS_ENVIRONMENT): No need to export 'AM_TESTS_REEXEC' to "no" anymore. * t/self-check-explicit-skips.sh: Likewise. * t/self-check-exit.tap: Likewise. * t/self-check-me.tap: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-reexec.tap: Remove as obsolete. * t/list-of-tests.mk: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini f2436f5b 2012-07-01T18:01:35 Merge branch 'maint' * maint: tests: don't trust the exit status of "make -k" for non-GNU makes parallel-tests: reimplement fix for bug#11791 tests setup: unset CDPATH tests setup: more namespace safeness tests setup: remove an unused variable tests setup: less hard-coding of the test subdirectory
Stefano Lattarini 0a442c18 2012-07-01T11:50:54 tests setup: unset CDPATH So that our test scripts can safely chdir around using relative paths as well, without having to worry abut possible CDPATH interferences. * defs-static.in: unset CDPATH. * t/ax/tests-init.sh: Remove a now-unneeded workaround. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 285eed2f 2012-07-01T11:01:02 tests setup: more namespace safeness * t/ax/tests-init.sh ($testSubDir): Rename ... ($am_test_subdir): ... to this, throughout the file. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini c59c8d15 2012-07-01T10:36:34 tests setup: less hard-coding of the test subdirectory * t/ax/tests-init.sh: Make the code creating the temporary test subdirectory smart enough to automatically create it in the same subdirectory of the test that is being run. * defs-static.in ($MKDIR_P, $am_rel_srcdir): New variables, AC_SUBST'd from @MKDIR_P@ and @srcdir@ respectively, and used in the new 'tests-init.sh' code. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini e2f007f0 2012-06-30T13:42:10 Merge branch 'maint' * maint: tests: fix a spurious failure configure: fix detection of POSIX shell to work in a VPATH build tests: prefer "test ! -e FILE" to check that a file doesn't exist Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini bf258729 2012-06-29T00:48:11 tests: prefer "test ! -e FILE" to check that a file doesn't exist Once, for the sake of (at least) Solaris 10 /bin/sh, we had to use "test ! -f FILE" or "test ! -r FILE" or "test ! -d FILE" instead, because the that shell's 'test' built-in didn't grok the '-e' option. Note however that we still can't use "test ! -e" in the Makefile recipes used in the test cases; that is because those recipes are run with the shell detected by 'configure', and Autoconf-generated configure scripts do no guarantee to find or provide a POSIX-compatible shell. * Several tests: Adjust. * t/yacc-clean-cxx: Adjust, and remove a couple of useless commands. * t/parallel-tests-dry-run-2.sh: Adjust, and add invocation to "make -n" forgotten in previous versions of the test. * t/txinfo26.sh: Adjust, and don't bother to skip the test when it's run in a directory whose absolute path contain whitespace: that setup is not supported anyway. * t/maken3.sh: Adjust, and fix a typo that could cause a minor false negative. * t/test-trs-recover2.sh: Prefer using 'skip_' with a suitable error message over a bare 'exit 77'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 8c784651 2012-06-28T23:35:18 Merge branch 'maint' * maint: configure: move a misplaced "section" comment configure: clump check for ${var%...} and ${var#...} expansion together cosmetics: improve wording of a couple of configure messages configure: the testsuite shell must support 'test -e' and 'test ! -e' tests: remove stale workarounds for Solaris /bin/sh
Stefano Lattarini d3a51faf 2012-06-28T21:52:37 tests: remove stale workarounds for Solaris /bin/sh Our testsuite cannot be run with that shell anymore (as it is not POSIX-compliant). * t/tap-more.sh: Remove outdated workarounds for Solaris /bin/sh. * t/self-check-exit.tap: Likewise. * t/ansi2knr-no-more.sh: Likewise. * t/add-missing.tap: Likewise. * t/dist-auxfile.sh: Likewise. * t/test-driver-custom-multitest-recheck2.sh: Likewise. * t/ax/test-init.sh: Remove obsolete references to Solaris /bin/sh. * t/confh5.sh: Likewise. * t/uninstall-fail.sh: Likewise. And update comments about quirks of Solaris /bin/ksh and /usr/xpg4/bin/sh. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5d99776c 2012-06-28T18:38:02 Merge branch 'maint' * maint: tests init: don't bother allowing '$me' to be overridable tests init: typofixes in comments Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 07321606 2012-06-28T18:21:33 tests init: don't bother allowing '$me' to be overridable We once used that feature in our wrapper tests; but now (and probably even since commit 'v1.11-1308-g375f23d' of 2011-09-08, "testsuite: revamp generation of autogenerated tests") it is not needed anymore. By removing it we can simplify our growingly complex testsuite framework a little. * t/ax/test-inist.sh ($me): Do not initialize it here (and only if not already set), instead ... * defs-static.in ($me): ... initialize it here unconditionally. Do not check anymore that $me doesn't come from the environment: that wouldn't cause any problem now. Now that '$me' is defined early, prefer it over 'argv0' in early error messages, both here ... * defs: ... and here. * Makefile.am (AM_TESTS_ENVIRONMENT): Do not bother "nullifying" $me anymore. * t/self-check-env-sanitize.tap: Adjust. * t/self-check-me.tap: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b0df891f 2012-06-28T17:50:20 tests init: typofixes in comments Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 60a69375 2012-06-28T17:41:27 Merge branch 'maint' * maint: tests: one test was not executable, make it so readme: fix typo in t/README: s/$((...)/$((...))/ tests: don't skip if $(abs_builddir) or $(abs_srcdir) contain whitespace tests: avoid spurious maintainer-check failures tests: remove obsolete hacks around maintainer-check false positives
Stefano Lattarini de20404a 2012-06-28T17:08:34 tests: don't skip if $(abs_builddir) or $(abs_srcdir) contain whitespace We used to explicitly skip libtool and gettext tests if the absolute path of the builddir or of the srcdir which Automake was configured with contained any whitespace (or other metacharacters). But several other tests would spuriously fail in such an unholy setup. To be precise, it would cause 61 'FAIL's and 42 'ERROR's in the whole Automake testsuite. The fact that, as of today, nobody has reported any failure of this kind means that (thankfully) nobody is building automake with $(abs_srcdir) or $(abs_builddir) mangled by whitespace. So, instead of trying to cater to such a broken setup consistently, we just drop the extra check in the libtool/gettext tests. In case someone will ever reports a failure due to extra whitespace in either $(abs_srcdir) or or $(abs_builddir), we will simply enhance our 'configure.ac' to bail out flatly and loudly at such a setup. * t/ax/test-init.sh: Simplify accordingly. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 9c906bc7 2012-06-28T16:43:43 tests: avoid spurious maintainer-check failures * t/ax/test-init.sh: Here, by adding extra quoting for two occurrences of the string "perl". Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b21d6869 2012-06-28T13:27:49 Merge branch 'maint' * maint: tests: simpler workaround for shells losing the exit status in exit trap + Extra non-trivial edits: * Several tests: Adjusted to use 'exit' rather than 'Exit'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b4b6a1a6 2012-06-27T15:29:42 Merge branch 'maint' * maint: tests: fix a spurious failure with Solaris make tests: avoid several spurious failures on Solaris tests: avoid a spurious failure on NetBSD
Stefano Lattarini c2a52bf0 2012-06-24T10:36:15 tests: simpler workaround for shells losing the exit status in exit trap Now that we can assume our tests are run by a decent POSIX shell, we can simplify our workaround aimed at having the exit status propagated correctly to the code in the exit trap. Unfortunately, we cannot dispense with such a workaround altogether, because it's still required by some shells we need to support (at least Solaris 10 /bin/ksh and /usr/xpg4/bin/sh). For more information about the need of that workaround, see the entry about 'trap' in the section "Limitations of Shell Builtins" in the Autoconf manual: <http://www.gnu.org/software/autoconf/manual/autoconf.html#trap> The new workaround has been tested successfully with the following shells: - Bash 4.1 - Bash 3.2 - Bash 3.0 - Bash 2.05b - dash 0.5.5.1 - dash 0.5.2 - AT&T Ksh 93u (from official Debian package) - MirBSD Korn Shell 40.2 (from official Debian package) - Solaris 9, 10 and 11 /bin/ksh - Solaris 9, 10 and 11 /usr/xpg4/bin/sh - NetBSD 5.1 /bin/sh - NetBSD 5.1 /bin/ksh * t/ax/test-init.sh (Exit): Rename ... (_am_exit): ... like this. (exit): New alias for '_am_exit'. We cannot simply redefine 'exit' as a shell function, because some shells (dash 0.5.5.1, Solaris 10 /bin/ksh and /usr/xpg4/bin/sh) do not allow it. (_am_exit, trap): Add extra escaping for 'exit' calls, to ensure we really invoke the 'exit' builtin and not our alias with the same name. * configure.ac: Check that the shell selected to run our testsuite supports aliases named like shell builtins. * t/REAMDE: Adjust. * All tests: Adjust, by simply using 'exit' instead of 'Exit'. * t/self-check-explicit-skips.sh: Adjust: the first usage of 'exit' after it has been redefined as an alias must be on a new line w.r.t. that where the alias is defined, in order for the redefinition to be honored. * syntax-checks.mk (sc_tests_Exit_not_exit): Delete. (sc_tests_exit_not_Exit): New. (syntax_check_rules): Adjust. (sc_tests_automake_fails): Simplify the recipe a little. * Several tests: Remove now useless spurious quoting once required to placate the 'sc_tests_Exit_not_exit' maintainer check. * gen-testsuite-part: Likewise. Also, avoid uses of 'Exit' in the generated scripts. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini aa630a92 2012-06-24T17:57:10 tests: avoid several spurious failures on Solaris * t/ax/is_newest: Rewrite to be Bourne-compatible, for /bin/sh shells like Solaris' that are not POSIX-conforming. The script is so small that such a rewrite is easier than going through the hoops that would be required to ensure this script is always executed with a POSIX shell. * t/ax/is: Add a comment stating that this script is to be kept Bourne compatible as well. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini edbcd8f1 2012-06-23T00:16:26 Merge branch 'maint' * maint: tests: avoid one last `...` command substitution in 'test-init.sh' maintcheck: guard against `...` for command substitution in test cases tests: more uses of $(...) over `...` for command substitution cosmetics: quote `like this', not 'like this', in a couple of tests readme: clarify/extend few entries in 't/README' readme: better separation of entries in 't/README' readme: subsections "Do" and "Do not" in 't/README' merged tests: new requirement 'grep-nonprint' cosmetics: fix description of an expected error message in a test tests: assume automake quotes 'like this', not `like this' Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5898641f 2012-06-23T00:12:08 tests: avoid one last `...` command substitution in 'test-init.sh' * t/ax/test-init.sh (me): In the definition of this variable. This also shave off a couple of forks, and the need for a sanity check only required in fringe situations. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 465111cd 2012-06-22T23:31:00 tests: more uses of $(...) over `...` for command substitution Somehow missed by the previous changes. No big deal, fix them now. * t/ax/depcomp.sh: Here. * t/autodist.sh: And here. * t/autodist-no-duplicate.sh: And here. * t/autodist-subdir.sh: Ad here. * t/remake11.sh t/self-check-me.tap: And here. * t/perf/testsuite-summary.sh: And here. * t/perf/testsuite-recheck.sh: And here. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 19c56b1d 2012-06-22T22:31:35 tests: new requirement 'grep-nonprint' * t/ax/test-inist.sh ($esc): New, a literal escape character. (grep-nonprint): New requirement, check that the grep implementation available that can handle non-printing characters correctly. * t/color.sh: Use it instead of hand-rolled equivalent, and do not (re)define '$esc' explicitly. * t/color2.sh: Likewise. * t/tap-color.sh: Likewise. * t/vtexi4.sh: Likewise. * t/parallel-tests-no-color-in-log.sh: Likewise. Also prefer the use of grep over that of $FGREP, because the new requirement only check grep, and ensure we only grep non-printing characters from a pipe, to avoid hitting a BSD grep limitation. * t/parallel-tests-reset-term.sh: Likewise. * t/ax/tap-summary-aux.sh: Use '$esc' instead of hard-coding the literal escape character. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 3b83966e 2012-06-22T18:55:01 Merge branch 'maint' * maint: tests: prefer using 'is_newest' over 'ls -t' hacks tests: implement is_newest as an auxiliary script, not shell function tests: fix some spurious failures in VPATH setup tests: work in VPATH setup again
Stefano Lattarini dc18830f 2012-06-22T16:18:28 tests: implement is_newest as an auxiliary script, not shell function This will allow to also use it in the makefile recipes used in our test cases. * t/ax/test-init.sh (is_newest): Remove. * t/ax/is_newest: New script. * Makefile.am (EXTRA_DIST): Add it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 062aeb24 2012-06-22T14:52:21 Merge branch 'maint' * maint: tests: automatic re-execution works for non-POSIX shells too tests: use more POSIX shell features our test scripts + Extra non-trivial edits: * t/ax/test-init.sh: Apply the diffs between past versions of 'defs' in maint and in master; i.e., remove handling of $required entries 'texi2dvi-o' and 'makeinfo-html', and adjust to the fact that the parallel testsuite harness is now the default. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 3a1a8c6d 2012-06-22T11:40:39 tests: automatic re-execution works for non-POSIX shells too Some inferior shells are too greedy in parsing their input. If a non-POSIX Bourne shell (like Solaris 10 /bin/sh) was used to launch one of our test scripts, it would fail unconditionally, because it unexpectedly saw some (by it) unsupported constructs, notwithstanding such constructs being placed *after* the code implementing automatic test re-execution with a better shell. In conclusion, the shell bailed out like this: $ /bin/sh t/ar.sh $ t/ar.sh: syntax error at line 257: `is_newest_files=$' unexpected By moving all the potentially problematic code in a separate file, to be sourced only after the code for automatic re-execution with a better shell, we ensure that inferior shell cannot see such code by mistake. * defs: All code after automatic shell re-execution moved out ... * t/ax/test-init.sh: ... to this new file. * syntax-checks.mk (xdefs): Add it. * Makefile.am (dist_check_DATA): Add it. Also move in 'defs' from a less explicit 'check_DATA' declaration. (nodist_check_DATA): Move in 'defs-static' from a less explicit 'check_DATA' declaration. (check_DATA): Remove. * t/self-check-sanity.sh: Remove, it was actually too hacky and brittle, sanity-checking situations we don0t actually care about. * t/list-of-tests.mk: Adjust. * t/self-check-explicit-skips.sh: Adjust, and fix a botched heading comments while we are at it. * t/self-check-reexec.tap: Adjust. * t/self-check-cleanup.tap: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 6e713425 2012-06-14T13:13:58 tests: use more POSIX shell features our test scripts Since commit 'v1.12-36-g2d68fd9' of 2012-05-07, "configure: search a sturdy POSIX shell to be used in the testsuite", the shell running our test script is assured to be a POSIX-conforming shell, so we can use the more modern and flexible idioms and features that we couldn't use when we also aimed at compatibility with non-POSIX Bourne shells, like Solaris /bin/sh. * t/README: Suggest to use POSIX shell features liberally in test cases, with possible exception of Makefile recipes and configure shell code. * Several tests: Adjust to use more POSIX shell features; e.g., $(...) rather than `...`, $((...)) rather than `expr ...`, "if ! CMD; then ..." instead of "if CMD; then :; else ...", and so on. In several places, when using the 'test' built-in, prefer '-eq' over '=' for numeric comparisons, and prefer "grep -c PATTERN FILE" over "grep PATTERN FILE | wc -l". Throw in other low-hanging easy improvements and fixlets while we are at it. * t/ax/depcomp.sh, t/ax/tap-summary-aux.sh, t/ax/tap-functions.sh, defs, defs-static.in: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 5e771b27 2012-05-18T18:00:51 check: the parallel-tests driver is now the default The old serial testsuite driver is still supported, and can be activated by the 'serial-tests' option (introduced by commit 'v1.11-1989-gdeb7773' of 2012-02-20, "automake: new option 'serial-tests'"). * lib/Automake/Options.pm: Now, the 'parallel-tests' option is a no-op (but still explicitly recognized, for backward compatibility), while the 'serial-tests' option is recognized and registered. * automake.in: Adjust to ensure that the parallel test harness is used by default; mostly, this amounts at using "! option('serial-tests')" where "option('parallel-tests')" was used before. * NEWS, doc/automake.texi: Update. * configure.ac (AM_INIT_AUTOMAKE): Drop 'parallel-tests' option, is redundant now. * defs: In the generated AM_INIT_AUTOMAKE call, use the 'serial-tests' when the serial testsuite harness is to be used, and no option otherwise. * lib/am/check.am: Use (static) conditional '%SERIAL_TESTS%' instead of conditional '%PARALLEL_TESTS%'. * lib/test-driver: Adjust heading comments. * t/ax/extract-testsuite-summary.pl: Likewise. * t/check-exported-srcdir.sh: Likewise. * t/check-subst.sh: Likewise. * t/java-compile-run-nested.sh: Likewise. * t/parallel-tests-dry-run-1.sh: Likewise. * t/parallel-tests-dry-run-2.sh: Likewise. * t/parallel-tests-interrupt.sh: Likewise. * t/parallel-tests-many.sh: Likewise. * t/test-metadata-global-log.sh: Likewise. * t/tests-environment-backcompat.sh: Likewise. * m4/init.m4 (AM_INIT_AUTOMAKE): Adjust a comment. * t/check8.sh: Likewise. * t/check-tests-in-builddir.sh: Likewise. * t/java-compile-run-flat.sh: Adjust a botched comment. * t/ax/tap-summary-aux.sh: Remove explicit use of the 'parallel-tests' option. * t/ax/testsuite-summary-checks.sh: Likewise. * t/aclocal-path-precedence.sh: Likewise. * t/dist-aux-many-subdirs.sh: Likewise. * t/parallel-tests-no-color-in-log.sh: Likewise. * t/parallel-tests-reset-term.sh: Likewise. * t/parallel-tests-subdir.sh: Likewise. * t/check-no-test-driver.sh: Fix setup to adjust to the changed semantics. Update heading comments. * t/parallel-tests-driver-install.sh: Adjust to the new semantic. * t/test-driver-is-distributed.sh: Simplified according to the new semantics. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 7b142a7b 2012-05-15T16:12:09 tests: use 'parallel-tests' Automake option by default This will help our transition from 'serial-tests' to 'parallel-tests' as the default test suite driver enabled by a TESTS assignment in the input Makefile.am. Note that that change of default will only take place in master, though. * defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE invocation in the created 'configure.ac' stub, unless the variable 'am_serial_tests' is set to "yes". Don't pay attention anymore to the 'am_parallel_tests' variable, that's obsolete now. * defs-static.in: Warn if the 'am_serial_tests' variable is set in the environment; conversely, don't warn anymore about 'am_parallel_tests' being set in the environment. * Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests' variable instead of the now-obsolete 'am_parallel_tests' one. * syntax-checks.mk (sc_tests_obsolete_variables): Also warn against uses of 'am_parallel_tests', which is now deprecated in favor of 'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen, suggest using 'am_serial_tests' instead, not 'am_parallel_tests'. * gen-testsuite-part: Now that we use the 'parallel-tests' by default in our tests, we need to completely change the logic and semantics of generation of sibling tests for those tests that check the Automake generated testsuite harness itself. Do that, and give a complete explanation of the new logic and semantics in the relevant comments. * t/README: Update. * Lots of test cases: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini b105d40d 2012-04-30T20:44:50 color-tests: coloring can be forced on non-ANSI terminals as well Before this change, colorization of testsuite output was suppressed whenever the terminal was recognized to be a "dumb" one, incapable of handling ANSI coloring (i.e., when the environment variable TERM had a value of "dumb"). This happened even when the AM_COLOR_TESTS variable was set to a value of "always". Such a behaviour was suboptimal and slightly confusing; in fact, if a user wants to force coloring of testsuite output that is being redirected to a regular file, he should be able to do so even if his terminal is not capable of handling ANSI colors -- in fact, such terminal wouldn't be involved with the testsuite output in any way, so why should it be allowed to influence it? Thus, we now enable coloring of test output whenever AM_COLOR_TESTS is set to "always", irrespective of the value of the TERM environment variable. * NEWS: Update. * lib/am/check.am [%?COLOR%] (am__tty_colors): Activate colorization of testsuite output whenever AM_COLOR_TESTS has the value of "always". * t/ax/tap-summary-aux.sh: Export the TERM environment variable to "dumb" when forcing colorization of the testsuite output; this should *not* prevent such colorization from taking place, and we want to check that this expectation really holds. * t/ax/testsuite-summary-checks.sh: Likewise. * t/color.sh: Likewise, and adjust some grepping checks. * t/tap-color.sh: Likewise. Also, remove redundant "make check" invocation since we are at it. * t/color2.sh: Likewise, and check that exporting TERM=dumb actually prevents testsuite output colorization when AM_COLOR_TESTS is unset. * t/parallel-tests-reset-term.sh: Relax, to prevent it from failing spuriously due to the new semantic. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 89ba2a73 2012-04-20T10:23:35 tests: fix spurious failure with non-ANSI terminals * t/ax/tap-summary-aux.sh: When checking colored testsuite output, be sure to export the TERM variable to the value "ansi"; otherwise the automake testsuite driver will not display colored output, not even if AM_COLOR_TESTS is exported to "always". Failure revealed by NixOS Hydra. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 8e02441b 2012-04-06T15:20:54 maint: no more make recursion in Automake's build system We finally merge testsuite-related part of the Automake build system with the top-level one. See also yesterday's commit 'v1.11-2124-ga4b0f4b', "maint: remove most recursion in automake's own build system". * t/gen-testsuite-part: Move ... * gen-testsuite-part: ... here, with required adjustments. * t/defs-static.in: Move ... * defs-static.in: ... here, with required adjustments. In particular ... ($testsrcdir, $testbuilddir): Removed. ($top_testsrcdir): Renamed ... ($am_top_srcdir): ... to this. ($top_testbuilddir): Renamed ... ($am_top_builddir): ... to this. * t/defs: Move ... * defs: ... here, and adjust as required. * t/Makefile.am: Merge ... * Makefile.am: ... in here, with related adjustments and simplifications. * bootstrap: Adjust as required. * syntax-checks.mk: Likewise. * .gitignore: Likewise. * t/ax/tap-setup.sh: Likewise. * t/get-sysconf.sh: Likewise. * t/help-multilib.sh: Likewise. * t/multlib.sh: Likewise. * t/parallel-tests2.sh: Likewise. * t/self-check-cleanup.tap: Likewise. * t/self-check-exit.tap: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-explicit-skips.sh: Likewise. * t/self-check-reexec.tap: Likewise. * t/self-check-sanity.sh: Likewise. * t/yacc-d-cxx.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Stefano Lattarini 8a5096d1 2012-04-06T13:12:25 tests: rename 'tests/' => 't/', '*.test' => '*.sh' When we (soon) convert the Automake testsuite to a non-recursive make setup, we'll have to fix the entries of $(TESTS) to be prepended with the subdirectory they are in; this will increase the length of $(TESTS), and thus increase the possibility of exceeding the command-line length limits on some systems (most notably, MinGW/MSYS). See automake bug#7868 for more information. Thus we rename the 'tests/' subdirectory to 't/', and each 'x.test' script in there to 'x.sh'; this way, the $(TESTS) entry 'foo.test' will become 't/foo.sh', which have the same number of characters. * tests/: Rename ... * t/: ... to this. * t/*.test: Rename ... * t/*.sh: ... to this. * t/.gitignore: Removed as obsolete. * t/defs: Adjust. * t/gen-testsuite-part: Likewise. * t/list-of-tests.mk: Likewise. * t/ccnoco.sh: Likewise. * t/ccnoco3.sh: Likewise. * t/self-check-cleanup.tap: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-me.tap: Likewise. * t/self-check-reexec.tap: Likewise. * README: Likewise. * bootstrap: Likewise * configure.ac: Likewise. * Makefile.am: Likewise. * .gitignore: Likewise. * syntax-check.mk: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
kmx.io     mail     discord kmxgit v0.4.0