|
88b04360
|
2013-05-29T13:55:48
|
|
cosmetics: tiny whitespace fixlets
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
3fd6f164
|
2013-05-26T20:51:57
|
|
test-lib: typofix in comments
* t/ax/am-test-lib.sh: In here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
2ea48f45
|
2013-05-26T12:42:04
|
|
tests: adjust or remove some obsolete comments
* t/ax/am-test-lib.sh: Here.
* t/distcheck-configure-flags.sh: And here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
74c1336d
|
2013-05-24T20:54:41
|
|
tests: rename $am_make_rc_got -> $am_make_rc
The latter is more natural More importantly, I keep misspelling
the former over and over.
* t/ax/am-test-lib.sh (run_make): Update.
* t/ax/tap-summary-aux.sh: Adjust.
* t/ax/testsuite-summary-checks.sh: Likewise.
* t/parallel-tests-exit-statuses.sh: Likewise.
* t/parallel-tests-extra-programs.sh: Likewise.
* t/parallel-tests-fd-redirect-exeext.sh: Likewise.
* t/parallel-tests-fd-redirect.sh: Likewise.
* t/parallel-tests-fork-bomb.sh: Likewise.
* t/parallel-tests-no-spurious-summary.sh: Likewise.
* t/parallel-tests-recheck-pr11791.sh: Likewise.
* t/parallel-tests-reset-term.sh: Likewise.
* t/tap-signal.tap: Likewise.
* t/test-driver-acsubst.sh: Likewise.
* t/test-driver-cond.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
d19eb85c
|
2013-05-24T01:13:40
|
|
tests: use append mode to capture parallel make output
This will allow us to run the Automake testsuite forcing all the make
invocations in the test cases to run in parallel mode [1], but without
hitting the spurious failure reported in automake bug#11413.
See also:
<http://lists.gnu.org/archive/html/bug-make/2013-05/msg00135.html>
The make invocations in the test cases can be forced to run in parallel
mode by, e.g., invoking "make check" like this:
make check AM_TESTSUITE_MAKE="make -j16"
The possible spurious failures hinted above are due to the fact that
the redirected output of parallel make can racily lose lines. For
example, if GNU make (3.82) is run with -j10 on a Makefile like this:
all = 0 1 2 3 4 5 6 7 8 9
default: $(all)
$(all):
@sleep 0.$$(($RANDOM % 10)); echo $@
and has its standard output redirected in overwrite-mode to a regular
file, it looses a line of that output every 15 runs or so on a Fedora
17 system with 64 ppc64 cores. Redirection in append mode does not
suffer of this problem, as explained here:
<http://lists.gnu.org/archive/html/automake-patches/2009-03/msg00073.html>
See also previous commits v1.12-63-g45c1fcd of 2012-05-05 (tests: use
append mode to capture parallel make output) and Release-1-10-280-g6426999
of 2009-03-10 (Use append mode to capture parallel test output).
* t/ax/am-test-lib.sh (run_make): Adjust and enhance.
* t/lisp8.sh: Take advantage of the enhancement, nd stop doing output
redirection for $MAKE by hand.
* t/tap-more.sh: Likewise.
* t/parallel-tests-concurrency.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
15996acc
|
2013-05-22T13:00:48
|
|
tests: remove some code duplication
* t/ax/am-test-lib (null_install): New function.
* t/instdir-java.sh: Use it instead of copied & pasted code.
* t/instdir-lisp.sh: Likewise.
* t/instdir-ltlib.sh: Likewise.
* t/instdir-prog.sh: Likewise.
* t/instdir-python.sh: Likewise.
* t/instdir-texi.sh: Likewise.
* t/instdir.sh: Likewise.
* t/instdir2.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
c7cfa093
|
2013-05-21T20:22:17
|
|
tests: run_make: options to do command redirection
Let's improve the API of the 'run_make()' helper shell function by
adding three new options:
-O Save the standard output from make on disk, in a regular file
named 'stdout'.
-E Save the standard error from make on disk, in a regular file
named 'stderr'.
-M Save both the standard output and standard error from make on
disk, in a regular file named 'output'. This option supersedes
both the '-O' and '-E' options.
This new API has two main advantages.
1. Its use will allow us to get rid of more cumbersome idioms
like, e.g.,
$MAKE check >stdout && { cat stdout; exit 1; }
cat stdout
That can now be substituted with a simpler one:
run_make -e FAIL -O check
2. More importantly, using the new API we will prevent any extra output
from the shell traces of the code in run_make to be redirected along
with the make stderr (where that was redirected). This problem was
present in usages like, e.g.,
run_make TESTS=foo.test check 2>stderr && exit 1
grep 'expected error message' stderr
Such usages are now to be rewritten as follows:
run_make -e FAIL -E TESTS=foo.test check
grep 'expected error message' stderr
ensuring that 'stderr' won't end up containing unrelated stuff.
Note that we do not convert in bulk the old idioms and the use of
redirected 'run_make' invocations with this patch. We only convert
some occurrences, to ensure that the new implementation of 'run_make'
is sound enough. More sweeping conversions will likely be done in
follow-up patches.
* t/ax/am-test-lib.sh (run_make): Enhance and implement the extended API.
* t/tap-xfail-tests.sh: Use the new 'run_make' API.
* t/test-driver-cond.sh: Likewise.
* t/tests-environment-fd-redirect.sh: Likewise.
* t/uninstall-fail.sh: Likewise.
* t/yacc-dist-nobuild.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
ffd4ecc9
|
2013-05-20T11:23:01
|
|
tests: better idiom to override make macro defs on the cmdline
We now do so with a new wrapper shell function 'run_make()', which
relies on the use of AM_MAKEFLAGS, avoiding the use of the '-e' make
option. The use of that option (that causes the environment variables
to take precedence over the macro definitions in the Makefile) has
proved to be quite brittle in the past, causing annoying and sometimes
problematic spurious failures. This has worsened in some still
unpublished work going on in development branches. It's time to solve
the issue once and for all.
Note that we do not convert all uses of $MAKE in the testsuite right
away; we might do so in follow-up changes, with leisure, to avoid a
"patch bomb" effect (this commit is already too much of a bomb itself).
What we do in this commit is to get rid of all "$MAKE -e" invocations.
We admit that the implementation and feature-set of 'run_make()' are
far from perfect, but good enough for our current purposes. We'll
improve 'run_make()' if and when the need arises.
* syntax-checks.mk (sc_tests_no_make_e): New syntax check, guard against
the use of "$MAKE -e".
(syntax_check_rules): Add it.
(sc_tests_overriding_macros_on_cmdline): Adjust.
(lint): New, alias for 'maintainer-check', for lazy typists. Idea
backported from the 'maint' branch (Automake 1.13a).
* t/ax/am-test-lib.sh (run_make): New function. Run $MAKE with the
given command-line arguments, handling command-line override of variable
definitions in a smart way (using AM_MAKEFLAGS if a non-GNU make
implementation is detected to be in use).
(useless_vpath_rebuild): Adjust to use 'run_make', to avoid a spurious
maintainer check failure.
(yl_distcheck): Use 'run_make' rather than bare '$MAKE'.
(single_quote, append_single_quoted, is_valid_varname): New auxiliary
function, used, directly or indirectly, by it.
* Many tests: Adjust to avoid the use of "$MAKE -e", and prefer the
use of 'run_make' in few other contexts as well, where it makes sense.
Other minor fixlets while at it.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
12cfd38f
|
2013-05-20T15:17:11
|
|
test-lib: minor style changes
* t/ax/am-test-lib.sh (require_tool): Here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
8a39c6de
|
2013-05-20T15:07:48
|
|
test-lib: fix botched function name in an error message
* t/ax/am-test-lib.sh (useless_vpath_rebuild): Here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
84e12c32
|
2013-05-17T13:45:44
|
|
tests: sanitize 'unset' usages
In some shells (e.g., Solaris 10 /bin/ksh, or NetBSD 5.1 /bin/sh),
"unset VAR" returns a non-zero exit status in case the VAR variable
is already unset. This doesn't interact well with our usage of
"set -e" in the testsuite. So far, we've avoided spurious failures
by either explicitly ignoring the exit status from unset:
unset VAR || :
or explicitly ensuring that a variable is set, before trying to
unset it:
VAR=; unset VAR
But we can do better, by aliasing the 'unset' command to a custom
function that will take care of these details for us. This will
avoid us annoying spurious failures in the future, failures that
have already bitten us too much times. For an example, refer to
commit 'v1.12.2-88-g5b1dae5' of 2012-08-05 (tests: avoid tons of
spurious failures on NetBSD).
* t/ax/test-lib.sh (_am_unset): New function.
(unset): New alias to it.
(_am_exit): Adjust comments.
* t/ax/am-test-lib.sh: No need to temporary disable the 'errexit'
shell flag when unsetting variables that are potentially already
unset.
(am_process_requirements): Adjust to remove a now-useless
workaround related to unset.
* t/aclocal-macrodir.tap: Likewise.
* t/aclocal-macrodirs.tap: Likewise.
* t/auxdir-autodetect.sh: Likewise.
* t/ax/am-test-lib.sh: Likewise.
* t/ax/test-lib.sh: Likewise.
* t/check-tests-in-builddir.sh: Likewise.
* t/dist-formats.tap: Likewise.
* t/distcheck-configure-flags-am.sh: Likewise.
* t/distcheck-configure-flags.sh: Likewise.
* t/java-empty-classpath.sh: Likewise.
* t/javaflags.sh: Likewise.
* t/lflags.sh: Likewise.
* t/lflags2.sh: Likewise.
* t/lisp-flags.sh: Likewise.
* t/lisp6.sh: Likewise.
* t/missing-auxfile-stops-makefiles-creation.sh: Likewise.
* t/parallel-am.sh: Likewise.
* t/parallel-am2.sh: Likewise.
* t/parallel-am3.sh: Likewise.
* t/parallel-tests-log-override-recheck.sh: Likewise.
* t/pkg-config-macros.sh: Likewise.
* t/python-missing.sh: Likewise.
* t/python-too-old.sh: Likewise.
* t/python11.sh: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-report.sh: Likewise.
* t/self-check-seq.tap: Likewise.
* t/silent-configsite.sh: Likewise.
* t/suffix6c.sh: Likewise.
* t/tar-override.sh: Likewise.
* t/tests-environment-and-log-compiler.sh: Likewise.
* t/vala-configure.sh: Likewise.
* t/werror3.sh: Likewise.
* t/yflags-cmdline-override.sh: Likewise.
* t/yflags.sh: Likewise.
* t/yflags2.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
4348577b
|
2013-05-15T22:05:21
|
|
vala tests: skip in a cross compiler setup
The "vala + pkg-config" doesn't interact well with cross-compilation:
<https://mail.gnome.org/archives/vala-list/2012-September/msg00094.html>
That is causing annoying failures in our testsuite when it's run in
cross-compilation mode. It's not worth trying too be overly smart and
work around these problems; just skip the affected tests instead.
* t/ax/am-test-lib.sh (require_tool): A new requirement 'valac', telling
that the test needs to compile Vala-generated C files. In particular,
this causes the test to be skipped when using a cross-compiler.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
43e25e38
|
2013-02-14T11:46:58
|
|
tests: remove obsolete (and now wrong) comment
* t/ax/am-test-lib.sh: Here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
7df8b28c
|
2012-12-31T18:18:37
|
|
maint: update copyright year for 2013 (in branch maint)
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
de736a22
|
2012-12-14T10:43:21
|
|
Merge branch 'maint'
* maint:
docs: fixlet about xz compression levels and command line options
maint: more files excluded in 'update-copyright'
maint: adapt 'update-copyright' recipe to the new $(FETCHFILES) format
HACKING: update instructions to update the manuals at www.gnu.org
maint: add an explicative comment in Makefile
maint: factor out the name of the dir where web manuals are generated
maint: rename rules to build/upload web manuals
cosmetic: minor variable and rules reordering in Makefile.am
docs: build the web manuals for CVS in the builddir, not in the srcdir
release: automate uploading of web manuals in CVS
docs: AM_CFLAGS: remove reference to non-existing "more detailed" desc
tests: avoid two spurious failures on FreeBSD
tests: improve 'pkg-config' requirement
|
|
|
7a4e1937
|
2012-12-13T09:46:18
|
|
tests: improve 'pkg-config' requirement
This improves (and unifies among different tests) the detection of
both the 'pkg-config' program and m4 macros provided by it; they are
needed in particular by tests on Vala support.
Before this, some Vala-related tests were failing spuriously when the
pkg-config program was available but the macros from pkg.m4 were not
made explicitly available in ACLOCAL_PATH. These tests were:
vala-mix vala-mix2 vala-parallel vala-vapi vala-vpath
See also similar commit 'v1.11-581-gb7d67d5' of 2011-12-22,
"tests: better handling of gettext and libtool requirements".
* t/ax/am-test-lib.sh (process_requirements): If a requirement
matches the 'pkg-config*' wildcard, try to make available to it
the pkg-config m4 macros, and skip it if it's not possible.
This is done by sourcing the 'get.sh' fragment generated by ...
* t/pkg-config-macros.sh: ... this new test, which looks in
advance for the 'pkg-config' program and the m4 macros that
comes with it.
* t/list-of-tests.mk: Add the new test.
* gen-testsuite-part (%deps_extractor): Tests requiring pkg-config
will need the m4 macros coming with it; so these tests should be
run after the 'pkg-config-macros.sh' test. Automatically generate
proper declarations for such dependencies.
* t/vala-header.sh: No need to explicitly check that the macro
PKG_CHECK_MODULES has been expanded by autoconf.
* t/vala-libs.sh: Likewise.
* t/vala-non-recursive-setup.sh: Likewise.
* t/vala-per-target-flags.sh: Likewise.
* t/vala-recursive-setup.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
cb6a19d9
|
2012-11-21T14:13:02
|
|
Merge branch 'maint'
* maint:
tests: fix a spurious failure when $PYTHON is in the environment
python tests: support PEP-3147 installation layout
python: uninstall cater to PEP-3147
tests: improve a comment
tests: honour $PYTHON override
tests: typofix in message
news: document fix for bug#8847 (PEP-3147, __pycache__)
python: improve support for modern python (CPython 3.2 and PyPy)
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
e1acf4a0
|
2012-11-19T22:19:21
|
|
python tests: support PEP-3147 installation layout
This fixes several spurious testsuite failures with python >= 3.2,
introduced by recent commit v1.12.4-43-ge0e99ed, "python: improve
support for modern python (CPython 3.2 and PyPy)".
* t/ax/am-test-lib.sh (python_has_pep3147, pyc_location, py_installed):
New functions.
($am_pep3147_tag): New variable.
* t/py-compile-basic2.sh: Deleted, too difficult to adapt for the
gain it would offer; move the still relevant parts ...
* t/py-compile-basic.sh: ... here. Adapt and adjust the rest of
the test as well.
* t/nobase-python.sh: Adapt and adjust.
* t/py-compile-basedir.sh: Likewise.
* t/py-compile-destdir.sh: Likewise.
* t/py-compile-option-terminate.sh: Likewise.
* t/python-pr10995.sh: Likewise.
* t/python-virtualenv.sh: Likewise.
* t/python10.sh: Likewise.
* t/python12.sh: Likewise.
* t/python3.sh: Likewise.
* t/list-of-tests.mk: Adjust list of tests.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
cae0d860
|
2012-11-11T12:39:40
|
|
tests: improve a comment
* t/ax/am-test-lib.sh (require_tool): Here, about python support
for '--version' option.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
c3dc9003
|
2012-11-11T12:03:46
|
|
tests: honour $PYTHON override
* t/ax/am-test-lib.sh (require_tool): Here.
* t/python-too-old.sh: And here.
* t/python-vars.sh: And here.
* t/python-virtualenv.sh: And here. Also add some sanity
checks while at it.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
2a7c9aea
|
2012-08-05T17:45:07
|
|
Merge branch 'maint'
* maint:
tests: avoid spurious failures with older Texinfo
tests: avoid tons of spurious failures on NetBSD
runtest: avoid spurious failures on NetBSD
news: dependency tracking for Portland Group Compilers is now supported
|
|
|
5b1dae57
|
2012-08-05T12:14:19
|
|
tests: avoid tons of spurious failures on NetBSD
* t/ax/am-test-lib.sh (process_requirements): Set the '$am_tool' variable
to the empty string before trying to unset it; otherwise, we might be
attempting to unset an already-unset variable, which (together with the
presence of the 'errexit' shell flag) causes spurious failures at least
with the /bin/sh shell from NetBSD 5.1. This was actually causing the
great majority of the Automake tests (all those not using a "required=..."
declaration) to fail spuriously on that platform!
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
106428fe
|
2012-07-26T18:16:47
|
|
Merge branch 'maint'
* maint: (38 commits)
maintcheck: fixup list of files in $(xdefs)
tests: never source test-defs.sh directly, source test-lib.sh instead
runtest: sanitize test environment
tests: remove an obsolescent self test
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
tests: protect test libs against multiple inclusion
configure: testsuite shell can return early from "dot-sourced" files
tests: move sanitization and "Bournification" in the generic test lib
tests: source test defs in the generic test lib
test defs: no need to re-add $srcdir/t/ax to $PATH
tests: split test libs into "generic" and "automake-specific"
test setup: move actual calling of testsuite setup in ./defs
test setup: merge definitions of function for simple tests
test init: refactor: new function 'am_test_setup'
test init: refactor: move displaying of debugging info later
test init: refactor: new function 'am_setup_testdir'
test init: refactor: new function 'am_set_exit_traps'
configure: testsuite shell set exit traps in shell functions
test init: refactor: new function 'am_exit_trap'
test init: refactor: new function 'process_requirements'
...
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
e82c9156
|
2012-07-26T10:35:09
|
|
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
And similarly, "am_using_tap=no" -> "am_test_protocol=none".
The new '$am_test_protocol' name is clearer, and will allow the easy
addition of further test protocols in the future. This is not truly
relevant for automake, but we are trying to make some parts of our
testsuite framework as general as possible, in view of a future move
to a more generic project like Gnulib.
* Makefile.am, t/ax/am-test-lib.sh, t/ax/test-defs.in, t/ax/test-lib.sh,
t/self-check-env-sanitize.tap, t/self-check-tap.sh: Adjust.
* syntax-checks.mk (sc_tests_obsolete_variables): Add 'am_using_tap' to
the list of obsolete variables to check against.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
d8b3c5e4
|
2012-07-26T12:21:40
|
|
tests: protect test libs against multiple inclusion
* t/ax/test-lib.sh, t/ax/am-test-lib.sh: Return early if already sourced.
Use the witness variables '$test_lib_sourced' and '$am_test_lib_sourced',
respectively, for this purpose.
* runtest.in, Makefile.am (AM_TESTS_ENVIRONMENT): Unset 'test_lib_sourced'
and 'am_test_lib_sourced', to avoid interferences from the environment.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
55fb7140
|
2012-07-25T22:49:02
|
|
tests: split test libs into "generic" and "automake-specific"
This is the first step in the quest to merge the generically useful
parts of our test suite framework in a more generic project, like
Gnulib. Time will tell if we'll succeed, and whether the success will
be worth the extra hassle.
* t/ax/test-init.sh: Split out ...
* t/ax/am-test-lib.sh, t/ax/test-lib.sh: ... into these two tests.
* defs, Makefile.am: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|