|
72cae0a5
|
2012-02-19T14:53:53
|
|
tests: be stricter in determining whether "make -jN" works
* tests/parallel-tests3.test: With at least Solaris 10 CCS make,
"make -jN" does not fail, but merely prints a warning about
"DistributedMake -j option" being ignored. This was causing this
test case to fail spuriously. So we now use a stricter check in
determining whether make works in parallel mode. Since we are at
it, we try to cater also to Sun Distributed make, which understands
"-j N" but *not* "-jN".
|
|
|
e1606b32
|
2012-02-18T13:59:26
|
|
parallel-tests: fix another BSD parallel make issue
When BSD make is run in parallel mode, it apparently strips any
leading directory component from the automatic variable '$*' (of
course, against what POSIX mandates). This was causing FreeBSD 9.0
make and NetBSD 5.1 make to spuriously fail with automake-generated
test harnesses if subdir tests were present *and* make was being
run in parallel mode. This issue affected also the Automake own
testsuite.
* lib/am/check2.am (am__set_b): New internal variable.
(%OBJ%, %EXT%.log, %EXT%$(EXEEXT).log): Use it to work around
the described BSD make issue.
* tests/parallel-tests3.test: Enhanced to expose the bug.
* tests/parallel-tests-subdir.test: Enhance a little, since we
are at it.
* NEWS: Update.
|
|
|
641a5a4b
|
2012-02-16T10:46:23
|
|
maint: run "make update-copyright"
|
|
|
2a0534da
|
2011-12-23T21:01:59
|
|
tests: remove FIXME comments from tricks to pacify syntax checks
Originally, I myself had insisted on adding those "FIXME" comments
near every unusual construct or use of "creative quoting" whose
only purpose was to avoid false positive from out syntax checks.
I did that with the expectation that we could have easily and soon
added a whitelisting capability to our maintainer-specific syntax
checks. This hasn't been the case unfortunately, and now those
"FIXME" in the comments are just distracting noise, making it more
difficult to grep the test cases for stuff that could really use a
fix or an improvement. So just get rid of them.
* tests/instdir-ltlib.test: Remove that extra "FIXME".
* tests/instdir-prog.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests-dry-run.test: Likewise.
* tests/parallel-tests-console-output.test: Likewise.
* tests/parallel-tests-no-color-in-log.test: Likewise.
* tests/tap-realtime.test: Likewise.
* tests/test-trs-recover.test: Likewise.
* tests/tap-deps.test: Likewise.
|
|
|
3c45ea72
|
2011-10-20T23:20:54
|
|
Merge branch 'master' into testsuite-work
* master:
tests: fix spurious failures with "chatty" make implementations
tests: fix aclocal-print-acdir.test
tests: fix spurious failure on fast machines
tests: avoid spurious failure of 'uninstall-fail.test' on Solaris
tests: avoid spurious failure of 'uninstall-fail.test' on Cygwin
tests: avoid spurious failure in 'parallel-tests3.test'
tests: fix spurious failure with FreeBSD make and Yacc in VPATH
tests: fix spurious failure with autoconf 2.62
refactor: improve signature of 'check_directory' sub in automake
docs: avoid using colon character inside arguments of @pxref
|
|
|
9579333e
|
2011-10-20T23:10:03
|
|
Merge branch 'maint'
* maint:
tests: fix spurious failures with "chatty" make implementations
tests: fix spurious failure on fast machines
tests: avoid spurious failure of 'uninstall-fail.test' on Solaris
tests: avoid spurious failure in 'parallel-tests3.test'
tests: fix spurious failure with FreeBSD make and Yacc in VPATH
|
|
|
f5b69b8a
|
2011-10-18T21:05:24
|
|
tests: avoid spurious failure in 'parallel-tests3.test'
This fixes automake bug#8788.
* tests/parallel-tests3.test: To ensure that the serial run of
the dummy testsuite is still ongoing when the parallel run has
terminated, use `kill -0', not a bare `kill'. This will prevent
a testsuite crash on NetBSD 5.1, and a testsuite hang on FreeBSD
8.2. Also, since we are at it, try harder to avoid possible
hangs of the script in other unusual situations.
|
|
|
e00d6c98
|
2011-09-10T21:59:09
|
|
tests defs: more namespace-safeness
($using_tap): Renamed ...
($am_using_tap): ... to this.
($parallel_tests): Renamed ...
($am_parallel_tests): ... to this.
($test_prefer_config_shell): Renamed ...
($am_test_prefer_config_shell): ... to this.
($original_AUTOMAKE): Renamed ...
($am_original_AUTOMAKE): ... to this.
($original_ACLOCAL): Renamed ...
($am_original_ACLOCAL): ... to this.
* tests/defs, tests/defs-static.in, tests/gen-testsuite-part,
tests/README, and all callers: Adjusted.
* tests/Makefile.am (AM_TESTS_ENVIRONMENT): Adjust, and rewrite
to make future adjustments easier.
* tests/defs (funcs_file_): Renamed ...
(am_funcs_file): ... to this.
* Makefile.am (sc_tests_obsolete_variables): New maintainer
check.
(syntax_check_rules): Add it.
|
|
|
dffe171f
|
2011-08-03T15:46:02
|
|
Merge branch 'test-protocols' into tap-testsuite-work
|
|
|
eaac33fe
|
2011-07-01T11:12:31
|
|
parallel-tests: simplify testsuite summary
Prefer a more deterministic, "tabular" format for the testsuite
summary, always listing the numbers of passed, failed, xfailed,
xpassed, skipped and errored tests, even when these numbers are
zero. This simplify the logic of testsuite summary creation,
makes it more easily machine-parseable, and will probably allow
for easier addition of new kinds of test results in the future.
* lib/am/check.am (am__tty_colors_dummy): New make variable, to
reduce code duplication. Extracted from previous versions of
$(am__tty_colors), and extended by defining two new variables
`$mgn' and `$brg'.
[%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
(am__text_box): Delete, is not needed anymore.
($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
new testsuite summary format.
* NEWS: Update.
* tests/check10.test: Don't run with the parallel-tests harness
too, that makes no sense anymore.
* tests/color.test: Update and adjust.
* tests/color2.test: Likewise.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests6.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests-unreadable-log.test: Likewise.
* tests/parallel-tests-empty-testlogs.test: Likewise.
* tests/parallel-tests-log-override-recheck.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/parallel-tests-no-color-in-log.test: New test.
* tests/testsuite-summary-color.test: Likewise.
* tests/testsuite-summary-count.test: Likewise.
* tests/testsuite-summary-count-many.test: Likewise.
* tests/testsuite-summary-reference-log.test: Likewise.
* tests/testsuite-summary-checks.sh: New auxiliary script, used
by the new tests above.
* tests/extract-testsuite-summary: Likewise.
* tests/trivial-test-driver: Optimize for speed when there are
lots of of tests.
* tests/Makefile.am (EXTRA_DIST): Distribute them.
(testsuite-summary-color.log, testsuite-summary-count.log): Depend
on them.
(testsuite-summary-count-many.log): Depend on the auxiliary scripts
'trivial-test-driver' and 'extract-testsuite-summary'.
(TESTS): Update.
|
|
|
ae610ad3
|
2011-05-23T14:26:58
|
|
tests: tweak and improve tests on "Simple Tests" driver
* tests/check.test: Make grepping of generated Makefile.in
slightly stricter. Add trailing `:' command.
* tests/check2.test: Ensure verbose printing of captured make
output. Make grepping of generated Makefile.in and of make
output stricter. Add trailing `:' command.
* tests/check3.test: Likewise. Also, fix minor error in a
comment.
* tests/check4.test: Stricter grepping of make output. Use
`using_gmake' subroutine instead of copy&paste from defs.in
to determine whether $MAKE is GNU make. Add a trailing `:'
command.
* tests/check6.test: Add trailing `:' command.
* tests/check10.test: Prefer redirection `>' over `>>' when
creating new files. Ensure verbose printing of captured make
output.
* tests/check11.test: Likewise.
* tests/check7.test: Likewise. Make grepping of make output
stricter. Prefer "make distcheck" over a simple "make check"
in a VPATH build. Add trailing `:' command.
(a.c): Format following GNU coding standards.
(Makefile.am): Explicitly declare target `print-xfail-tests'
as PHONY. Distribute $(check_SCRIPTS), for distcheck.
* tests/parallel-tests.test: Prefer redirection `>' over `>>'
when creating new files.
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests8.test: Likewise. Make grepping of
automake stderr stricter.
* tests/check8.test: Likewise. Make formatting of created
*.c files more consistent with GNU Coding Standards.
* tests/check-subst-prog.test (configure.in): Break overly
long lines.
(pass-prog.c, xfail-prog.c): Format them more consistently
with GNU Coding Standards.
* tests/parallel-tests3.test: Use a shorter and clearer skip
message. Use the `unindent' subroutine to improve readability
of indented loop bodies. Simplify the hack used to please
maintainer-check.
* tests/parallel-tests5.test: Make formatting of created *.c
files more consistent with GNU Coding Standards. Use the
`unindent' subroutine to improve readability of indented loop
bodies.
* tests/parallel-tests10.test: Add blank line, for clarity.
Prefer trailing `:' over trailing `Exit 0', for consistency
with other tests.
|
|
|
4f73c9f7
|
2010-06-03T00:30:03
|
|
testsuite: use `skip_' for skipping of tests
* tests/defs: Use the `skip_' subroutine for test skipping. Also
give sometimes more detailed messages about the reasons for the
skipping.
* tests/canon7.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/compile2.test: Likewise.
* tests/dejagnu7.test: Likewise.
* tests/depcomp6.test: Likewise.
* tests/depcomp9.test: Likewise.
* tests/depcomp10.test: Likewise.
* tests/distlinks.test: Likewise.
* tests/distlinksbrk.test: Likewise.
* tests/fn99.test: Likewise.
* tests/fn99subdir.test: Likewise.
* tests/forcemiss2.test: Likewise.
* tests/fort5.test: Likewise.
* tests/gettext3.test: Likewise.
* tests/install2.test: Likewise.
* tests/instfail-info.test: Likewise.
* tests/instfail-java.test: Likewise.
* tests/instfail-libtool.test: Likewise.
* tests/instfail.test: Likewise.
* tests/instmany-mans.test: Likewise.
* tests/instmany-python.test: Likewise.
* tests/instmany.test: Likewise.
* tests/instsh3.test: Likewise.
* tests/ltinit.test: Likewise.
* tests/makej2.test: Likewise.
* tests/mdate6.test: Likewise.
* tests/mkinst3.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests-reset-term.test: Likewise.
* tests/parallel-tests-unreadable-log.test: Likewise,
* tests/python-virtualenv.test: Likewise.
* tests/remake-gnulib-remove-header.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/symlink2.test: Likewise.
* tests/tar.test: Likewise.
* tests/tar2.test: Likewise.
* tests/txinfo26.test: Likewise.
* tests/vala2.test: Likewise.
* tests/vala3.test: Likewise.
* tests/vala5.test: Likewise.
* tests/vtexi4.test: Likewise.
* tests/instdir-texi.test: Likewise.
* tests/txinfo21.test: Likewise.
SquashMe
|
|
|
79103003
|
2011-01-08T19:40:04
|
|
tests: enable 'errexit' shell flag by default.
* tests/defs: Enable `errexit' shell flag (near the end).
Removed redundant comment about the enabling of shell traces.
* tests/README (Writing test cases): Update, and use nicer
formatting in a couple of places.
* All tests: Adjusted by removing now-redundant calls to
'set -e'.
|
|
|
a7d14dfb
|
2011-01-02T15:49:16
|
|
Tests: normalize use of the 'errexit' shell flag.
* tests/maken3.test (check_targets): Remove redundant call to
'set -e'.
* tests/maken4.test: Likewise.
* tests/ansi5.test: Call 'set -e' just after './defs' has been
sourced.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/cond16.test: Likewise.
* tests/cond17.test: Likewise.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond20.test: Likewise.
* tests/cond21.test: Likewise.
* tests/instdat2.test: Likewise.
* tests/instdir-texi.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/remake1a.test: Likewise.
* tests/ccnoco.test: Likewise, and add trailing `:' command.
* tests/comment4.test: Likewise.
* tests/gcj4.test: Likewise.
* tests/nodist2.test: Likewise.
* tests/nodist3.test: Enable 'errexit' shell flag (this should
have been done in commit v1.11-248-g317e17b, but the relevant
hunk has been forgotten somehow).
* tests/output.test: Likewise.
* tests/gnits2.test: Likewise, and display captured stderr to
script's stderr, not to script's stdout.
* tests/gnits3.test: Likewise. Also, prefer 'cat' over 'echo'
to append to Makefile.am, and really check that the exit status
of "make installcheck" indicates failure.
|
|
|
03449117
|
2010-03-04T20:22:17
|
|
Avoid generation of `tests/defs-p' file
* tests/defs.in: In the generated `configure.in' snippet: call
`AM_INIT_AUTOMAKE' with the `parallel-tests' option if the shell
variable `parallel_tests' is set to `yes'.
* tests/Makefile.am (defs-p): Target removed.
(check_SCRIPTS): Removed `defs-p'.
(clean-local-check): Do not unlink `defs-p' anymore.
($(parallel_tests)): Transformation rules for the test scripts
adjusted.
* tests/gen-parallel-tests: Selection rules for the test
scripts adjusted.
* tests/parallel-tests.test: Set `$parallel_tests' to `yes'
then include `./defs' (rather than simply including `./defs-p').
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests4.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/parallel-tests6.test: Likewise.
* tests/parallel-tests7.test: Likewise.
* tests/parallel-tests8.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests10.test: Likewise.
* tests/README (Section "Writing Test Cases", subsection "Do"):
Adjusted the parts referring to tests checking `parallel-tests'
behaviour. Some other minor related improvements.
* tests/.gitignore (defs-p): Removed.
|
|
|
f923b46e
|
2010-03-04T21:09:13
|
|
Remove redundant unset of variable TESTS from some test scripts.
* tests/color.test: Do not unset the `TESTS' variable, as it's
already unset in the `defs' file.
* tests/check5.test: Likewise.
* tests/check8.test: Likewise.
* tests/check9.test: Likewise.
* tests/check10.test: Likewise.
* tests/check11.test: Likewise.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests4.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/parallel-tests6.test: Likewise.
* tests/parallel-tests7.test: Likewise.
|
|
fcf2f560
|
2009-05-17T15:32:40
|
|
Revert Automake license to GPLv2+.
Automake will move to GPLv3+ once the Exception statement has
been rewritten to use the new GPLv3 exception language. This
change does not impact the COPYING file that may be installed
by `automake --add-missing'.
* COPYING: Revert to GPLv2. All uses changed.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
|
|
|
e37e045a
|
2009-04-19T17:57:23
|
|
testsuite: minor w32 fixes.
* tests/depend5.test: Don't fail if the system or file system
cannot handle weird characters, instead just try a few.
* tests/parallel-tests3.test: Really wait until the serial test
has finished, before letting the cleanup trap do its work.
Report by Bruno Haible.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
|
|
|
1cf48284
|
2009-04-14T23:09:15
|
|
testsuite: parallel make fixes.
This patch fixes a couple of testsuite bugs exposed with
`MAKE=make\ -jN make check'.
* tests/libtoo10.test: Do not run `clean' in same $MAKE
invocation as `all check'. Fixes test failure with parallel
NetBSD `make -jN'.
* tests/nobase.test: Be sure to create a directory before
creating files in it. Fixes test failure with MAKE=`make -jN'
for NetBSD make. This issue is hidden with parallel GNU make
due to its parallel breadth first update order.
* tests/parallel-tests3.test: Skip if $MAKE contains `-j',
GNU make will use the environment variable $MAKE for recursion
and thus run in parallel even if `$MAKE -j1' was used on the
command line in the test. Also, after running the test proper,
wait long enough so that background jobs have finished and there
are no open files left when the cleanup code runs.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
|
|
|
64269997
|
2009-03-10T21:33:47
|
|
Use append mode to capture parallel test output.
* tests/lisp8.test: Use append mode for output from `make -j',
to avoid dropped lines.
* tests/parallel-tests3.test: Likewise.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
|
|
|
099f454e
|
2009-03-08T14:05:56
|
|
New tests for `parallel-tests'.
* tests/parallel-tests.test: New file, to expose a bunch of
features of `parallel-tests': VERBOSE, clean, TEST_SUITE_LOG,
test dependencies, DISABLE_HARD_ERRORS.
* tests/parallel-tests2.test: New file, test check-html.
* tests/parallel-tests3.test: New file, test concurrency.
* tests/parallel-tests4.test: New file, test suffix rules.
* tests/parallel-tests5.test: New file, demonstrate compile/test
concurrency.
* tests/defs.in: Add a `required' check for rst2html.
* tests/Makefile.am: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
|