|
aecd1be6
|
2012-10-28T14:02:46
|
|
tests: can check our recipes avoid trailing backslashes
This is related to commit v1.11-1704-g254227b of 2012-05-01,
"parallel-tests: avoid trailing backslashes in make recipes",
and automake bug#10436.
Recipes with a trailing backslash character (possibly followed by
blank characters only) can cause spurious syntax errors with at
least older bash versions (e.g., bash 2.05b), and can be potentially
be unportable to other weaker shells.
So provide a target that runs the testsuite looking for this kind
of breakage (without requiring a real bugged shell).
* t/ax/shell-no-trail-bslash.in: New, a "shell" that chokes on '-c'
commands having a trailing '\' (possibly followed by whitespace only).
* Makefile (t/ax/shell-no-trail-bslash): Generate this script from it.
(noinst_SCRIPTS, CLEANFILES): Add it.
(EXTRA_DIST): Add 't/ax/shell-no-trail-bslash.in'.
(check-no-trailing-backslash-in-recipes): New target, runs the testsuite
with 'shell-no-trail-bslash' as the CONFIG_SHELL, to catch possible
recipes having a trailing backslash character (possibly followed by
* .gitignore: Update.
* t/self-check-shell-no-trail-bslash.sh: New testsuite self-check.
* t/parallel-tests-trailing-bslash.sh: Remove as obsolete.
* t/list-of-tests.mk: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
0610fc8e
|
2012-10-27T12:59:41
|
|
tests: prepare to move ./defs to t/ax/test-init.sh
We don't do this in a sweeping passage, because that would cause
endless headaches in the synchronization between the maint, master
and ng/master branches. Instead, we setup our framework to allow
test scripts to work by sourcing either './defs' or 'test-init.sh',
so that we'll be able to make the transition gradual and painless.
* t/ax/test-init.sh: New, copied from the previous ./defs file.
* defs: Simply work by sourcing the new file.
* Makefile.am (dist_noinst_DATA): List the new file.
* t/README: Adjust to mandate the sourcing of 'test-init.sh' rather
than of './defs'.
* t/c-demo.sh: Source 'test-init.sh' instead of ./defs. This is
done to verify our new setup actually works.
* t/ac-output-old.tap: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
af1ac94a
|
2012-08-13T18:00:57
|
|
maintcheck: fix spurious warnings
* t/distcheck-override-infodir.sh: Be sure that valid occurences
of the "aclocal" and "automake" strings, which can confuse the
'sc_tests_plain_automake' check, are protected by leading "#"
characters.
* t/ax/test-lib.sh: Always use '$(...)' for command subtitution,
to avoid triggering the 'sc_tests_command_subst' check; there was
still once place where `...` was used. While at it, fix a related
comment.
* t/ax/test-defs.in ($sleep): Use creative quoting to avoid
spuriously triggering the 'sc_tests_plain_sleep' check.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
5b1dae57
|
2012-08-05T12:14:19
|
|
tests: avoid tons of spurious failures on NetBSD
* t/ax/am-test-lib.sh (process_requirements): Set the '$am_tool' variable
to the empty string before trying to unset it; otherwise, we might be
attempting to unset an already-unset variable, which (together with the
presence of the 'errexit' shell flag) causes spurious failures at least
with the /bin/sh shell from NetBSD 5.1. This was actually causing the
great majority of the Automake tests (all those not using a "required=..."
declaration) to fail spuriously on that platform!
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
db188c29
|
2012-07-26T11:04:31
|
|
runtest: sanitize test environment
* runtest.in: Here, similarly to what is done by AM_TESTS_ENVIRONMENT
in Makefile.am, unset variables that should be under the complete control
of the test framework, and that could create havoc if inherited from the
environment. This remove the need to check against possible environment
"pollution" ...
* t/ax/test-defs.in: ... in here.
* Makefile.am (AM_TESTS_ENVIRONMENT): Add a comment about the need of
synchronization with 'runtest.in'.
* t/self-check-env-sanitize.tap: Remove as obsolete.
* t/list-of-tests.mk: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
e82c9156
|
2012-07-26T10:35:09
|
|
tests: "am_using_tap=yes" -> "am_test_protocol=tap"
And similarly, "am_using_tap=no" -> "am_test_protocol=none".
The new '$am_test_protocol' name is clearer, and will allow the easy
addition of further test protocols in the future. This is not truly
relevant for automake, but we are trying to make some parts of our
testsuite framework as general as possible, in view of a future move
to a more generic project like Gnulib.
* Makefile.am, t/ax/am-test-lib.sh, t/ax/test-defs.in, t/ax/test-lib.sh,
t/self-check-env-sanitize.tap, t/self-check-tap.sh: Adjust.
* syntax-checks.mk (sc_tests_obsolete_variables): Add 'am_using_tap' to
the list of obsolete variables to check against.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
d8b3c5e4
|
2012-07-26T12:21:40
|
|
tests: protect test libs against multiple inclusion
* t/ax/test-lib.sh, t/ax/am-test-lib.sh: Return early if already sourced.
Use the witness variables '$test_lib_sourced' and '$am_test_lib_sourced',
respectively, for this purpose.
* runtest.in, Makefile.am (AM_TESTS_ENVIRONMENT): Unset 'test_lib_sourced'
and 'am_test_lib_sourced', to avoid interferences from the environment.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
1ab1f546
|
2012-07-26T10:22:52
|
|
tests: move sanitization and "Bournification" in the generic test lib
* t/ax/test-defs.sh: From here ...
* t/ax/test-lib.sh: ... to here. Also move the initialization of
'$argv0' and '$me'.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
4fcf43cd
|
2012-07-26T10:09:04
|
|
tests: source test defs in the generic test lib
* t/ax/test-lib.sh: That is, here ...
* defs: ... rather than here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
7a42af01
|
2012-07-26T10:03:51
|
|
test defs: no need to re-add $srcdir/t/ax to $PATH
* t/ax/test-defs.sh: Here. That is already done by both 'runtest' and
AM_TESTS_ENVIRONMENT, and the presence of '$srcdir/t/ax' in $PATH is
anyway required, after the recent changes, for our testsuite framework
to work at all.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
55fb7140
|
2012-07-25T22:49:02
|
|
tests: split test libs into "generic" and "automake-specific"
This is the first step in the quest to merge the generically useful
parts of our test suite framework in a more generic project, like
Gnulib. Time will tell if we'll succeed, and whether the success will
be worth the extra hassle.
* t/ax/test-init.sh: Split out ...
* t/ax/am-test-lib.sh, t/ax/test-lib.sh: ... into these two tests.
* defs, Makefile.am: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
feab5c26
|
2012-07-25T21:12:30
|
|
test setup: move actual calling of testsuite setup in ./defs
* t/ax/test-init.sh: Move call to 'am_test_setup' ...
* defs: ... here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
3bb5a901
|
2012-07-25T21:04:30
|
|
test setup: merge definitions of function for simple tests
* t/ax/plain-functions.sh: Delete, moving the definitions of ...
($stderr_fileno_): ... this variable ...
(warn_, fail_, skip_, skip_all_, fatal_, framework_failure_): ... and
these functions ...
* t/ax/test-init.sh: ... in here. This allow us to use those functions
earlier in this file (instead of having to duplicate their behaviour
with inlined code). The TAP-based tests are still able to override these
functions later to their TAP-enhanced equivalents when 'tap-functions.sh'
is sourced.
* Makefile.am (dist_noinst_DATA): Remove 't/ax/plain-functions.sh'.
* t/ax/tap-functions.sh: Adjust a comment.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
472e4e47
|
2012-07-25T20:31:47
|
|
test init: refactor: new function 'am_test_setup'
* t/ax/test-init.sh (am_test_setup): Here.
Call it from the main code. Remove other calls of functions and
settings that are now duly called by 'am_test_setup'.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
22443fe7
|
2012-07-25T20:25:55
|
|
test init: refactor: move displaying of debugging info later
* t/ax/test-init.sh: Here, about the values of $PATH, $am_using_tap
and $am_running_installcheck.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
1f4d8c22
|
2012-07-25T20:20:09
|
|
test init: refactor: new function 'am_setup_testdir'
* t/ax/test-init.sh (am_setup_testdir): Here.
Use it instead of inlining its contents in the main code.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
d9686f93
|
2012-07-25T20:17:46
|
|
test init: refactor: new function 'am_set_exit_traps'
* t/ax/test-init.sh (am_set_exit_traps): Here.
(trap): Use it instead of inlining the cleanup/finalization code.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
7f3690db
|
2012-07-25T19:45:31
|
|
test init: refactor: new function 'am_exit_trap'
* t/ax/test-init.sh (am_exit_trap): Here.
(trap): Use it instead of inlining the cleanup/finialization code.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
1a915b9a
|
2012-07-25T19:41:57
|
|
test init: refactor: new function 'process_requirements'
* t/ax/test-init.sh (process_requirements): Here.
Use it in to check that the tools in $required are present.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
42bd480c
|
2012-07-25T19:23:29
|
|
test init: refactor: new function 'require_tool'
* t/ax/test-init.sh (require_tool): Here.
Use it in the loop in the prerequisites in $required.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
2bb312c8
|
2012-07-25T18:49:50
|
|
test init: remove a couple of paranoid sanity checks
* t/ax/test-init.sh: Here, about 'am_top_srcdir' and 'am_top_builddir';
the values of those variables are AC_SUBST'd, so it's nigh impossible
for the to be wrong or get "messed up".
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
ecbceb3e
|
2012-07-25T18:14:23
|
|
test defs: move in 't/ax/test-defs.sh'
We can do so with minimal churn, now that the early setup of $PATH in
both 'runtest' and AM_TESTS_ENVIRONMENT allow the '.' built-in to find
to-be-sourced shell scripts in 't/ax' automatically, both in in-tree
and VPATH builds.
With this change, we take another step forward a more rational and
"segregated" organization of our testsuite framework.
* defs-static.in: Rename ...
* t/ax/test-defs.in: ... like this, and adjust.
* Makefile.am, syntax-checks.mk, t/README, gen-testsuite-part,
t/tests-init.sh, t/ax/tap-setup.sh, t/tap-summary.sh, .gitignore,
t/testsuite-summary-color.sh, t/testsuite-summary-count.sh,
t/self-check-tap.sh, defs: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
42d9439d
|
2012-07-25T19:06:22
|
|
perf tests: reorganize
* t/list-of-tests.mk (handwritten_TESTS): Don't list ($perf_TESTS) in
here anymore.
* Makefile.am (EXTRA_DIST): Add them explicitly.
(perf): New target, run the performance tests and save the logs of
the ones with unexpected results in the file 't/perf/test-suite.log',
which is thus ...
(PERF_TEST_SUITE_LOG): ... defined in this variable ...
(CLEANFILES): ... and added to this.
(test_subdirs): Remove 't/perf': the tests in there are no more
listed in $(TESTS).
* t/ax/test-init.sh: Don't explicitly skip "perf" test here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
ea93d8d0
|
2012-07-07T12:08:05
|
|
tests: move 'runtest' into the top-level directory
Move our wrapper script 'runtest' (meant to allow the execution of
Automake test cases from the command line) from the 't/ax/' directory
to the top-level one. This makes the script easier to find and to
invoke. Much more importantly, our DejaGNU-checking test cases won't
try anymore to use that script instead of the 'runtest' program provided
by DejaGNU (that happened because '$(srcdir)/t/ax/' is automatically
added early to the $PATH variable in our test cases), which was causing
spurious SKIPs.
* t/ax/runtest.in: Move ...
* runtest.in: ... here.
* Makefile.am, t/README, .gitignore: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
5197aba1
|
2012-07-06T23:42:11
|
|
cosmetics: rename t/ax/test-runner => t/ax/runtest
The latter is shorter and clearer. Better to do the rename early,
before other developers or contributors begin to get used to the
'test-runner' name.
* t/ax/test-runner.in: Rename ...
* t/ax/runtest.in: ... like this.
* Makefile.am, t/README, .gitignore: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
b6e6409a
|
2012-07-06T12:24:33
|
|
tests init: don't automatically re-execute tests with a POSIX shell
I've unwittingly broken support for that feature *again* in some of
my recent testsuite tweaking. In this case, the re-execution code
works correctly when the tests are executed with a POSIX shells, but
breaks when they are invoked by an old-style Bourne shells (e.g.,
/bin/sh on Solaris).
It's time to face it: that feature is too much brittle, and too seldom
used (because the Makefile takes care of running the tests with the
correct shell anyway, so that a breakage is only experienced when
running the tests by hand). It just don't remain working for long, not
when we often touch the testsuite setup (which we are going to do again
when we'll try to move part of our testsuite framework to Gnulib, or a
similar project).
So, instead of trying to be extra-smart and automatically re-execute the
tests with the correct shell, we now offer a simple wrapper script that
the user can employ to run the test scripts with the proper shell. And
while we are at it, we write this wrapper to also deal with TAP tests in
a better way, running them through the prove(1) utility, so that their
results are correctly recognized and reported.
* t/ax/test-runner.in: New file; the wrapper script we were talking about.
* configure.ac (AC_CONFIG_FILES): Process it into 't/ax/test-runner'.
* .gitignore: Add 't/ax/test-runner'.
* defs: Remove code for automatic re-execution of the scripts with the
correct shell. This file now just a very thin layer around 'defs-static'
and 't/ax/test-init.sh'.
* t/README: Adjust, and remove or fix some imprecise or outdated text in
the process (like "... test scripts are written with portability in mind,
so that they should run with any decent Bourne-compatible shell ..." ).
* Makefile.am (AM_TESTS_ENVIRONMENT): No need to export 'AM_TESTS_REEXEC'
to "no" anymore.
* t/self-check-explicit-skips.sh: Likewise.
* t/self-check-exit.tap: Likewise.
* t/self-check-me.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-reexec.tap: Remove as obsolete.
* t/list-of-tests.mk: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
0a442c18
|
2012-07-01T11:50:54
|
|
tests setup: unset CDPATH
So that our test scripts can safely chdir around using relative
paths as well, without having to worry abut possible CDPATH
interferences.
* defs-static.in: unset CDPATH.
* t/ax/tests-init.sh: Remove a now-unneeded workaround.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
285eed2f
|
2012-07-01T11:01:02
|
|
tests setup: more namespace safeness
* t/ax/tests-init.sh ($testSubDir): Rename ...
($am_test_subdir): ... to this, throughout the file.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
c59c8d15
|
2012-07-01T10:36:34
|
|
tests setup: less hard-coding of the test subdirectory
* t/ax/tests-init.sh: Make the code creating the temporary
test subdirectory smart enough to automatically create it
in the same subdirectory of the test that is being run.
* defs-static.in ($MKDIR_P, $am_rel_srcdir): New variables,
AC_SUBST'd from @MKDIR_P@ and @srcdir@ respectively, and
used in the new 'tests-init.sh' code.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
bf258729
|
2012-06-29T00:48:11
|
|
tests: prefer "test ! -e FILE" to check that a file doesn't exist
Once, for the sake of (at least) Solaris 10 /bin/sh, we had to use
"test ! -f FILE" or "test ! -r FILE" or "test ! -d FILE" instead,
because the that shell's 'test' built-in didn't grok the '-e' option.
Note however that we still can't use "test ! -e" in the Makefile recipes
used in the test cases; that is because those recipes are run with the
shell detected by 'configure', and Autoconf-generated configure scripts
do no guarantee to find or provide a POSIX-compatible shell.
* Several tests: Adjust.
* t/yacc-clean-cxx: Adjust, and remove a couple of useless commands.
* t/parallel-tests-dry-run-2.sh: Adjust, and add invocation to
"make -n" forgotten in previous versions of the test.
* t/txinfo26.sh: Adjust, and don't bother to skip the test when it's
run in a directory whose absolute path contain whitespace: that setup
is not supported anyway.
* t/maken3.sh: Adjust, and fix a typo that could cause a minor false
negative.
* t/test-trs-recover2.sh: Prefer using 'skip_' with a suitable error
message over a bare 'exit 77'.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
d3a51faf
|
2012-06-28T21:52:37
|
|
tests: remove stale workarounds for Solaris /bin/sh
Our testsuite cannot be run with that shell anymore (as it is
not POSIX-compliant).
* t/tap-more.sh: Remove outdated workarounds for Solaris /bin/sh.
* t/self-check-exit.tap: Likewise.
* t/ansi2knr-no-more.sh: Likewise.
* t/add-missing.tap: Likewise.
* t/dist-auxfile.sh: Likewise.
* t/test-driver-custom-multitest-recheck2.sh: Likewise.
* t/ax/test-init.sh: Remove obsolete references to Solaris
/bin/sh.
* t/confh5.sh: Likewise.
* t/uninstall-fail.sh: Likewise. And update comments about
quirks of Solaris /bin/ksh and /usr/xpg4/bin/sh.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
07321606
|
2012-06-28T18:21:33
|
|
tests init: don't bother allowing '$me' to be overridable
We once used that feature in our wrapper tests; but now (and probably
even since commit 'v1.11-1308-g375f23d' of 2011-09-08, "testsuite:
revamp generation of autogenerated tests") it is not needed anymore.
By removing it we can simplify our growingly complex testsuite framework
a little.
* t/ax/test-inist.sh ($me): Do not initialize it here (and only if not
already set), instead ...
* defs-static.in ($me): ... initialize it here unconditionally.
Do not check anymore that $me doesn't come from the environment: that
wouldn't cause any problem now.
Now that '$me' is defined early, prefer it over 'argv0' in early error
messages, both here ...
* defs: ... and here.
* Makefile.am (AM_TESTS_ENVIRONMENT): Do not bother "nullifying" $me
anymore.
* t/self-check-env-sanitize.tap: Adjust.
* t/self-check-me.tap: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
b0df891f
|
2012-06-28T17:50:20
|
|
tests init: typofixes in comments
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
de20404a
|
2012-06-28T17:08:34
|
|
tests: don't skip if $(abs_builddir) or $(abs_srcdir) contain whitespace
We used to explicitly skip libtool and gettext tests if the absolute
path of the builddir or of the srcdir which Automake was configured
with contained any whitespace (or other metacharacters).
But several other tests would spuriously fail in such an unholy setup.
To be precise, it would cause 61 'FAIL's and 42 'ERROR's in the whole
Automake testsuite.
The fact that, as of today, nobody has reported any failure of this kind
means that (thankfully) nobody is building automake with $(abs_srcdir)
or $(abs_builddir) mangled by whitespace. So, instead of trying to cater
to such a broken setup consistently, we just drop the extra check in the
libtool/gettext tests.
In case someone will ever reports a failure due to extra whitespace in
either $(abs_srcdir) or or $(abs_builddir), we will simply enhance our
'configure.ac' to bail out flatly and loudly at such a setup.
* t/ax/test-init.sh: Simplify accordingly.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
9c906bc7
|
2012-06-28T16:43:43
|
|
tests: avoid spurious maintainer-check failures
* t/ax/test-init.sh: Here, by adding extra quoting for two
occurrences of the string "perl".
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
c2a52bf0
|
2012-06-24T10:36:15
|
|
tests: simpler workaround for shells losing the exit status in exit trap
Now that we can assume our tests are run by a decent POSIX shell, we
can simplify our workaround aimed at having the exit status propagated
correctly to the code in the exit trap. Unfortunately, we cannot
dispense with such a workaround altogether, because it's still required
by some shells we need to support (at least Solaris 10 /bin/ksh and
/usr/xpg4/bin/sh).
For more information about the need of that workaround, see the entry
about 'trap' in the section "Limitations of Shell Builtins" in the
Autoconf manual:
<http://www.gnu.org/software/autoconf/manual/autoconf.html#trap>
The new workaround has been tested successfully with the following
shells:
- Bash 4.1
- Bash 3.2
- Bash 3.0
- Bash 2.05b
- dash 0.5.5.1
- dash 0.5.2
- AT&T Ksh 93u (from official Debian package)
- MirBSD Korn Shell 40.2 (from official Debian package)
- Solaris 9, 10 and 11 /bin/ksh
- Solaris 9, 10 and 11 /usr/xpg4/bin/sh
- NetBSD 5.1 /bin/sh
- NetBSD 5.1 /bin/ksh
* t/ax/test-init.sh (Exit): Rename ...
(_am_exit): ... like this.
(exit): New alias for '_am_exit'. We cannot simply redefine 'exit'
as a shell function, because some shells (dash 0.5.5.1, Solaris 10
/bin/ksh and /usr/xpg4/bin/sh) do not allow it.
(_am_exit, trap): Add extra escaping for 'exit' calls, to ensure we
really invoke the 'exit' builtin and not our alias with the same
name.
* configure.ac: Check that the shell selected to run our testsuite
supports aliases named like shell builtins.
* t/REAMDE: Adjust.
* All tests: Adjust, by simply using 'exit' instead of 'Exit'.
* t/self-check-explicit-skips.sh: Adjust: the first usage of 'exit'
after it has been redefined as an alias must be on a new line w.r.t.
that where the alias is defined, in order for the redefinition to be
honored.
* syntax-checks.mk (sc_tests_Exit_not_exit): Delete.
(sc_tests_exit_not_Exit): New.
(syntax_check_rules): Adjust.
(sc_tests_automake_fails): Simplify the recipe a little.
* Several tests: Remove now useless spurious quoting once required
to placate the 'sc_tests_Exit_not_exit' maintainer check.
* gen-testsuite-part: Likewise. Also, avoid uses of 'Exit' in the
generated scripts.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
aa630a92
|
2012-06-24T17:57:10
|
|
tests: avoid several spurious failures on Solaris
* t/ax/is_newest: Rewrite to be Bourne-compatible, for /bin/sh shells like
Solaris' that are not POSIX-conforming. The script is so small that such
a rewrite is easier than going through the hoops that would be required to
ensure this script is always executed with a POSIX shell.
* t/ax/is: Add a comment stating that this script is to be kept Bourne
compatible as well.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
5898641f
|
2012-06-23T00:12:08
|
|
tests: avoid one last `...` command substitution in 'test-init.sh'
* t/ax/test-init.sh (me): In the definition of this variable. This
also shave off a couple of forks, and the need for a sanity check
only required in fringe situations.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
465111cd
|
2012-06-22T23:31:00
|
|
tests: more uses of $(...) over `...` for command substitution
Somehow missed by the previous changes. No big deal, fix them now.
* t/ax/depcomp.sh: Here.
* t/autodist.sh: And here.
* t/autodist-no-duplicate.sh: And here.
* t/autodist-subdir.sh: Ad here.
* t/remake11.sh t/self-check-me.tap: And here.
* t/perf/testsuite-summary.sh: And here.
* t/perf/testsuite-recheck.sh: And here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
19c56b1d
|
2012-06-22T22:31:35
|
|
tests: new requirement 'grep-nonprint'
* t/ax/test-inist.sh ($esc): New, a literal escape character.
(grep-nonprint): New requirement, check that the grep implementation
available that can handle non-printing characters correctly.
* t/color.sh: Use it instead of hand-rolled equivalent, and do not
(re)define '$esc' explicitly.
* t/color2.sh: Likewise.
* t/tap-color.sh: Likewise.
* t/vtexi4.sh: Likewise.
* t/parallel-tests-no-color-in-log.sh: Likewise. Also prefer the use
of grep over that of $FGREP, because the new requirement only check
grep, and ensure we only grep non-printing characters from a pipe, to
avoid hitting a BSD grep limitation.
* t/parallel-tests-reset-term.sh: Likewise.
* t/ax/tap-summary-aux.sh: Use '$esc' instead of hard-coding the
literal escape character.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
dc18830f
|
2012-06-22T16:18:28
|
|
tests: implement is_newest as an auxiliary script, not shell function
This will allow to also use it in the makefile recipes used in our
test cases.
* t/ax/test-init.sh (is_newest): Remove.
* t/ax/is_newest: New script.
* Makefile.am (EXTRA_DIST): Add it.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
3a1a8c6d
|
2012-06-22T11:40:39
|
|
tests: automatic re-execution works for non-POSIX shells too
Some inferior shells are too greedy in parsing their input. If a
non-POSIX Bourne shell (like Solaris 10 /bin/sh) was used to launch
one of our test scripts, it would fail unconditionally, because it
unexpectedly saw some (by it) unsupported constructs, notwithstanding
such constructs being placed *after* the code implementing automatic
test re-execution with a better shell. In conclusion, the shell
bailed out like this:
$ /bin/sh t/ar.sh
$ t/ar.sh: syntax error at line 257: `is_newest_files=$' unexpected
By moving all the potentially problematic code in a separate file, to
be sourced only after the code for automatic re-execution with a better
shell, we ensure that inferior shell cannot see such code by mistake.
* defs: All code after automatic shell re-execution moved out ...
* t/ax/test-init.sh: ... to this new file.
* syntax-checks.mk (xdefs): Add it.
* Makefile.am (dist_check_DATA): Add it. Also move in 'defs' from
a less explicit 'check_DATA' declaration.
(nodist_check_DATA): Move in 'defs-static' from a less explicit
'check_DATA' declaration.
(check_DATA): Remove.
* t/self-check-sanity.sh: Remove, it was actually too hacky and brittle,
sanity-checking situations we don0t actually care about.
* t/list-of-tests.mk: Adjust.
* t/self-check-explicit-skips.sh: Adjust, and fix a botched heading
comments while we are at it.
* t/self-check-reexec.tap: Adjust.
* t/self-check-cleanup.tap: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
6e713425
|
2012-06-14T13:13:58
|
|
tests: use more POSIX shell features our test scripts
Since commit 'v1.12-36-g2d68fd9' of 2012-05-07, "configure: search a
sturdy POSIX shell to be used in the testsuite", the shell running
our test script is assured to be a POSIX-conforming shell, so we can
use the more modern and flexible idioms and features that we couldn't
use when we also aimed at compatibility with non-POSIX Bourne shells,
like Solaris /bin/sh.
* t/README: Suggest to use POSIX shell features liberally in test cases,
with possible exception of Makefile recipes and configure shell code.
* Several tests: Adjust to use more POSIX shell features; e.g., $(...)
rather than `...`, $((...)) rather than `expr ...`, "if ! CMD; then ..."
instead of "if CMD; then :; else ...", and so on.
In several places, when using the 'test' built-in, prefer '-eq' over
'=' for numeric comparisons, and prefer "grep -c PATTERN FILE" over
"grep PATTERN FILE | wc -l".
Throw in other low-hanging easy improvements and fixlets while we are
at it.
* t/ax/depcomp.sh, t/ax/tap-summary-aux.sh, t/ax/tap-functions.sh,
defs, defs-static.in: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
7b142a7b
|
2012-05-15T16:12:09
|
|
tests: use 'parallel-tests' Automake option by default
This will help our transition from 'serial-tests' to 'parallel-tests'
as the default test suite driver enabled by a TESTS assignment in the
input Makefile.am. Note that that change of default will only take
place in master, though.
* defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE
invocation in the created 'configure.ac' stub, unless the variable
'am_serial_tests' is set to "yes". Don't pay attention anymore to
the 'am_parallel_tests' variable, that's obsolete now.
* defs-static.in: Warn if the 'am_serial_tests' variable is set in the
environment; conversely, don't warn anymore about 'am_parallel_tests'
being set in the environment.
* Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests'
variable instead of the now-obsolete 'am_parallel_tests' one.
* syntax-checks.mk (sc_tests_obsolete_variables): Also warn against
uses of 'am_parallel_tests', which is now deprecated in favor of
'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen,
suggest using 'am_serial_tests' instead, not 'am_parallel_tests'.
* gen-testsuite-part: Now that we use the 'parallel-tests' by default
in our tests, we need to completely change the logic and semantics of
generation of sibling tests for those tests that check the Automake
generated testsuite harness itself. Do that, and give a complete
explanation of the new logic and semantics in the relevant comments.
* t/README: Update.
* Lots of test cases: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
b105d40d
|
2012-04-30T20:44:50
|
|
color-tests: coloring can be forced on non-ANSI terminals as well
Before this change, colorization of testsuite output was suppressed
whenever the terminal was recognized to be a "dumb" one, incapable
of handling ANSI coloring (i.e., when the environment variable TERM
had a value of "dumb"). This happened even when the AM_COLOR_TESTS
variable was set to a value of "always".
Such a behaviour was suboptimal and slightly confusing; in fact, if
a user wants to force coloring of testsuite output that is being
redirected to a regular file, he should be able to do so even if his
terminal is not capable of handling ANSI colors -- in fact, such
terminal wouldn't be involved with the testsuite output in any way,
so why should it be allowed to influence it?
Thus, we now enable coloring of test output whenever AM_COLOR_TESTS
is set to "always", irrespective of the value of the TERM environment
variable.
* NEWS: Update.
* lib/am/check.am [%?COLOR%] (am__tty_colors): Activate colorization
of testsuite output whenever AM_COLOR_TESTS has the value of "always".
* t/ax/tap-summary-aux.sh: Export the TERM environment variable to
"dumb" when forcing colorization of the testsuite output; this should
*not* prevent such colorization from taking place, and we want to
check that this expectation really holds.
* t/ax/testsuite-summary-checks.sh: Likewise.
* t/color.sh: Likewise, and adjust some grepping checks.
* t/tap-color.sh: Likewise. Also, remove redundant "make check"
invocation since we are at it.
* t/color2.sh: Likewise, and check that exporting TERM=dumb actually
prevents testsuite output colorization when AM_COLOR_TESTS is unset.
* t/parallel-tests-reset-term.sh: Relax, to prevent it from failing
spuriously due to the new semantic.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
89ba2a73
|
2012-04-20T10:23:35
|
|
tests: fix spurious failure with non-ANSI terminals
* t/ax/tap-summary-aux.sh: When checking colored testsuite output,
be sure to export the TERM variable to the value "ansi"; otherwise
the automake testsuite driver will not display colored output, not
even if AM_COLOR_TESTS is exported to "always". Failure revealed
by NixOS Hydra.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
8e02441b
|
2012-04-06T15:20:54
|
|
maint: no more make recursion in Automake's build system
We finally merge testsuite-related part of the Automake build
system with the top-level one. See also yesterday's commit
'v1.11-2124-ga4b0f4b', "maint: remove most recursion in automake's
own build system".
* t/gen-testsuite-part: Move ...
* gen-testsuite-part: ... here, with required adjustments.
* t/defs-static.in: Move ...
* defs-static.in: ... here, with required adjustments. In
particular ...
($testsrcdir, $testbuilddir): Removed.
($top_testsrcdir): Renamed ...
($am_top_srcdir): ... to this.
($top_testbuilddir): Renamed ...
($am_top_builddir): ... to this.
* t/defs: Move ...
* defs: ... here, and adjust as required.
* t/Makefile.am: Merge ...
* Makefile.am: ... in here, with related adjustments and
simplifications.
* bootstrap: Adjust as required.
* syntax-checks.mk: Likewise.
* .gitignore: Likewise.
* t/ax/tap-setup.sh: Likewise.
* t/get-sysconf.sh: Likewise.
* t/help-multilib.sh: Likewise.
* t/multlib.sh: Likewise.
* t/parallel-tests2.sh: Likewise.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-exit.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-explicit-skips.sh: Likewise.
* t/self-check-reexec.tap: Likewise.
* t/self-check-sanity.sh: Likewise.
* t/yacc-d-cxx.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
8a5096d1
|
2012-04-06T13:12:25
|
|
tests: rename 'tests/' => 't/', '*.test' => '*.sh'
When we (soon) convert the Automake testsuite to a non-recursive
make setup, we'll have to fix the entries of $(TESTS) to be
prepended with the subdirectory they are in; this will increase
the length of $(TESTS), and thus increase the possibility of
exceeding the command-line length limits on some systems (most
notably, MinGW/MSYS). See automake bug#7868 for more information.
Thus we rename the 'tests/' subdirectory to 't/', and each 'x.test'
script in there to 'x.sh'; this way, the $(TESTS) entry 'foo.test'
will become 't/foo.sh', which have the same number of characters.
* tests/: Rename ...
* t/: ... to this.
* t/*.test: Rename ...
* t/*.sh: ... to this.
* t/.gitignore: Removed as obsolete.
* t/defs: Adjust.
* t/gen-testsuite-part: Likewise.
* t/list-of-tests.mk: Likewise.
* t/ccnoco.sh: Likewise.
* t/ccnoco3.sh: Likewise.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-me.tap: Likewise.
* t/self-check-reexec.tap: Likewise.
* README: Likewise.
* bootstrap: Likewise
* configure.ac: Likewise.
* Makefile.am: Likewise.
* .gitignore: Likewise.
* syntax-check.mk: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|