|
ff022f46
|
2012-05-29T11:58:02
|
|
perf: beginning of a performance testsuite
Some tests in the Automake testsuite already aims only at verifying
the performance, rather than the correctness, of some operations.
Still, they are somewhat shoehorned and forced into the PASS/FAIL
framework (say, with the 'ulimit' shell builtin used to verify some
operation doesn't take up too much time or memory), but that is
conceptually a stretch, and has already caused problems in practice
(see automake bug#11512 for an example).
So we start moving the "performance tests" out of the testsuite proper,
and make them run only "on demand" (when the user exports the variable
'AM_TESTSUITE_PERF' to "yes"). Ideally, we should provide those tests
with a custom runner/driver that measures and displays the relevant
performance information, but doing that correctly and with the right
APIs is definitely more difficult, so we leave it for a later step
(an hope we'll take such a step eventually).
* t/cond29.sh: Move ...
* t/perf/cond.sh: ... here, and adjust.
* t/testsuite-recheck-speed.sh: Move ...
* t/perf/testsuite-recheck.sh: ... here.
* t/testsuite-summary-speed.sh: Move ...
* t/perf/testsuite-summary.sh: ... here.
* t/list-of-tests.mk (perf_TESTS): New variable, listing the tests in
the 't/perf' directory.
(handwritten_TESTS): Adjust.
* defs: Skip any tests in the 't/perf/' subdirectory unless the
'AM_TESTSUITE_PERF' variable is set to "yes" or "y".
* .gitignore: Update.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
7b142a7b
|
2012-05-15T16:12:09
|
|
tests: use 'parallel-tests' Automake option by default
This will help our transition from 'serial-tests' to 'parallel-tests'
as the default test suite driver enabled by a TESTS assignment in the
input Makefile.am. Note that that change of default will only take
place in master, though.
* defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE
invocation in the created 'configure.ac' stub, unless the variable
'am_serial_tests' is set to "yes". Don't pay attention anymore to
the 'am_parallel_tests' variable, that's obsolete now.
* defs-static.in: Warn if the 'am_serial_tests' variable is set in the
environment; conversely, don't warn anymore about 'am_parallel_tests'
being set in the environment.
* Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests'
variable instead of the now-obsolete 'am_parallel_tests' one.
* syntax-checks.mk (sc_tests_obsolete_variables): Also warn against
uses of 'am_parallel_tests', which is now deprecated in favor of
'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen,
suggest using 'am_serial_tests' instead, not 'am_parallel_tests'.
* gen-testsuite-part: Now that we use the 'parallel-tests' by default
in our tests, we need to completely change the logic and semantics of
generation of sibling tests for those tests that check the Automake
generated testsuite harness itself. Do that, and give a complete
explanation of the new logic and semantics in the relevant comments.
* t/README: Update.
* Lots of test cases: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
1cf7d188
|
2011-12-28T13:23:31
|
|
test defs: fix indentation (cosmetic change)
* defs (exit trap): Fix indentation left botched by previous
patch 'v1.11b-51-g626bf65'.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
6b641de5
|
2011-12-28T13:23:31
|
|
tests: remove obsolete uses of $sh_errexit_works
After the last changes, configure will ensure that the shell
selected to run the test scripts can correctly propagate exit
status to the exit trap when 'set -e' is in effect.
* configure.ac (sh_errexit_works): Do not AC_SUBST it anymore.
* defs-static.in (sh_errexit_works): Do not initialize
from the AC_SUBST value anymore.
* defs (trap): Trap the EXIT signal unconditionally.
* t/self-check-explicit-skips.sh: Do not skip the test if
'$sh_errexit_works' is != "yes", this check doesn't make
sense anymore.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-exit.tap: Assume the exit trap is always
installed by ./defs.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
21d1b934
|
2011-12-28T13:23:31
|
|
tests: shell running test scripts is now named AM_TEST_RUNNER_SHELL
This is just a preparatory refactoring for future changes.
* configure.ac (AM_TEST_RUNNER_SHELL): New variable, defined
to $SHELL', and AC_SUBST'd.
* Makefile.am (LOG_COMPILER): Redefine from $(SHELL) to
$(AM_TEST_RUNNER_SHELL).
* defs: Adjust to use $AM_TEST_RUNNER_SHELL instead
of $CONFIG_SHELL and/or $SHELL, where appropriate. Minor
opportunistic changes.
* t/self-check-explicit-skips.sh: Likewise.
* t/self-check-sanity.sh: Likewise.
* t/self-check-tap.sh: Likewise.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-env-sanitize.tap: Likewise.
* t/self-check-exit.tap: Likewise.
* t/self-check-me.tap: Likewise.
* t/self-check-reexec.tap: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
44cae262
|
2012-04-13T11:11:03
|
|
test defs: more environment cleanups
* defs (SH_LOG_COMPILER, SH_LOG_COMPILE, SH_LOG_FLAGS,
AM_SH_LOG_FLAGS, SH_LOG_DRIVER, SH_LOG_DRIVER_FLAGS,
AM_SH_LOG_DRIVER_FLAGS): Unset these variable to avoid
potential interferences from the environment.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
4cc65f75
|
2012-04-13T10:07:54
|
|
test defs: unset 'AM_UPDATE_INFO_DIR' environment variable
* defs (AM_UPDATE_INFO_DIR): Unset, to be sure to avoid unduly
interferences from the environment. See also automake bug#11204.
* THANKS: Add entry for David Fang.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
8e02441b
|
2012-04-06T15:20:54
|
|
maint: no more make recursion in Automake's build system
We finally merge testsuite-related part of the Automake build
system with the top-level one. See also yesterday's commit
'v1.11-2124-ga4b0f4b', "maint: remove most recursion in automake's
own build system".
* t/gen-testsuite-part: Move ...
* gen-testsuite-part: ... here, with required adjustments.
* t/defs-static.in: Move ...
* defs-static.in: ... here, with required adjustments. In
particular ...
($testsrcdir, $testbuilddir): Removed.
($top_testsrcdir): Renamed ...
($am_top_srcdir): ... to this.
($top_testbuilddir): Renamed ...
($am_top_builddir): ... to this.
* t/defs: Move ...
* defs: ... here, and adjust as required.
* t/Makefile.am: Merge ...
* Makefile.am: ... in here, with related adjustments and
simplifications.
* bootstrap: Adjust as required.
* syntax-checks.mk: Likewise.
* .gitignore: Likewise.
* t/ax/tap-setup.sh: Likewise.
* t/get-sysconf.sh: Likewise.
* t/help-multilib.sh: Likewise.
* t/multlib.sh: Likewise.
* t/parallel-tests2.sh: Likewise.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-exit.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-explicit-skips.sh: Likewise.
* t/self-check-reexec.tap: Likewise.
* t/self-check-sanity.sh: Likewise.
* t/yacc-d-cxx.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|