Hash :
ff022f46
Author :
Date :
2012-05-29T11:58:02
perf: beginning of a performance testsuite Some tests in the Automake testsuite already aims only at verifying the performance, rather than the correctness, of some operations. Still, they are somewhat shoehorned and forced into the PASS/FAIL framework (say, with the 'ulimit' shell builtin used to verify some operation doesn't take up too much time or memory), but that is conceptually a stretch, and has already caused problems in practice (see automake bug#11512 for an example). So we start moving the "performance tests" out of the testsuite proper, and make them run only "on demand" (when the user exports the variable 'AM_TESTSUITE_PERF' to "yes"). Ideally, we should provide those tests with a custom runner/driver that measures and displays the relevant performance information, but doing that correctly and with the right APIs is definitely more difficult, so we leave it for a later step (an hope we'll take such a step eventually). * t/cond29.sh: Move ... * t/perf/cond.sh: ... here, and adjust. * t/testsuite-recheck-speed.sh: Move ... * t/perf/testsuite-recheck.sh: ... here. * t/testsuite-summary-speed.sh: Move ... * t/perf/testsuite-summary.sh: ... here. * t/list-of-tests.mk (perf_TESTS): New variable, listing the tests in the 't/perf' directory. (handwritten_TESTS): Adjust. * defs: Skip any tests in the 't/perf/' subdirectory unless the 'AM_TESTSUITE_PERF' variable is set to "yes" or "y". * .gitignore: Update. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
#!/bin/sh
# Copyright (C) 2003-2012 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that many conditions do not lead to combinatorial explosion.
# (This is related to PR/352.)
. ./defs || Exit 1
echo AC_PROG_CC >>configure.ac
cat >Makefile.am <<EOF
bin_PROGRAMS = a
a02_LDADD =
a03_SOURCES =
EOF
# On this test, Automake 1.7.x would compute all 2**24 = 16777216
# possible combinations of conditionals (it would do this five times,
# to define a01_DEPENDENCIES, a02_DEPENDENCIES, a03_OBJECTS,
# a04_OBJECTS, and to rewrite bin_PROGRAM), eating all memory, swap,
# or cpu time it can found.
for i in \
01 02 03 04 05 06 07 08 09 10 \
11 12 13 14 15 16 17 18 19 20 \
21 22 23 24
do
unindent >>Makefile.am <<EOF
if C$i
bin_PROGRAMS += a$i
a01_LDADD = foo${i}.o
a02_LDADD += bar${i}.o
a03_SOURCES += baz${i}.c
a04_SOURCES = quux${i}.c
endif C$i
EOF
echo "AM_CONDITIONAL([C$i], [:])" >>configure.ac
done
$ACLOCAL
$AUTOMAKE
: