Hash :
ffd4ecc9
Author :
Date :
2013-05-20T11:23:01
tests: better idiom to override make macro defs on the cmdline We now do so with a new wrapper shell function 'run_make()', which relies on the use of AM_MAKEFLAGS, avoiding the use of the '-e' make option. The use of that option (that causes the environment variables to take precedence over the macro definitions in the Makefile) has proved to be quite brittle in the past, causing annoying and sometimes problematic spurious failures. This has worsened in some still unpublished work going on in development branches. It's time to solve the issue once and for all. Note that we do not convert all uses of $MAKE in the testsuite right away; we might do so in follow-up changes, with leisure, to avoid a "patch bomb" effect (this commit is already too much of a bomb itself). What we do in this commit is to get rid of all "$MAKE -e" invocations. We admit that the implementation and feature-set of 'run_make()' are far from perfect, but good enough for our current purposes. We'll improve 'run_make()' if and when the need arises. * syntax-checks.mk (sc_tests_no_make_e): New syntax check, guard against the use of "$MAKE -e". (syntax_check_rules): Add it. (sc_tests_overriding_macros_on_cmdline): Adjust. (lint): New, alias for 'maintainer-check', for lazy typists. Idea backported from the 'maint' branch (Automake 1.13a). * t/ax/am-test-lib.sh (run_make): New function. Run $MAKE with the given command-line arguments, handling command-line override of variable definitions in a smart way (using AM_MAKEFLAGS if a non-GNU make implementation is detected to be in use). (useless_vpath_rebuild): Adjust to use 'run_make', to avoid a spurious maintainer check failure. (yl_distcheck): Use 'run_make' rather than bare '$MAKE'. (single_quote, append_single_quoted, is_valid_varname): New auxiliary function, used, directly or indirectly, by it. * Many tests: Adjust to avoid the use of "$MAKE -e", and prefer the use of 'run_make' in few other contexts as well, where it makes sense. Other minor fixlets while at it. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148
#! /bin/sh
# Copyright (C) 2011-2013 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check parallel-tests features:
# - If $(TEST_SUITE_LOG) is in $(TEST_LOGS), we get a diagnosed
# error, not a make hang or a system freeze.
. test-init.sh
# We don't want localized error messages from make, since we'll have
# to grep them. See automake bug#11452.
LANG=C LANGUAGE=C LC_ALL=C
export LANG LANGUAGE LC_ALL
# The tricky part of this test is to avoid that make hangs or even
# freezes the system in case infinite recursion (which is the bug we
# are testing against) is encountered. The following hacky makefile
# should minimize the probability of that happening.
cat > Makefile.am << 'END'
TEST_LOG_COMPILER = true
TESTS =
errmsg = ::OOPS:: Recursion too deep
if IS_GNU_MAKE
is_too_deep := $(shell test $(MAKELEVEL) -lt 10 && echo no)
## Indenteation here required to avoid confusing Automake.
ifeq ($(is_too_deep),no)
else
$(error $(errmsg), $(MAKELEVEL) levels)
endif
else !IS_GNU_MAKE
# We use mkdir to detect the level of recursion, since it is easy
# to use and assured to be portably atomical. Also use an higher
# number than with GNU make above, since the level used here can
# be incremented by tow or more per recursion.
recursion-not-too-deep:
@ok=no; \
for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 \
18 19 20 21 22 23 24 25 26 27 28 29; \
do \
echo " mkdir rec-$$i.d"; \
if mkdir rec-$$i.d; then \
ok=yes; break; \
else :; fi; \
done; \
test $$ok = yes || { echo '$(errmsg)' >&2; exit 1; }
.PHONY: recursion-not-too-deep
clean-local:
rmdir rec-[0-9].d
targets = all check recheck $(TESTS) $(TEST_LOGS) $(TEST_SUITE_LOG)
$(targets): recursion-not-too-deep
# For BSD make.
.BEGIN: recursion-not-too-deep
endif !IS_GNU_MAKE
END
if using_gmake; then
cond=:
else
cond=false
fi
cat >> configure.ac << END
AM_CONDITIONAL([IS_GNU_MAKE], [$cond])
AC_OUTPUT
END
# Another helpful idiom to avoid hanging on capable systems. The subshell
# is needed since 'ulimit' might be a special shell builtin.
if (ulimit -t 8); then ulimit -t 8; fi
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a -Wno-portability
./configure
do_check ()
{
st=0
log=$1; shift
run_make "$@" check >output 2>&1 || st=$?
cat output
$FGREP '::OOPS::' output && exit 1 # Possible infinite recursion.
# Check that at least we don't create a botched global log file.
test ! -e "$log"
if using_gmake; then
grep "[Cc]ircular.*dependency" output | $FGREP "$log"
test $st -gt 0
else
# Look for possible error messages about circular dependencies from
# either make or our own recipes. At least one such a message must
# be present. OTOH, some make implementations (e.g., NetBSD's), while
# smartly detecting the circular dependency early and diagnosing it,
# still exit with a successful exit status (yikes!). So don't check
# the exit status of non-GNU make, to avoid spurious failures.
# this case.
err_seen=no
for err_rx in \
'circular.* depend' \
'depend.* circular' \
'graph cycle' \
'infinite (loop|recursion)' \
'depend.* on itself' \
; do
$EGREP -i "$err_rx" output | $FGREP "$log" || continue
err_seen=yes
break
done
test $err_seen = yes || exit 1
fi
}
: > test-suite.test
do_check test-suite.log TESTS=test-suite.test
rm -f *.log *.test
: > 0.test
: > 1.test
: > 2.test
: > 3.test
: > foobar.test
do_check foobar.log TEST_LOGS='0.log 1.log foobar.log 2.log 3.log' \
TEST_SUITE_LOG=foobar.log
rm -f *.log *.test
: