kmx git

Commit

Hash : c7cfa093
Author :
Date : 2013-05-21T20:22:17

tests: run_make: options to do command redirection

Let's improve the API of the 'run_make()' helper shell function by
adding three new options:

  -O   Save the standard output from make on disk, in a regular file
       named 'stdout'.

  -E   Save the standard error from make on disk, in a regular file
       named 'stderr'.

  -M   Save both the standard output and standard error from make on
       disk, in a regular file named 'output'. This option supersedes
       both the '-O' and '-E' options.

This new API has two main advantages.

  1. Its use will allow us to get rid of more cumbersome idioms
     like, e.g.,

       $MAKE check >stdout && { cat stdout; exit 1; }
       cat stdout

     That can now be substituted with a simpler one:

       run_make -e FAIL -O check

  2. More importantly, using the new API we will prevent any extra output
     from the shell traces of the code in run_make to be redirected along
     with the make stderr (where that was redirected).  This problem was
     present in usages like, e.g.,

       run_make TESTS=foo.test check 2>stderr && exit 1
       grep 'expected error message' stderr

     Such usages are now to be rewritten as follows:

       run_make -e FAIL -E TESTS=foo.test check
       grep 'expected error message' stderr

     ensuring that 'stderr' won't end up containing unrelated stuff.

Note that we do not convert in bulk the old idioms and the use of
redirected 'run_make' invocations with this patch.  We only convert
some occurrences, to ensure that the new implementation of 'run_make'
is sound enough.  More sweeping conversions will likely be done in
follow-up patches.

* t/ax/am-test-lib.sh (run_make): Enhance and implement the extended API.
* t/tap-xfail-tests.sh: Use the new 'run_make' API.
* t/test-driver-cond.sh: Likewise.
* t/tests-environment-fd-redirect.sh: Likewise.
* t/uninstall-fail.sh: Likewise.
* t/yacc-dist-nobuild.sh: Likewise.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>

t/tap-xfail-tests.sh

#! /bin/sh
# Copyright (C) 2011-2013 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

# TAP and $(XFAIL_TESTS): test results without directives are turned from
# PASS to XPASS and from FAIL to XFAIL; other results are unchanged.

. test-init.sh

. tap-setup.sh

echo 'XFAIL_TESTS = $(TESTS)' >> Makefile

cat > all.test <<END
1..6
ok 1
not ok 2
ok 3 # TODO
not ok 4 # TODO
ok 5 # SKIP
# The next should count as a failure (and thus will be
# rendered into an expected failure by XFAIL_TESTS).
not ok 6 # SKIP
Bail out!
END

run_make -O -e FAIL check
count_test_results total=7 pass=0 fail=0 xpass=2 xfail=3 skip=1 error=1

grep '^XPASS: all\.test 1$' stdout
grep '^XFAIL: all\.test 2$' stdout
grep '^XPASS: all\.test 3 # TODO' stdout
grep '^XFAIL: all\.test 4 # TODO' stdout
grep '^SKIP: all\.test 5 # SKIP' stdout
grep '^XFAIL: all\.test 6 # SKIP' stdout
grep '^ERROR: all\.test - Bail out!' stdout

# Check that the exit status of the testsuite is indeed 0 when we
# would expect success.

cat > all.test <<END
1..3
not ok 1
ok 2 # SKIP
not ok 3 # TODO
END

run_make -O check
count_test_results total=3 pass=0 fail=0 xpass=0 xfail=2 skip=1 error=0

:

kc3-lang/automake/t/tap-xfail-tests.sh

Commit

t/tap-xfail-tests.sh