kc3-lang/automake/tests/tap-diagnostic.test
-
Show log
Commit
-
Hash : 6483ac07
Author :
Date : 2011-08-01T16:14:44
testsuite: optimize tests on TAP for speed and against duplication The creation and configuration of common files and data used by many tests on TAP is, with this change, factored out into a new dedicated auxiliary test, from which those tests will depend upon. This reduces code duplication in tests and, more importantly, offers a noticeable speedup in the involved tests (30-40%). * tests/tap-common-setup.test: New test, setting up the common files and data used by various tests on TAP support. * tests/tap-setup.sh: New file, to be sourced by tests wanting to bring in data generated by `tap-common-setup.test'. * tests/tap-autonumber.test: Update to use the pre-computed data files. * tests/tap-bailout.test: Likewise. * tests/tap-color.test: Likewise. * tests/tap-deps.test: Likewise. * tests/tap-diagnostic.test: Likewise. * tests/tap-empty-diagnostic.test: Likewise. * tests/tap-empty.test: Likewise. * tests/tap-escape-directive.test: Likewise. * tests/tap-exit.test: Likewise. * tests/tap-signal.test: Likewise. * tests/tap-fancy.test: Likewise. * tests/tap-fancy2.test: Likewise. * tests/tap-global-log.test: Likewise. * tests/tap-global-result.test: Likewise. * tests/tap-html.test: Likewise. * tests/tap-log.test: Likewise. * tests/tap-merge-stdout-stderr.test: Likewise. * tests/tap-no-merge-stdout-stderr.test: Likewise. * tests/tap-message-0.test: Likewise. * tests/tap-no-disable-hard-error.test: Likewise. * tests/tap-no-spurious-summary.test: Likewise. * tests/tap-no-spurious.test: Likewise. * tests/tap-not-ok-skip.test: Likewise. * tests/tap-numeric-description.test: Likewise. * tests/tap-out-of-order.test: Likewise. * tests/tap-passthrough.test: Likewise. * tests/tap-passthrough-exit.test: Likewise. * tests/tap-plan.test: Likewise. * tests/tap-plan-corner.test: Likewise. * tests/tap-plan-corner2.test: Likewise. * tests/tap-plan-errors.test: Likewise. * tests/tap-realtime.test: Likewise. * tests/tap-recheck-logs.test: Likewise. * tests/tap-skip-whole.test: Likewise. * tests/tap-todo-skip-together.test: Likewise. * tests/tap-todo-skip-whitespace.test: Likewise. * tests/tap-todo-skip.test: Likewise. * tests/tap-unplanned.test: Likewise. * tests/tap-with-and-without-number.test: Likewise. * tests/tap-xfail-tests.test: Likewise. * tests/tap-skip-whole-whitespace.test: Likewise, and remove redundant definitions of `$sp' and `$tab' (they are already defined in `tests/defs'). * tests/tap-whitespace-normalization.test: Likewise. * tests/Makefile.am (TESTS): Update. (EXTRA_DIST): Distribute `tap-setup.sh'. (tap_with_common_setup_tests, tap_with_common_setup_logs)): New variables, holding respectively the list of tests using the files pre-computed by `tap-common-setup.test', and the list of their corresponding log files. (tap_other_tests): New variable, holding the list of other tests on TAP support. ($(tap_with_common_setup_logs)): Depend on `tap-common-setup.log' and `tap-setup.sh'.
-
tests/tap-diagnostic.test
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124
#! /bin/sh # Copyright (C) 2011 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2, or (at your option) # any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. # TAP support: # - diagnostic messages (TAP lines with leading "#") # - flags '--comments' and '--no-comments' of the TAP test driver parallel_tests=yes . ./defs || Exit 1 . "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh" metacharacters=\''"\$!&()[]<>#;^?*' cat > all.test <<END 1..4 # Hi! I'm a comment. # Tests begin. ok 1 not ok 2 - foo # TODO ok 3 - bar # SKIP # Tests end. ok - zardoz # Shell metacharacters here: $metacharacters .# Leading characters before "#", not a TAP diagnostic line. x # Leading characters before "#", not a TAP diagnostic line. # Leading whitespace before "#", not a TAP diagnostic line. ${tab}# Leading whitespace before "#", not a TAP diagnostic line. ${tab} # Leading whitespace before "#", not a TAP diagnostic line. END cat > exp <<END # all.test: Hi! I'm a comment. # all.test: Tests begin. PASS: all.test 1 XFAIL: all.test 2 - foo # TODO SKIP: all.test 3 - bar # SKIP # all.test: Tests end. PASS: all.test 4 - zardoz # all.test: Shell metacharacters here: $metacharacters END $MAKE check >stdout || { cat stdout; Exit 1; } cat stdout $EGREP -i "#.*all\\.test|a comment|(Tests|Shell) " stdout && Exit 1 count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0 echo 'AM_TEST_LOG_DRIVER_FLAGS = --comments' >> Makefile $MAKE check >stdout || { cat stdout; Exit 1; } cat stdout $FGREP ' all.test' stdout > got cat exp cat got diff exp got count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0 TEST_LOG_DRIVER_FLAGS="--no-comments" $MAKE -e check >stdout \ || { cat stdout; Exit 1; } cat stdout $EGREP -i "#.*all\\.test|a comment|(Tests|Shell) " stdout && Exit 1 count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0 # The "#"-prepended lines here shouldn't be parsed as test results. cat > all.test <<END 1..1 ok # ok #ok # not ok #not ok # Bail out! #Bail out! # SKIP #SKIP # TODO #TODO END $MAKE check >stdout || { cat stdout; Exit 1; } cat stdout count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0 # Diagnostic without leading whitespace, or with extra leading whitespace, # is ok. Be laxer in the grepping checks, to allow for whitespace # normalization by the TAP driver. ws="[ $tab]" ws0p="${ws}*" ws1p="${ws}${ws0p}" cat > all.test <<END 1..1 ok 1 #foo #bar${tab} # zardoz ${tab} # ${tab} ${tab}${tab}foo bar${tab}baz ${tab} END $MAKE check >stdout || { cat stdout; Exit 1; } cat stdout count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0 grep "^# all.test:${ws0p}foo$" stdout grep "^# all.test:${ws0p}bar${ws0p}$" stdout grep "^# all.test:${ws1p}zardoz${ws0p}$" stdout grep "^# all.test:${ws1p}foo${ws1p}bar${ws1p}baz${ws0p}$" stdout test `grep -c '^# all\.test:' stdout` -eq 4 :