test


Log

Author Commit Date CI Message
Behdad Esfahbod efb8d3eb 2012-09-05T15:50:47 Fixup test failure reporting After we implemented dotted-circle, we were still ignoring any tests that had dottedcircle in it for any of the shapers. That meant that if we wrongly outputted dottedcircle, the test was being ignored. Ouch! Fixing that shows regressions across the board. Most are Uniscribe bugs: NOT inserting dotted-circle when it should. Some are arou machine bugs. This is in fact a nice way to catch Indic-machine deficiencies and when I fix the regressions, our clusters should be much closer to Uniscribe. For now, we regressed from: BENGALI: 353997 out of 354285 tests passed. 288 failed (0.0812905%) DEVANAGARI: 707339 out of 707394 tests passed. 55 failed (0.00777502%) GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%) GURMUKHI: 60769 out of 60809 tests passed. 40 failed (0.0657797%) KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%) KHMER: 299106 out of 299124 tests passed. 18 failed (0.00601757%) LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%) MALAYALAM: 1048104 out of 1048416 tests passed. 312 failed (0.0297592%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271747 out of 271847 tests passed. 100 failed (0.0367854%) TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%) TELUGU: 970558 out of 970573 tests passed. 15 failed (0.00154548%) TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%) To: BENGALI: 353990 out of 354285 tests passed. 295 failed (0.0832663%) DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%) GUJARATI: 366447 out of 366506 tests passed. 59 failed (0.016098%) GURMUKHI: 60707 out of 60809 tests passed. 102 failed (0.167738%) KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%) KHMER: 298962 out of 299124 tests passed. 162 failed (0.0541581%) LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%) MALAYALAM: 1048074 out of 1048416 tests passed. 342 failed (0.0326206%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%) TAMIL: 1091835 out of 1091837 tests passed. 2 failed (0.000183178%) TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%) TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%) Investigating.
Behdad Esfahbod a4e75e41 2012-08-27T15:54:15 Minor
Behdad Esfahbod 206ab605 2012-08-10T09:06:30 [test] Move around
Behdad Esfahbod 7a484c60 2012-08-10T09:05:29 [test] Add Urdu ligature sequences from CRULP
Behdad Esfahbod 378d279b 2012-07-31T21:36:16 Implement Unicode compatibility decompositions Based on patch from Philip Withnall. https://bugs.freedesktop.org/show_bug.cgi?id=41095
Behdad Esfahbod 70b3dc32 2012-07-30T12:40:18 Add Hebrew test
Behdad Esfahbod a973b5ce 2012-07-30T01:46:34 [GSUB] Further adjustments to mark-attachment vs ligation interaction The d1d69ec52e75a78575b620a1c456d528b6078170 change broke Kannada badly, since it was ligating consonants, pushing matra out, and then ligating with the matra. Adjust for that. See comments.
Behdad Esfahbod 97a201be 2012-07-29T20:31:36 Add Arabic tests for mark ligature component attachments
Behdad Esfahbod 5d874d56 2012-07-28T21:05:25 [GPOS] Fix mark-to-mark positioning when one of the marks is a ligature This commit: a3313e54008167e415b72c780ca7b9cda958d07e broke MarkMarkPos when one of the marks itself is a ligature. That regressed 26 Tibetan tests (up from zero!). Fix that. Tibetan back to zero.
Behdad Esfahbod 6411e74c 2012-07-24T13:48:49 [Indic] Reposition Gurmukhi top matras to after post The font is forming a post-base consonant in some samples, and Uniscribe positions top matra on the post-base. Do the same. Gurmukhi failures down from 59 to 41 (0.0674242%).
Behdad Esfahbod c3f769ba 2012-07-24T13:26:32 [Indic] Ignore Uniscribe output containing two zero-width space glyphs Uniscribe is buggy and sometimes /eats/ a mark next to a non-joiner. Most of Malayalam failures where actually hitting this bug. Ignore test output with two zero-width space glyphs. This is a hack until we build up the test suite infrastructure better. Bengali went down by 9, Devanagari by 2, Kannada by 130, Malayalm down from 1197 to 307, Sinhala down by 16, Telugu down by 26. New stats: BENGALI: 353996 out of 354285 tests passed. 289 failed (0.0815727%) DEVANAGARI: 693573 out of 693628 tests passed. 55 failed (0.00792932%) GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%) GURMUKHI: 60750 out of 60809 tests passed. 59 failed (0.0970251%) KANNADA: 951086 out of 951913 tests passed. 827 failed (0.0868777%) KHMER: 299094 out of 299124 tests passed. 30 failed (0.0100293%) MALAYALAM: 1048109 out of 1048416 tests passed. 307 failed (0.0292823%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271715 out of 271847 tests passed. 132 failed (0.0485567%) TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%) TELUGU: 970550 out of 970573 tests passed. 23 failed (0.00236973%)
Behdad Esfahbod 65c43acc 2012-07-24T03:36:47 [Indic] Better position left-matra in Malayalam Just put it before base, which is what's expected. Malayalam failures down from 1559 to 1197 (0.114172%). BENGALI: 353988 out of 354285 tests passed. 297 failed (0.0838308%) DEVANAGARI: 693571 out of 693628 tests passed. 57 failed (0.00821766%) GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%) GURMUKHI: 60750 out of 60809 tests passed. 59 failed (0.0970251%) KANNADA: 950956 out of 951913 tests passed. 957 failed (0.100534%) KHMER: 299094 out of 299124 tests passed. 30 failed (0.0100293%) MALAYALAM: 1047219 out of 1048416 tests passed. 1197 failed (0.114172%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271699 out of 271847 tests passed. 148 failed (0.0544424%) TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%) TELUGU: 970524 out of 970573 tests passed. 49 failed (0.00504856%)
Behdad Esfahbod 88f413b5 2012-07-24T03:04:36 [Indic] Implement Reph+Ya-Phalaa interaction The sequence Ra,H,Ya in Bengali is ambigious and Unicode encoded that to get Ya-Phalaa, one would place ZWJ before Halant. Ie. a ZWJ,H sequence requests subjoining, while a H,ZWJ requests Half form. Implement that. Bengali failures go down from 377 to 297 (0.0838308%). Gujarati is down by 4 to 17 (0.0046384%). Kannada is down by 226 to 957 (0.100534%). Current status: BENGALI: 353988 out of 354285 tests passed. 297 failed (0.0838308%) DEVANAGARI: 693571 out of 693628 tests passed. 57 failed (0.00821766%) GUJARATI: 366489 out of 366506 tests passed. 17 failed (0.0046384%) GURMUKHI: 60750 out of 60809 tests passed. 59 failed (0.0970251%) KANNADA: 950956 out of 951913 tests passed. 957 failed (0.100534%) KHMER: 299094 out of 299124 tests passed. 30 failed (0.0100293%) MALAYALAM: 1046857 out of 1048416 tests passed. 1559 failed (0.148701%) ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%) SINHALA: 271699 out of 271847 tests passed. 148 failed (0.0544424%) TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%) TELUGU: 970524 out of 970573 tests passed. 49 failed (0.00504856%)
Behdad Esfahbod 330b329c 2012-07-24T02:25:26 [Indic] Unmark U+17D1 KHMER SIGN VIRIAM to NOT be a Virama Fixes another 1 Khmer failure. Down to 30 (0.0100293%) now.
Behdad Esfahbod d90b8e84 2012-07-24T02:10:20 [Indic] Reposition Khmer prebase-reordering Ra around split matras In Khmer coeng model, a V,Ra can go *after* matras. If it goes after a split matra, it should be reordered to *before* the left part of such matra. Khmer failures down from 136 to 39 (0.0130381%).
Behdad Esfahbod 75737991 2012-07-24T01:32:07 [Indic] Position Khmer U+17CE Fixes another 6 Khmer failures. Now at 136 (0.0454661%).
Behdad Esfahbod 2278eefc 2012-07-24T00:26:43 [Indic] In Sinhala, form forced Reph even if no other consonant found Fixes another 10 Sinhala failures. Down to 148 (0.0544424%).
Behdad Esfahbod 71fd5e80 2012-07-24T00:21:16 [Indic] Further adjust base algorithm for Sinhala Apparently if there is C,V,ZWJ,C, the first C will be base, but if it's C,ZWJ,V,C, the second one will be. Note that Uniscribe implements this differently, by breaking syllable in the case of C,ZWJ,V,C and putting the first consonant in one syllable and the rest in the next syllable. Sinhala failures down from 208 to 158 (0.0581209%). No changes to Khmer.
Behdad Esfahbod 73d71cc5 2012-07-24T00:09:12 [Indic] End Vowel-based syllable at ZWJ One Devanagari test regressed, plus 10 Malayalam (at 1545 now). Fixed 120 Sinhala failures. Now at 208 (0.0765136%).
Behdad Esfahbod 34c21503 2012-07-23T23:51:29 [Indic] Improve Sinhala base algorithm and reph positioning Sinhala does not have half forms. And most (all?) consonants can be base, except when preceded by ZWJ, which would request a subjoined form. Hence switch the base algorithm to categorize with Khmer, start search at start, and stop at a ZWJ. Also, mark all pos=base consonants after base to be subjoined. Mark base itself to have pos=base. Finally, adjust Sinhala's reph position to after-main. Brings down Sinhala failures from 455 to 328 (0.120656%).
Behdad Esfahbod 771a8f50 2012-07-23T20:07:50 [Indic] exclude ligatures when matching on Indic category If, say, a H,ZWJ,C ligature was formed, we don't want the code to detec that as a Halant. So, ignore ligatures when matching category in final_reordering. Sinhala failures down from 514 to 455 (0.167374%).
Behdad Esfahbod 42848453 2012-07-23T13:52:07 [Thai] Reorder U+0E3A THAI VOWEL SIGN PHINTHU Uniscribe reorders U+0E3A to be after U+0E38 and U+0E39. We do that by modifying the ccc for U+0E3A. Fixes the two remaining Thai failures (see previous commit).
Behdad Esfahbod 4a7f4f3e 2012-07-23T13:15:33 [Thai] Adjust SARA AM reordering to match Uniscribe Adjust the list of marks before SARA AM that get the reordering treatment. Also adjust cluster formation to match Uniscribe. With Wikipedia test data, now I see: - For Thai, with the Angsana New font from Win7, I see 54 failures out of over 4M tests (0.00129107%). Of the 54, two are legitimate reordering issues (fix coming soon), and the other 52 are simply Uniscribe using a zero-width space char instead of an unknown character for missing glyphs. No idea why. The missing-glyph sequences include one that is a Thai character followed by an Arabic Sokun. Someone confused it with Nikhahit I assume! - For Lao, with the Dokchampa font from Win7, 33 tests fail out of 54k (0.0615167%). All seem to be insignificant mark positioning with two marks on a base. Have to investigate.
Behdad Esfahbod 60554f14 2012-07-22T23:23:56 [Indic] Merge in Malayalam tests From: http://silpa.org.in/pub/tests/hb/ml/ml-harfbuzz-testdata.txt
Behdad Esfahbod 5c708177 2012-07-22T23:20:27 [Indic] Add extensive Sinhala tests Generated by: http://git.savannah.gnu.org/cgit/sinhala.git/plain/utils/gen-unicode-sinhala.py
Behdad Esfahbod 2efe4707 2012-07-22T23:17:59 [Indic] Add Sinhala tests Merge tests from: http://git.savannah.gnu.org/cgit/sinhala.git/plain/patches/icu-sinhala-rendering.txt
Behdad Esfahbod 3d4c111b 2012-07-20T19:34:39 Add a test case
Behdad Esfahbod bdd08043 2012-07-20T16:03:09 [Indic] Reposition Oriya Candrabindu Oriya failures down from 0.65% to 0.20%.
Behdad Esfahbod 87cd6326 2012-07-19T21:17:48 [Indic] Recategorize some Kannada right matras Kannada failures down from 3.5% to 2.93%.
Behdad Esfahbod c87bcddb 2012-07-19T20:03:25 [Indic] Add failing test for Kannada
Behdad Esfahbod deeb540a 2012-07-19T11:30:48 [test] Ignore tests with DOTTED CIRCLE in the output
Behdad Esfahbod 422ecd2d 2012-07-18T23:25:58 [Indic] Accept a forced Rakar sequence at the end of syllable In Sinhala, Rakar is formed by Al-Lakuna,ZWJ,Ra. If you put that at the end of a Consonant,Matra syllable, you get a dotted-circle from Uniscribe. Apparently adding a ZWJ before the Al-Lakuna "fixes" that. And people have been encoding that sequence... So, allow a forced "ZWJ,Virama,ZWJ,Ra" sequence at the of syllables. Fixes some 100 or more of Sinhala failures. Now at 622 only (0.23%).
Behdad Esfahbod 10cdc94e 2012-07-18T17:42:34 [Indic] In final reordering, find base, even if it disappeared POS_BASE can disappear if base ligated backward. Define base as last with position not after base. Fixes a few hundred of Sinhala failures with Iskoola Pota.
Behdad Esfahbod 3285e107 2012-07-18T17:22:14 [Indic] Implement Sinhala "Al Lakuna" Reph behavior In Sinhala, Reph is formed only explicitly, by the presence of a ZWJ.
Behdad Esfahbod 552d19b7 2012-07-18T16:00:49 [Indic] Treat Register Shifters like Nukta Really this time. Fixes another 18 Khmer tests.
Behdad Esfahbod 69f26bf3 2012-07-18T15:45:43 [Indic] Fix Matra reordering when base is at end of syllable For example: U+915,U+200c,U+93f Fixes last Tamil failure!
Behdad Esfahbod 391cc033 2012-07-18T15:10:05 [Indic] Allow halant group in Vowel and placeholder syllables Fixes 2 out of 560 Devanagari failures. AND: Fixes 1 out of 2 Tamil failures.
Behdad Esfahbod 418d00df 2012-07-18T14:57:28 [Indic] Minor
Behdad Esfahbod 25bc4894 2012-07-17T17:53:03 [Indic] Better categorize Register Shifters and Khmer Various signs Down another 500 or so Khmer failures!
Behdad Esfahbod 34b57149 2012-07-17T14:09:32 [Indic] Treat Khmer Register Shifters more like Nuktas Except that there may be a ZWNJ before a Register Shifter.
Behdad Esfahbod 0201e0a4 2012-07-17T13:55:10 [Indic] Apply 'cfar' for Khmer Mark stuff after a pre-base reordering Ro 'cfar'. Used in Khmer. This allows distinguishing the following cases with MS Khmer fonts: U+1784,U+17D2,U+179A,U+17D2,U+1782 U+1784,U+17D2,U+1782,U+17D2,U+179A
Behdad Esfahbod 55f70ebf 2012-07-17T12:50:13 [Indic] Position final subjoined consonants (and vowels) after matras In Khmer, a final subjoined consonant or independent vowel can occur after matras. This final subjoined thing should NOT be reordered to before the matra even though it's subjoined. Fixes another 1k of the Khmer failures. Not much left really.
Behdad Esfahbod c50ed71e 2012-07-17T11:54:28 [Indic] Recategorize Khmer coeng sign as a separate category OT_Coeng Amend the syllable structure to allow a final subscripted consonant (Coeng+C) and a final subscripted independent vowel (Coeng+V). Fixes another 2k of Khmer failures.
Behdad Esfahbod 74ccc6a1 2012-07-17T11:16:19 [Indic] Move Halant with after-base consonants Normally, we attach the Halant to the previous character and move it with it. For after-base consonants however, the Halant "belongs" to the consonant after, so attach it so. This fixes Bengali sequences involving post-base consonant Ya, which should ligate with the Halant to form Ya Phala, but previously a reordered matras was blocking the ligation.
Behdad Esfahbod d5c4edcd 2012-07-17T10:40:59 [Indic] Apply presentation-forms features all at once Seems like this is what Uniscribe is doing, and does not break any fonts we tested (with Devanagari, Malayalam, Khmer, and Bengali), while fixing some Ra Phala sequences for Bengali with Vrinda. Fixes another 2% of Bengali failures (a couple more to go).
Behdad Esfahbod 6de10354 2012-07-16T22:46:06 [test/arabic] Add Arabic tests for mark skipping Expose a bug with Khaled's Hussaini Nastaleeq font.
Behdad Esfahbod 1167c7bf 2012-07-03T11:38:46 Minor
Behdad Esfahbod aa116582 2012-06-29T10:50:12 Minor
Behdad Esfahbod 5e113a4b 2012-06-16T15:26:13 g_thread_init() is deprecated
Behdad Esfahbod a18280a8 2012-06-07T15:44:12 Fix warnings produced by clang analyzer
Behdad Esfahbod b0a6e58b 2012-06-04T10:21:22 s/script-punjabi/script-gurmukhi/
Behdad Esfahbod 4efdffec 2012-05-27T22:39:48 Minor Malayalam test case From https://bugs.freedesktop.org/show_bug.cgi?id=45166
Behdad Esfahbod dfff5b30 2012-05-27T22:07:04 Add Myanmar test case
Behdad Esfahbod ff3524c2 2012-05-23T21:50:43 Add Arabic diacritics tests
Behdad Esfahbod a6de5366 2012-05-18T15:04:35 Add CJK Compatibility Ideographs tests From: http://people.mozilla.org/~jdaggett/tests/cjkcompat.html
Behdad Esfahbod f538fcb5 2012-05-12T15:34:40 [test] Make tool usage easier by not requiring "--stdin" Just default to it. Added "--help" instead to get usage.
Behdad Esfahbod a3273e30 2012-05-12T13:34:18 [Indic] Add more Malayalam tests
Behdad Esfahbod 5b16de97 2012-05-11T19:55:42 [Indic] Add tests for dottedcircle
Behdad Esfahbod c071b99f 2012-05-11T16:22:46 [Indic] Add test for Left Matra with Halant Uniscribe doesn't move the Halant, we do. And do a broken job of it now.
Behdad Esfahbod b20c9eba 2012-05-10T18:31:17 [Indic] Add test for matra group The spec says: "[{M}+[N]+[H]]", and that's what Uniscribe implements. We instead do: "{M+[N]+[H]}", which means we allow Nukta and Halant after all Matras, not just the last one. It makes more sense.
Behdad Esfahbod 61a58e26 2012-05-10T14:43:53 [Indic] Add tricky reordering test cases In the case of Consonant,LeftMatra,Halant, Uniscribe leaves the Halant where it is, but we want to move it with the Matra as that makes more logical sense.
Behdad Esfahbod 3943293a 2012-05-09T15:27:56 [Indic] Add joiner test cases for Devanagari
Behdad Esfahbod 2214a039 2012-05-09T09:54:54 Add hb-diff-ngrams
Behdad Esfahbod 178e6dce 2012-05-09T08:57:29 Add N-gram generator
Behdad Esfahbod 98669ceb 2012-05-09T08:16:15 Use groupby()
Behdad Esfahbod c438a14b 2012-05-09T07:45:17 Add hb-diff-stat
Behdad Esfahbod 1058d031 2012-05-09T07:30:07 Make hb-diff-filter-failtures retain all test info for failed tests
Behdad Esfahbod f1eb008c 2012-05-08T23:41:41 Add hb-diff-colorize Accepts --format=html now.
Behdad Esfahbod 9155e4ff 2012-05-08T22:44:21 Cleanup diff Doesn't do --color anymore. That will go into a new hb-diff-colorize tool.
Behdad Esfahbod 7d22135b 2012-05-08T19:38:49 Make hb-diff faster
Behdad Esfahbod a93e238e 2012-05-08T18:55:29 More tests
Behdad Esfahbod 585b107c 2012-04-24T16:00:50 Add test caes for a minority language using Bengali U+0985 BENGALI LETTER A followed by U+09D7 BENGALI AU LENGTH MARK. According to Bobby de Vos on the mailing list, this results in a dotted circle with most shaping engines, but is a legitimate sequence in this minority language. We reached the consensus on the list to NOT implement dotted-circle in HarfBuzz.
Behdad Esfahbod 0290bbf8 2012-04-17T10:28:21 Add another Thai test
Behdad Esfahbod 4d85252b 2012-04-16T15:54:26 Add Japanese test data from Adobe's Kazuraki font ligatures
Behdad Esfahbod fe28b997 2012-04-14T19:19:26 Add HB_DIRECTION_IS_VALID
Behdad Esfahbod 4bf90f64 2012-04-12T17:38:23 Make HB_DIRECTION_INVALID be zero This changes all the HB_DIRECTION_* enum member values, but is nicer, in preparation for making hb_segment_properties_t public.
Behdad Esfahbod f9746b60 2012-04-12T09:59:26 Minor
Behdad Esfahbod 7470b0ff 2012-04-12T09:44:27 Add Mongolian test case
Behdad Esfahbod a4976447 2012-04-11T17:48:40 Add Hangul test
Behdad Esfahbod e95d912b 2012-04-11T17:33:02 Fix diff tool
Behdad Esfahbod e099dd65 2012-04-10T10:47:33 Add Thai test case for SARA AM decomposition
Behdad Esfahbod 4450dc93 2012-04-07T22:07:23 Move around
Behdad Esfahbod aaa25d5f 2012-04-05T17:27:23 Add Hangul test case Composed, and decomposed, of the same text.
Behdad Esfahbod 40604498 2012-03-06T20:24:31 Add Hebrew diacritics test cases From: https://bugzilla.mozilla.org/show_bug.cgi?id=662055
Behdad Esfahbod 7a70ca78 2012-02-21T11:31:47 Add test case from https://bugzilla.mozilla.org/show_bug.cgi?id=714067
Behdad Esfahbod 1a5a91dc 2012-01-22T19:57:00 Add a few more tests
Behdad Esfahbod 1795f3a2 2012-01-22T19:29:45 Add a couple Thai test cases from Thep
Behdad Esfahbod ec3f5066 2012-01-22T19:10:55 Add Devanagari test from Tom Hacohen
Behdad Esfahbod 71be4ca3 2012-01-22T16:26:49 Also ignore "ChangeLog" in manifests
Behdad Esfahbod 3c9a39ec 2012-01-22T16:21:19 Remove newline
Behdad Esfahbod e4ccbfe2 2012-01-22T16:07:32 Allow --color=html in hb-diff Not that useful right now as we don't escape < and >. Perhaps another tool can be added to convert the ANSI output to HTML.
Behdad Esfahbod 8f80f934 2012-01-21T20:03:25 More shoveling around
Behdad Esfahbod c78c6e98 2012-01-21T19:55:16 Cleanup
Behdad Esfahbod ab94a9c5 2012-01-21T19:43:58 Distribute testing tools
Behdad Esfahbod 3e86feb5 2012-01-21T19:40:30 Speed up colorless diff
Behdad Esfahbod 1e58df60 2012-01-21T19:37:31 Cleanup manifest code
Behdad Esfahbod 956d552e 2012-01-21T19:31:51 Port hb-manifest-update to Python
Behdad Esfahbod 3a34e9e3 2012-01-21T19:15:41 Ignore Broken Pipe errors
Behdad Esfahbod f22089ac 2012-01-20T21:21:47 Misc fixes
Behdad Esfahbod 96968bfa 2012-01-20T21:16:34 Port hb-manifest-read to Python