|
e8b5d640
|
2014-05-29T15:18:27
|
|
[indic] Do NOT allow reph formation on placeholders
Only allow it on DOTTED CIRCLE. No effect on test numbers.
Test: U+0930,094D,00A0
|
|
|
cf78dd48
|
2014-05-27T17:53:37
|
|
[indic/myanmar] Rename OT_NBSP to OT_PLACEHOLDER
|
|
|
ddbdfcbf
|
2014-05-23T15:39:55
|
|
[indic] Simplify grammar
No functional change.
|
|
|
4e9b1f66
|
2014-05-23T15:38:42
|
|
[indic] Always start new syllable for Avagraha
In fact, the previous grammar was ambigious. No functional
change.
|
|
|
9f9bd9bf
|
2014-05-23T15:33:13
|
|
[indic] Rename avagraha cluster to symbol cluster
In anticipation of adding more characters to that class of clusters.
|
|
|
e848bfae
|
2014-05-22T18:50:34
|
|
[indic] Recategorize U+A8E0..A8F1 as OT_VD
Up to two of them come after all OT_A characters.
|
|
|
c519536c
|
2014-05-22T18:43:14
|
|
[indic] Allow up to three tone marks
According to Roozbeh, there are valid combinations in Unicode
proposals for up to three. Previously we were allowing up to two.
|
|
|
a74f0de2
|
2013-10-24T11:46:33
|
|
[indic] Fix CM2, really
Followup from 6e613f3365bf4e9fd778758c53e7de00c64beca1.
|
|
|
c16012e9
|
2013-10-18T02:27:00
|
|
[indic] Add Javanese support!
Seems to be working just fine!
|
|
|
1a7de1ba
|
2013-10-16T19:55:06
|
|
[indic] Improve Avagraha support in machine
|
|
|
3756efaf
|
2013-10-16T19:06:29
|
|
[indic] Misc harmless fixes!
First, we were abusing OT_VD instead of OT_A. Fix that
but moving OT_A in the grammar where it belongs (which
is different from what the spec says).
Also, only allow medial consonants after all other
consonants. This doesn't affect any current character.
Finally, fix Halant attachment in presence of medial
consonants. Again, this currently doesn't affect any
sequence.
I lied. There's Gurmukhi U+0A75 which is Consonant_Medial.
Uniscribe allows one of those in each of these positions:
before matras, after matras and before syllable modifiers,
and after syllable modifiers! We currently just allow
unlimited numbers of it, before matras.
|
|
|
5c558877
|
2013-10-16T11:14:15
|
|
[indic] Allow up to two syllable modifiers
Bug 70509 - Candrabindu+Visarga doesn't work in Devanagari
https://bugs.freedesktop.org/show_bug.cgi?id=70509
We categorize both bindus and visarga as syllable-modifiers.
OT spec doesn't actually say what characters go in the syllable
modifier category, and allows one. We just allow up to two now.
Test case: U+0930,U+0941,U+0901,U+0903
Uniscribe currently doesn't support that and produces a
dotted circle.
|
|
|
3c7b3641
|
2013-10-15T11:21:01
|
|
[indic] Handle Avagraha
It can come either at the end(ish!) of the syllable, or independently.
When independent, it accepts a few bits and pieces.
|
|
|
e7562f53
|
2013-02-14T11:05:36
|
|
Fix compile warnings for ragel-generated machines
|
|
|
596740db
|
2012-12-21T19:41:04
|
|
[Indic] Insert dottedcircle after a lone Malayalam dot-reph
|
|
|
0beb66e3
|
2012-12-05T18:46:04
|
|
Fix warnings
|
|
|
9cac1338
|
2012-11-12T18:41:22
|
|
[Indic] Allow Consonant_Medial's after Consonant's
Mostly affects Myanmar, but also Tai Tham, Javanese, and Cham. The
latter three are untested (no fonts!).
|
|
|
166b5cf7
|
2012-09-07T14:55:07
|
|
[Indic] Find syllables before any features are applied
With FreeSerif, it seems that the 'ccmp' feature does ligature
substituttions. That was then causing syllable match failures. We now
find syllables before any features have been applied.
Test sequence: U+0D9A,U+0DCA,U+200D,U+0DBB,U+0DCF
|
|
|
f0b8ed1b
|
2012-09-05T17:32:57
|
|
[Indic] Allow "H,ZWJ,M"
Uniscribe accepts a Halant,ZWJ before matras. Allow that.
BENGALI down from 295 to 291
DEVANAGARI down from 69 to 57
GUJARATI down from 19 to 17
KANNADA down from 871 to 867
MALAYALAM down from 340 to 337
TELUGU down from 20 to 16
Currently at:
BENGALI: 353897 out of 354188 tests passed. 291 failed (0.0821598%)
DEVANAGARI: 707337 out of 707394 tests passed. 57 failed (0.00805774%)
GUJARATI: 366440 out of 366457 tests passed. 17 failed (0.00463902%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951046 out of 951913 tests passed. 867 failed (0.0910798%)
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047997 out of 1048334 tests passed. 337 failed (0.0321462%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970557 out of 970573 tests passed. 16 failed (0.00164851%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
|
|
|
4ed717ef
|
2012-09-05T17:21:17
|
|
[Indic] Relax grammar
Now that we insert dotted-circle, tests break more easily when our indic
machine breaks.
In particular, a few Devanagari tests were having sequences like
"C,H,ZWJ,N", and because of the ZWJ the Nukta does NOT get reordered to
before the Halant as the grammar used to expect... Fixup.
Another case is as simple as "C,ZWJ,SM".
Fixes 10 out of 79 failures:
DEVANAGARI: 707325 out of 707394 tests passed. 69 failed (0.00975411%)
|
|
|
aa7141ef
|
2012-09-05T15:54:21
|
|
[Indic] Fix Khmer syllable-final coeng-consonant
Brings down Khmer failures from 162 to 47.
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
Also rebaselined some of the test files that had only-inherited lines.
Removing those, the stats are:
BENGALI: 353893 out of 354188 tests passed. 295 failed (0.0832891%)
DEVANAGARI: 707315 out of 707394 tests passed. 79 failed (0.0111678%)
GUJARATI: 366438 out of 366457 tests passed. 19 failed (0.00518478%)
GURMUKHI: 60704 out of 60747 tests passed. 43 failed (0.0707854%)
KANNADA: 951042 out of 951913 tests passed. 871 failed (0.0915%)
KHMER: 299077 out of 299124 tests passed. 47 failed (0.0157125%)
LAO: 53611 out of 53644 tests passed. 33 failed (0.0615167%)
MALAYALAM: 1047994 out of 1048334 tests passed. 340 failed (0.0324324%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271666 out of 271847 tests passed. 181 failed (0.0665816%)
TAMIL: 1091754 out of 1091754 tests passed. 0 failed (0%)
TELUGU: 970553 out of 970573 tests passed. 20 failed (0.00206064%)
TIBETAN: 208469 out of 208469 tests passed. 0 failed (0%)
Still some regressions, but some of the more egregious cases are
addressed.
|
|
|
b85800f9
|
2012-08-31T18:12:01
|
|
[Indic] Implement dotted-circle insertion for broken clusters
No panic, we reeally insert dotted circle when it's absolutely broken.
Fixes most of the dotted-circle cases against Uniscribe. (for Devanagari
fixes 80% of them, for Khmer 70%; the rest look like Uniscribe being
really bogus...)
I had to make a decision. Apparently Uniscribe adds one dotted circle
to each broken character. I tried that, but that goes wrong easily with
split matras. So I made it add only one dotted circle to an entire
broken syllable tail. As in: "if there was a dotted circle here, this
would have formed a correct cluster." That works better for split
stuff, and I like it more.
|
|
|
327d14ef
|
2012-08-31T16:49:34
|
|
[Indic] Start adding dotted-circle instrastructure
|
|
|
85fc6c48
|
2012-08-02T12:21:44
|
|
[Indic] Move more stuff to the shape_plan
Almost done. Need to add per-script static tables.
|
|
|
8bb5deba
|
2012-08-02T10:07:58
|
|
[OT] Pipe shape_plan down to pause_callbacks
|
|
|
dff0ece1
|
2012-07-24T02:30:38
|
|
[Indic] Limit matras to 4 per syllable
Also limit joiners.
This limits our syllable length to a constant, and is
closer to what Uniscribe does anyway.
Two Devanagari tests regressed, but who cares about tests with 20
joiners in a row?! Devanagari at 57 (0.00821766%) now.
|
|
|
73d71cc5
|
2012-07-24T00:09:12
|
|
[Indic] End Vowel-based syllable at ZWJ
One Devanagari test regressed, plus 10 Malayalam (at 1545 now).
Fixed 120 Sinhala failures. Now at 208 (0.0765136%).
|
|
|
9fa05273
|
2012-07-23T18:19:17
|
|
[Indic] Limit syllables to at most five consonants
Seems to be about what Uniscribe does. Not exactly. But close enough.
More consonants will start a new cluster.
A few scripts went way down in failures. In particular:
- Devanagari failures went down from 490 to 56.
- Telugu went down from 113 to 49.
Other scripts went down slightly or didn't change. New numbers:
BENGALI: 353908 out of 354285 tests passed. 377 failed (0.106412%)
DEVANAGARI: 693572 out of 693628 tests passed. 56 failed (0.00807349%)
GUJARATI: 366485 out of 366506 tests passed. 21 failed (0.00572978%)
GURMUKHI: 60750 out of 60809 tests passed. 59 failed (0.0970251%)
KANNADA: 950730 out of 951913 tests passed. 1183 failed (0.124276%)
KHMER: 298613 out of 299124 tests passed. 511 failed (0.170832%)
MALAYALAM: 1046881 out of 1048416 tests passed. 1535 failed (0.146411%)
ORIYA: 42320 out of 42329 tests passed. 9 failed (0.021262%)
SINHALA: 271333 out of 271847 tests passed. 514 failed (0.189077%)
TAMIL: 1091837 out of 1091837 tests passed. 0 failed (0%)
TELUGU: 970524 out of 970573 tests passed. 49 failed (0.00504856%)
Some of the remaining Telugu and Devanagari issues seem to be Uniscribe
eating Anusvara when placed before a non-joiner. Ouch!
|
|
|
5791f329
|
2012-07-20T16:26:55
|
|
[Indic] Allow a ZWNJ after SM's
Malayalam failures go way down. Other scripts benefitted slightly too.
Sinhala had one or two test regressions, but...
|
|
|
9e4f94a7
|
2012-07-20T13:48:03
|
|
[Indic] Break syllables at Halant,ZWNJ
That's really what Uniscribe does, and explains a lot of pecularities of
Halant,ZWNJ before the base.
Sent Telugu from 1% failures to 0.03%. Improved Kannada and Malayalam
slightly. Fixed half of Bengali, and did NOT break anything!
|
|
|
422ecd2d
|
2012-07-18T23:25:58
|
|
[Indic] Accept a forced Rakar sequence at the end of syllable
In Sinhala, Rakar is formed by Al-Lakuna,ZWJ,Ra. If you put that at the
end of a Consonant,Matra syllable, you get a dotted-circle from
Uniscribe. Apparently adding a ZWJ before the Al-Lakuna "fixes" that.
And people have been encoding that sequence... So, allow a forced
"ZWJ,Virama,ZWJ,Ra" sequence at the of syllables.
Fixes some 100 or more of Sinhala failures. Now at 622 only (0.23%).
|
|
|
6fc17320
|
2012-07-18T17:49:19
|
|
[Indic] Allow joiners on both sides of Halant at the same time
The sequence <ZWJ,Al-Lakuna,ZWJ> is used in Sinhala to explicitly ask
for Rakar. Fixes two-thousand Sinhala tests. Not many left.
|
|
|
552d19b7
|
2012-07-18T16:00:49
|
|
[Indic] Treat Register Shifters like Nukta
Really this time.
Fixes another 18 Khmer tests.
|
|
|
dcb52724
|
2012-07-18T15:28:25
|
|
[Indic] Allow joiners before matras
Fixes 1 more Devanagari test!
|
|
|
391cc033
|
2012-07-18T15:10:05
|
|
[Indic] Allow halant group in Vowel and placeholder syllables
Fixes 2 out of 560 Devanagari failures. AND:
Fixes 1 out of 2 Tamil failures.
|
|
|
ca4e3d3e
|
2012-07-18T15:05:40
|
|
[Indic] Streamline halant/joiner in grammar
|
|
|
418d00df
|
2012-07-18T14:57:28
|
|
[Indic] Minor
|
|
|
4c3691d2
|
2012-07-18T14:23:55
|
|
[Indic] Hopefully minor!
Refactoring Indic machin. No semantic change.
|
|
|
db8981f1
|
2012-07-17T18:17:30
|
|
[Indic] Position Khmer Robat
It's a visual Repha.
Still not positioning logical Repha as occurs in Malayalam.
Another 200 Khmer failures fixed. 547 to go. That's better than
Devanagari!
|
|
|
25bc4894
|
2012-07-17T17:53:03
|
|
[Indic] Better categorize Register Shifters and Khmer Various signs
Down another 500 or so Khmer failures!
|
|
|
34b57149
|
2012-07-17T14:09:32
|
|
[Indic] Treat Khmer Register Shifters more like Nuktas
Except that there may be a ZWNJ before a Register Shifter.
|
|
|
11e2a601
|
2012-07-17T14:02:28
|
|
[Indic] Minor
|
|
|
c50ed71e
|
2012-07-17T11:54:28
|
|
[Indic] Recategorize Khmer coeng sign as a separate category OT_Coeng
Amend the syllable structure to allow a final subscripted consonant
(Coeng+C) and a final subscripted independent vowel (Coeng+V).
Fixes another 2k of Khmer failures.
|
|
|
deb521de
|
2012-07-17T11:37:32
|
|
[Indic] Add a separate Coeng class
No characters recategorized yet. No semantic change.
|
|
|
7d09c98a
|
2012-07-16T16:45:22
|
|
[Indic] Recognizer Register Shifter marks
Fixes another 6% of the Khmer failures.
|
|
|
a98d0ab1
|
2012-07-13T10:19:10
|
|
Make sure HB_BEGIN_DECLS / HB_END_DECLS is only used in public headers
So we can use them to switch default visibility to internal if desired,
and use these to make only declared symbols public.
|
|
|
27aba594
|
2012-05-24T15:00:01
|
|
Minor
|
|
|
18c06e18
|
2012-05-11T20:02:14
|
|
[Indic] Add Uniscribe bug feature for dotted circle
For dotted-circle independent clusters, Uniscribe does no Reph shaping
for the exact sequence Ra+Halant+25CC. Which also is the only possible
sequence with 25CC at the end.
|
|
|
9c099289
|
2012-05-11T18:46:35
|
|
[Indic] Allow multiple Consonants in Vowel/NBSP syllables
Uniscribe allows multiple Halant+Consonant after a Vowel.
Tests:
↦ * U+0905,U+094D,U+092B,U+094D,930,94d,930
|
|
|
8c0aa486
|
2012-05-11T18:13:42
|
|
[Indic] Allow two Nuktas per consonant
Uniscribe allows up to two nuktas per consonant and one per matra. It does so
indepent of whether the consonant already has a nukta in it. Tests:
* U+0916,U+093C,U+0941
* U+0959,U+093C,U+0941
* U+0916,U+093C,U+093C,U+0941
* U+0959,U+093C,U+093C,U+0941
* U+0916,U+093C,U+093C,U+093C,U+0941
* U+0959,U+093C,U+093C,U+093C,U+0941
* 915,93c,93c,,94d,U+0916,U+093C,U+093C,U+093e,93c,93c
|
|
|
3399a06e
|
2012-05-11T17:54:26
|
|
[Indic] Fix U+0952 and similar classification to match Uniscribe
See comments.
|
|
|
ff24d108
|
2012-05-11T17:07:08
|
|
[Indic] Don't use syllable serial value 0
|
|
|
4be46bad
|
2012-05-11T14:39:01
|
|
[Indic] Fix state machine to backtrack
|
|
|
cee71874
|
2012-05-11T11:41:39
|
|
[Indic] Move syllable tracking from Indic to generic layer
This is to incorporate it into GSUB/GPOS processing.
|
|
|
86e5dd38
|
2012-05-09T18:57:37
|
|
[Indic] Don't give up syllable parsing upon junk
|
|
|
ef24cc8c
|
2012-05-09T17:56:03
|
|
[Indic] Towards multi-cluster syllables and final reordering
|
|
|
9ceca3ae
|
2012-04-16T21:05:51
|
|
Fix ragel regexp in vowel-based syllable
As reported by datao zhang on the mailing list.
|
|
|
b870afcd
|
2012-04-16T21:05:11
|
|
Rewrite ragel expression to better match the one on MS spec
https://www.microsoft.com/typography/otfntdev/devanot/shaping.aspx
|
|
|
d4cc4471
|
2012-04-07T21:52:28
|
|
Move code around, in prep for Thai/Lao shaper
|
|
|
461b9b63
|
2012-03-01T18:11:19
|
|
Fix cluster formation in Indic
Makes number of failures against Uniscribe with hi_IN dictionary from
OO.o to go down from 6334 to 4290. Not bad for a one-line change!
Mozilla Bug 729626 - ASAN: heap-buffer-overflow HTML
|
|
|
743807a3
|
2011-07-29T16:37:02
|
|
[Indic] Apply Indic features
Find the base consonant and apply basic Indic features accordingly.
Nothing complete, but does something for now. Specifically:
no Ra handling right now, and no ZWJ/ZWNJ.
Number of failing shape-complex tests goes from 174 down to 125.
Next: reorder matras.
|
|
|
76f76812
|
2011-07-07T22:25:25
|
|
Shuffle code around, remove shape_plan from complex shapers
|
|
|
d69d5cea
|
2011-07-04T12:56:38
|
|
[Indic] Well, at least finding syllables works now :)
Still not much there.
|
|
|
c7fe56a1
|
2011-06-24T19:05:34
|
|
[Indic] Some of the basic features are global; Mark them so
|
|
|
867361c3
|
2011-06-17T18:35:46
|
|
[indic] Add syllable recognition state machine
Using an incredible tool called Ragel.
|